Adding the MlNetMklDeps package, adding the entry point, tests and docs for OLS.#578

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
sfilipi wants to merge 9 commits into from
Closed

Adding the MlNetMklDeps package, adding the entry point, tests and docs for OLS. #578

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
9e96175
WIP: adding the MlNetMklDeps package
sfilipi Jul 24, 2018
fa519e8
Fixing the MlNetMklDeps nukpproj
sfilipi Jul 26, 2018
19779fe
reverting test changes to the CSharpCodeGen
sfilipi Jul 26, 2018
94ff6d3
Adding the LoadableClass assembly attribute with the signature:Signat…
sfilipi Jul 26, 2018
dfc2b35
merging from master
sfilipi Jul 26, 2018
cd18b74
following the same naming convention for the doc attributes in tags
sfilipi Jul 27, 2018
80f7bd6
Fixing the path in the OLS doc include
sfilipi Jul 27, 2018
f910923
removing unecessary packing
sfilipi Jul 27, 2018
2965477
In Mac, release only, one of the OLS tests is failing because the act…
sfilipi Jul 27, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions build/Dependencies.props
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -8,5 +8,6 @@
<SystemReflectionEmitLightweightPackageVersion>4.3.0</SystemReflectionEmitLightweightPackageVersion>
<PublishSymbolsPackageVersion>1.0.0-beta-62824-02</PublishSymbolsPackageVersion>
<LightGBMPackageVersion>2.1.2.2</LightGBMPackageVersion>
<MlNetMklDepsPackageVersion>0.0.0.1</MlNetMklDepsPackageVersion>
</PropertyGroup>
</Project>
2 changes: 2 additions & 0 deletions src/Microsoft.ML.StandardLearners/Microsoft.ML.StandardLearners.csproj
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -11,6 +11,8 @@
<ProjectReference Include="..\Microsoft.ML.CpuMath\Microsoft.ML.CpuMath.csproj" />
<ProjectReference Include="..\Microsoft.ML.Data\Microsoft.ML.Data.csproj" />
<ProjectReference Include="..\Microsoft.ML\Microsoft.ML.csproj" />

<PackageReference Include="MlNetMklDeps" Version="$(MlNetMklDepsPackageVersion)" />
</ItemGroup>

</Project>
26 changes: 21 additions & 5 deletions src/Microsoft.ML.StandardLearners/Standard/OlsLinearRegression.cs
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -28,8 +28,11 @@
"OLS Linear Regression Executor",
OlsLinearRegressionPredictor.LoaderSignature)]

[assembly: LoadableClass(typeof(void), typeof(OlsLinearRegressionTrainer), null, typeof(SignatureEntryPointModule), OlsLinearRegressionTrainer.LoadNameValue)]

namespace Microsoft.ML.Runtime.Learners
{
/// <include file='doc.xml' path='doc/members/member[@name="OLS"]/*' />
public sealed class OlsLinearRegressionTrainer : TrainerBase<OlsLinearRegressionPredictor>
{
public sealed class Arguments : LearnerInputBaseWithWeight
Expand All@@ -51,11 +54,6 @@ public sealed class Arguments : LearnerInputBaseWithWeight
public const string ShortName = "ols";
internal const string Summary = "The ordinary least square regression fits the target function as a linear function of the numerical features "
+ "that minimizes the square loss function.";
internal const string Remarks = @"<remarks>
<a href='https://en.wikipedia.org/wiki/Ordinary_least_squares'>Ordinary least squares (OLS)</a> is a parameterized regression method.
It assumes that the conditional mean of the dependent variable follows a linear function of the dependent variables.
By minimizing the squares of the difference between observed values and the predictions, the parameters of the regressor can be estimated.
</remarks>";

private readonly Float _l2Weight;
private readonly bool _perParameterSignificance;
Expand DownExpand Up@@ -463,6 +461,24 @@ public static void Pptri(Layout layout, UpLo uplo, int n, Double[] ap)
}
}
}

[TlcModule.EntryPoint(Name = "Trainers.OrdinaryLeastSquaresRegressor",
Desc = "Train an OLS regression model.",
UserName = UserNameValue,
ShortName = ShortName,
XmlInclude = new[] { @"<include file='../Microsoft.ML.StandardLearners/Standard/doc.xml' path='doc/members/member[@name=""OLS""]/*' />" })]
public static CommonOutputs.RegressionOutput TrainRegression(IHostEnvironment env, Arguments input)
{
Contracts.CheckValue(env, nameof(env));
var host = env.Register("TrainOLS");
host.CheckValue(input, nameof(input));
EntryPointUtils.CheckInputArgs(host, input);

return LearnerEntryPointsUtils.Train<Arguments, CommonOutputs.RegressionOutput>(host, input,
() => new OlsLinearRegressionTrainer(host, input),
() => LearnerEntryPointsUtils.FindColumn(host, input.TrainingData.Schema, input.LabelColumn),
() => LearnerEntryPointsUtils.FindColumn(host, input.TrainingData.Schema, input.WeightColumn));
}
}

/// <summary>
Expand Down
21 changes: 21 additions & 0 deletions src/Microsoft.ML.StandardLearners/Standard/doc.xml
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -68,6 +68,27 @@
</code>
</example>
</example>

<member name="OLS">
<summary>
Train an OLS regression model.
</summary>
<remarks>
<a href='https://en.wikipedia.org/wiki/Ordinary_least_squares'>Ordinary least squares (OLS)</a> is a parameterized regression method.
It assumes that the conditional mean of the dependent variable follows a linear function of the dependent variables.
The parameters of the regressor can be estimated by minimizing the squares of the difference between observed values and the predictions.
</remarks>
<example>
<code language="csharp">
new OrdinaryLeastSquaresRegressor
{
L2Weight = 0.1,
PerParameterSignificance = false,
NormalizeFeatures = Microsoft.ML.Models.NormalizeOption.Yes
}
</code>
</example>
</member>

</members>
</doc>
99 changes: 99 additions & 0 deletions src/Microsoft.ML/CSharpApi.cs
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -754,6 +754,18 @@ public void Add(Microsoft.ML.Trainers.OnlineGradientDescentRegressor input, Micr
_jsonNodes.Add(Serialize("Trainers.OnlineGradientDescentRegressor", input, output));
}

public Microsoft.ML.Trainers.OrdinaryLeastSquaresRegressor.Output Add(Microsoft.ML.Trainers.OrdinaryLeastSquaresRegressor input)
{
var output = new Microsoft.ML.Trainers.OrdinaryLeastSquaresRegressor.Output();
Add(input, output);
return output;
}

public void Add(Microsoft.ML.Trainers.OrdinaryLeastSquaresRegressor input, Microsoft.ML.Trainers.OrdinaryLeastSquaresRegressor.Output output)
{
_jsonNodes.Add(Serialize("Trainers.OrdinaryLeastSquaresRegressor", input, output));
}

public Microsoft.ML.Trainers.PcaAnomalyDetector.Output Add(Microsoft.ML.Trainers.PcaAnomalyDetector input)
{
var output = new Microsoft.ML.Trainers.PcaAnomalyDetector.Output();
Expand DownExpand Up@@ -8824,6 +8836,93 @@ public OnlineGradientDescentRegressorPipelineStep(Output output)
}
}

namespace Trainers
{

/// <include file='../Microsoft.ML.StandardLearners/Standard/doc.xml' path='doc/members/member[@name="OLS"]/*' />
public sealed partial class OrdinaryLeastSquaresRegressor : Microsoft.ML.Runtime.EntryPoints.CommonInputs.ITrainerInputWithWeight, Microsoft.ML.Runtime.EntryPoints.CommonInputs.ITrainerInputWithLabel, Microsoft.ML.Runtime.EntryPoints.CommonInputs.ITrainerInput, Microsoft.ML.ILearningPipelineItem
{


/// <summary>
/// L2 regularization weight
/// </summary>
[TlcModule.SweepableDiscreteParamAttribute("L2Weight", new object[]{1E-06f, 0.1f, 1f})]
public float L2Weight { get; set; } = 1E-06f;

/// <summary>
/// Whether to calculate per parameter significance statistics
/// </summary>
public bool PerParameterSignificance { get; set; } = true;

/// <summary>
/// Column to use for example weight
/// </summary>
public Microsoft.ML.Runtime.EntryPoints.Optional<string> WeightColumn { get; set; }

/// <summary>
/// Column to use for labels
/// </summary>
public string LabelColumn { get; set; } = "Label";

/// <summary>
/// The data to be used for training
/// </summary>
public Var<Microsoft.ML.Runtime.Data.IDataView> TrainingData { get; set; } = new Var<Microsoft.ML.Runtime.Data.IDataView>();

/// <summary>
/// Column to use for features
/// </summary>
public string FeatureColumn { get; set; } = "Features";

/// <summary>
/// Normalize option for the feature column
/// </summary>
public Microsoft.ML.Models.NormalizeOption NormalizeFeatures { get; set; } = Microsoft.ML.Models.NormalizeOption.Auto;

/// <summary>
/// Whether learner should cache input training data
/// </summary>
public Microsoft.ML.Models.CachingOptions Caching { get; set; } = Microsoft.ML.Models.CachingOptions.Auto;


public sealed class Output : Microsoft.ML.Runtime.EntryPoints.CommonOutputs.IRegressionOutput, Microsoft.ML.Runtime.EntryPoints.CommonOutputs.ITrainerOutput
{
/// <summary>
/// The trained model
/// </summary>
public Var<Microsoft.ML.Runtime.EntryPoints.IPredictorModel> PredictorModel { get; set; } = new Var<Microsoft.ML.Runtime.EntryPoints.IPredictorModel>();

}
public Var<IDataView> GetInputData() => TrainingData;

public ILearningPipelineStep ApplyStep(ILearningPipelineStep previousStep, Experiment experiment)
{
if (previousStep != null)
{
if (!(previousStep is ILearningPipelineDataStep dataStep))
{
throw new InvalidOperationException($"{ nameof(OrdinaryLeastSquaresRegressor)} only supports an { nameof(ILearningPipelineDataStep)} as an input.");
}

TrainingData = dataStep.Data;
}
Output output = experiment.Add(this);
return new OrdinaryLeastSquaresRegressorPipelineStep(output);
}

private class OrdinaryLeastSquaresRegressorPipelineStep : ILearningPipelinePredictorStep
{
public OrdinaryLeastSquaresRegressorPipelineStep(Output output)
{
Model = output.PredictorModel;
}

public Var<IPredictorModel> Model { get; }
}
}
}

namespace Trainers
{

Expand Down
1 change: 1 addition & 0 deletions test/BaselineOutput/Common/EntryPoints/core_ep-list.tsv
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -59,6 +59,7 @@ Trainers.LogisticRegressionBinaryClassifier Logistic Regression is a method in s
Trainers.LogisticRegressionClassifier Logistic Regression is a method in statistics used to predict the probability of occurrence of an event and can be used as a classification algorithm. The algorithm predicts the probability of occurrence of an event by fitting data to a logistical function. Microsoft.ML.Runtime.Learners.LogisticRegression TrainMultiClass Microsoft.ML.Runtime.Learners.MulticlassLogisticRegression+Arguments Microsoft.ML.Runtime.EntryPoints.CommonOutputs+MulticlassClassificationOutput
Trainers.NaiveBayesClassifier Train a MultiClassNaiveBayesTrainer. Microsoft.ML.Runtime.Learners.MultiClassNaiveBayesTrainer TrainMultiClassNaiveBayesTrainer Microsoft.ML.Runtime.Learners.MultiClassNaiveBayesTrainer+Arguments Microsoft.ML.Runtime.EntryPoints.CommonOutputs+MulticlassClassificationOutput
Trainers.OnlineGradientDescentRegressor Train a Online gradient descent perceptron. Microsoft.ML.Runtime.Learners.OnlineGradientDescentTrainer TrainRegression Microsoft.ML.Runtime.Learners.OnlineGradientDescentTrainer+Arguments Microsoft.ML.Runtime.EntryPoints.CommonOutputs+RegressionOutput
Trainers.OrdinaryLeastSquaresRegressor Train an OLS regression model. Microsoft.ML.Runtime.Learners.OlsLinearRegressionTrainer TrainRegression Microsoft.ML.Runtime.Learners.OlsLinearRegressionTrainer+Arguments Microsoft.ML.Runtime.EntryPoints.CommonOutputs+RegressionOutput
Trainers.PcaAnomalyDetector Train an PCA Anomaly model. Microsoft.ML.Runtime.PCA.RandomizedPcaTrainer TrainPcaAnomaly Microsoft.ML.Runtime.PCA.RandomizedPcaTrainer+Arguments Microsoft.ML.Runtime.EntryPoints.CommonOutputs+AnomalyDetectionOutput
Trainers.PoissonRegressor Train an Poisson regression model. Microsoft.ML.Runtime.Learners.PoissonRegression TrainRegression Microsoft.ML.Runtime.Learners.PoissonRegression+Arguments Microsoft.ML.Runtime.EntryPoints.CommonOutputs+RegressionOutput
Trainers.StochasticDualCoordinateAscentBinaryClassifier Train an SDCA binary model. Microsoft.ML.Runtime.Learners.Sdca TrainBinary Microsoft.ML.Runtime.Learners.LinearClassificationTrainer+Arguments Microsoft.ML.Runtime.EntryPoints.CommonOutputs+BinaryClassificationOutput
Expand Down
143 changes: 143 additions & 0 deletions test/BaselineOutput/Common/EntryPoints/core_manifest.json
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -14050,6 +14050,149 @@
"ITrainerOutput"
]
},
{
"Name": "Trainers.OrdinaryLeastSquaresRegressor",
"Desc": "Train an OLS regression model.",
"FriendlyName": "Ordinary Least Squares (Regression)",
"ShortName": "ols",
"Inputs": [
{
"Name": "TrainingData",
"Type": "DataView",
"Desc": "The data to be used for training",
"Aliases": [
"data"
],
"Required": true,
"SortOrder": 1.0,
"IsNullable": false
},
{
"Name": "FeatureColumn",
"Type": "String",
"Desc": "Column to use for features",
"Aliases": [
"feat"
],
"Required": false,
"SortOrder": 2.0,
"IsNullable": false,
"Default": "Features"
},
{
"Name": "LabelColumn",
"Type": "String",
"Desc": "Column to use for labels",
"Aliases": [
"lab"
],
"Required": false,
"SortOrder": 3.0,
"IsNullable": false,
"Default": "Label"
},
{
"Name": "WeightColumn",
"Type": "String",
"Desc": "Column to use for example weight",
"Aliases": [
"weight"
],
"Required": false,
"SortOrder": 4.0,
"IsNullable": false,
"Default": "Weight"
},
{
"Name": "NormalizeFeatures",
"Type": {
"Kind": "Enum",
"Values": [
"No",
"Warn",
"Auto",
"Yes"
]
},
"Desc": "Normalize option for the feature column",
"Aliases": [
"norm"
],
"Required": false,
"SortOrder": 5.0,
"IsNullable": false,
"Default": "Auto"
},
{
"Name": "Caching",
"Type": {
"Kind": "Enum",
"Values": [
"Auto",
"Memory",
"Disk",
"None"
]
},
"Desc": "Whether learner should cache input training data",
"Aliases": [
"cache"
],
"Required": false,
"SortOrder": 6.0,
"IsNullable": false,
"Default": "Auto"
},
{
"Name": "L2Weight",
"Type": "Float",
"Desc": "L2 regularization weight",
"Aliases": [
"l2"
],
"Required": false,
"SortOrder": 50.0,
"IsNullable": false,
"Default": 1E-06,
"SweepRange": {
"RangeType": "Discrete",
"Values": [
1E-06,
0.1,
1.0
]
}
},
{
"Name": "PerParameterSignificance",
"Type": "Bool",
"Desc": "Whether to calculate per parameter significance statistics",
"Aliases": [
"sig"
],
"Required": false,
"SortOrder": 150.0,
"IsNullable": false,
"Default": true
}
],
"Outputs": [
{
"Name": "PredictorModel",
"Type": "PredictorModel",
"Desc": "The trained model"
}
],
"InputKind": [
"ITrainerInputWithWeight",
"ITrainerInputWithLabel",
"ITrainerInput"
],
"OutputKind": [
"IRegressionOutput",
"ITrainerOutput"
]
},
{
"Name": "Trainers.PcaAnomalyDetector",
"Desc": "Train an PCA Anomaly model.",
Expand Down
33 changes: 33 additions & 0 deletions test/BaselineOutput/SingleDebug/OLS/OLS-CV-wine-out.txt
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
maml.exe CV tr=OLS threads=- norm=No dout=%Output% loader=Text{col=Label:R4:11 col=Features:R4:0-10 sep=; header+} data=%Data% seed=1
Not adding a normalizer.
Trainer solving for 12 parameters across 2409 examples
Coefficient of determination R2 = 0.291173667189042, or 0.287920813763543 (adjusted)
Not training a calibrator because it is not needed.
Not adding a normalizer.
Trainer solving for 12 parameters across 2489 examples
Coefficient of determination R2 = 0.280280855195625, or 0.277084686203761 (adjusted)
Not training a calibrator because it is not needed.
L1(avg): 0.586798
L2(avg): 0.573048
RMS(avg): 0.756999
Loss-fn(avg): 0.573048
R Squared: 0.263841
L1(avg): 0.587999
L2(avg): 0.571859
RMS(avg): 0.756214
Loss-fn(avg): 0.571859
R Squared: 0.276072

OVERALL RESULTS
---------------------------------------
L1(avg): 0.587398 (0.0006)
L2(avg): 0.572454 (0.0006)
RMS(avg): 0.756606 (0.0004)
Loss-fn(avg): 0.572454 (0.0006)
R Squared: 0.269956 (0.0061)

---------------------------------------
Physical memory usage(MB): %Number%
Virtual memory usage(MB): %Number%
%DateTime% Time elapsed(s): %Number%

Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all 
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '

Skip to content
Adding the MlNetMklDeps package, adding the entry point, tests and docs for OLS.#578

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Closed

sfilipi
wants to merge
9
commits into

from

 Closed


Adding the MlNetMklDeps package, adding the entry point, tests and docs for OLS.
#578
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


9 commits


Select commit
Hold shift + click to select a range
9e96175

WIP: adding the MlNetMklDeps package


sfilipi Jul 24, 2018
fa519e8

Fixing the MlNetMklDeps nukpproj


sfilipi Jul 26, 2018
19779fe

reverting test changes to the CSharpCodeGen


sfilipi Jul 26, 2018
94ff6d3

Adding the LoadableClass assembly attribute with the signature:Signat…


sfilipi Jul 26, 2018
dfc2b35

merging from master


sfilipi Jul 26, 2018
cd18b74

following the same naming convention for the doc attributes in tags


sfilipi Jul 27, 2018
80f7bd6

Fixing the path in the OLS doc include


sfilipi Jul 27, 2018
f910923

removing unecessary packing


sfilipi Jul 27, 2018
2965477

In Mac, release only, one of the OLS tests is failing because the act…


sfilipi Jul 27, 2018

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


1 change: 1 addition & 0 deletions
build/Dependencies.props

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -8,5 +8,6 @@
 <SystemReflectionEmitLightweightPackageVersion>4.3.0</SystemReflectionEmitLightweightPackageVersion>
 <PublishSymbolsPackageVersion>1.0.0-beta-62824-02</PublishSymbolsPackageVersion>
 <LightGBMPackageVersion>2.1.2.2</LightGBMPackageVersion>
 <MlNetMklDepsPackageVersion>0.0.0.1</MlNetMklDepsPackageVersion>
 </PropertyGroup>
</Project>

2 changes: 2 additions & 0 deletions
src/Microsoft.ML.StandardLearners/Microsoft.ML.StandardLearners.csproj

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -11,6 +11,8 @@
 <ProjectReference Include="..\Microsoft.ML.CpuMath\Microsoft.ML.CpuMath.csproj" />
 <ProjectReference Include="..\Microsoft.ML.Data\Microsoft.ML.Data.csproj" />
 <ProjectReference Include="..\Microsoft.ML\Microsoft.ML.csproj" />

	<PackageReference Include="MlNetMklDeps" Version="$(MlNetMklDepsPackageVersion)" />
 </ItemGroup>

</Project>

26 changes: 21 additions & 5 deletions
src/Microsoft.ML.StandardLearners/Standard/OlsLinearRegression.cs

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -28,8 +28,11 @@
 "OLS Linear Regression Executor",
 OlsLinearRegressionPredictor.LoaderSignature)]

[assembly: LoadableClass(typeof(void), typeof(OlsLinearRegressionTrainer), null, typeof(SignatureEntryPointModule), OlsLinearRegressionTrainer.LoadNameValue)]

namespace Microsoft.ML.Runtime.Learners
{
 /// <include file='doc.xml' path='doc/members/member[@name="OLS"]/*' />
 public sealed class OlsLinearRegressionTrainer : TrainerBase<OlsLinearRegressionPredictor>
 {
 public sealed class Arguments : LearnerInputBaseWithWeight
Expand All@@ -51,11 +54,6 @@ public sealed class Arguments : LearnerInputBaseWithWeight
 public const string ShortName = "ols";
 internal const string Summary = "The ordinary least square regression fits the target function as a linear function of the numerical features "
 + "that minimizes the square loss function.";
 internal const string Remarks = @"<remarks>
<a href='https://en.wikipedia.org/wiki/Ordinary_least_squares'>Ordinary least squares (OLS)</a> is a parameterized regression method. 
It assumes that the conditional mean of the dependent variable follows a linear function of the dependent variables.
By minimizing the squares of the difference between observed values and the predictions, the parameters of the regressor can be estimated.
</remarks>";

 private readonly Float _l2Weight;
 private readonly bool _perParameterSignificance;
Expand DownExpand Up@@ -463,6 +461,24 @@ public static void Pptri(Layout layout, UpLo uplo, int n, Double[] ap)
 }
 }
 }

 [TlcModule.EntryPoint(Name = "Trainers.OrdinaryLeastSquaresRegressor",
 Desc = "Train an OLS regression model.",
 UserName = UserNameValue,
 ShortName = ShortName,
 XmlInclude = new[] { @"<include file='../Microsoft.ML.StandardLearners/Standard/doc.xml' path='doc/members/member[@name=""OLS""]/*' />" })]
 public static CommonOutputs.RegressionOutput TrainRegression(IHostEnvironment env, Arguments input)
 {
 Contracts.CheckValue(env, nameof(env));
 var host = env.Register("TrainOLS");
 host.CheckValue(input, nameof(input));
 EntryPointUtils.CheckInputArgs(host, input);

 return LearnerEntryPointsUtils.Train<Arguments, CommonOutputs.RegressionOutput>(host, input,
 () => new OlsLinearRegressionTrainer(host, input),
 () => LearnerEntryPointsUtils.FindColumn(host, input.TrainingData.Schema, input.LabelColumn),
 () => LearnerEntryPointsUtils.FindColumn(host, input.TrainingData.Schema, input.WeightColumn));
 }
 }

 /// <summary>
Expand Down

21 changes: 21 additions & 0 deletions
src/Microsoft.ML.StandardLearners/Standard/doc.xml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -68,6 +68,27 @@
 </code>
 </example>
 </example>

 <member name="OLS">
 <summary>
 Train an OLS regression model.
 </summary>
 <remarks>
 <a href='https://en.wikipedia.org/wiki/Ordinary_least_squares'>Ordinary least squares (OLS)</a> is a parameterized regression method. 
 It assumes that the conditional mean of the dependent variable follows a linear function of the dependent variables.
 The parameters of the regressor can be estimated by minimizing the squares of the difference between observed values and the predictions.
 </remarks>
 <example>
 <code language="csharp">
 new OrdinaryLeastSquaresRegressor
 {
 L2Weight = 0.1,
 PerParameterSignificance = false,
 NormalizeFeatures = Microsoft.ML.Models.NormalizeOption.Yes
 }
 </code>
 </example>
 </member>

 </members>
</doc>

99 changes: 99 additions & 0 deletions
src/Microsoft.ML/CSharpApi.cs

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -754,6 +754,18 @@ public void Add(Microsoft.ML.Trainers.OnlineGradientDescentRegressor input, Micr
 _jsonNodes.Add(Serialize("Trainers.OnlineGradientDescentRegressor", input, output));
 }

 public Microsoft.ML.Trainers.OrdinaryLeastSquaresRegressor.Output Add(Microsoft.ML.Trainers.OrdinaryLeastSquaresRegressor input)
 {
 var output = new Microsoft.ML.Trainers.OrdinaryLeastSquaresRegressor.Output();
 Add(input, output);
 return output;
 }

 public void Add(Microsoft.ML.Trainers.OrdinaryLeastSquaresRegressor input, Microsoft.ML.Trainers.OrdinaryLeastSquaresRegressor.Output output)
 {
 _jsonNodes.Add(Serialize("Trainers.OrdinaryLeastSquaresRegressor", input, output));
 }

 public Microsoft.ML.Trainers.PcaAnomalyDetector.Output Add(Microsoft.ML.Trainers.PcaAnomalyDetector input)
 {
 var output = new Microsoft.ML.Trainers.PcaAnomalyDetector.Output();
Expand DownExpand Up@@ -8824,6 +8836,93 @@ public OnlineGradientDescentRegressorPipelineStep(Output output)
 }
 }

 namespace Trainers
 {

 /// <include file='../Microsoft.ML.StandardLearners/Standard/doc.xml' path='doc/members/member[@name="OLS"]/*' />
 public sealed partial class OrdinaryLeastSquaresRegressor : Microsoft.ML.Runtime.EntryPoints.CommonInputs.ITrainerInputWithWeight, Microsoft.ML.Runtime.EntryPoints.CommonInputs.ITrainerInputWithLabel, Microsoft.ML.Runtime.EntryPoints.CommonInputs.ITrainerInput, Microsoft.ML.ILearningPipelineItem
 {


 /// <summary>
 /// L2 regularization weight
 /// </summary>
 [TlcModule.SweepableDiscreteParamAttribute("L2Weight", new object[]{1E-06f, 0.1f, 1f})]
 public float L2Weight { get; set; } = 1E-06f;

 /// <summary>
 /// Whether to calculate per parameter significance statistics
 /// </summary>
 public bool PerParameterSignificance { get; set; } = true;

 /// <summary>
 /// Column to use for example weight
 /// </summary>
 public Microsoft.ML.Runtime.EntryPoints.Optional<string> WeightColumn { get; set; }

 /// <summary>
 /// Column to use for labels
 /// </summary>
 public string LabelColumn { get; set; } = "Label";

 /// <summary>
 /// The data to be used for training
 /// </summary>
 public Var<Microsoft.ML.Runtime.Data.IDataView> TrainingData { get; set; } = new Var<Microsoft.ML.Runtime.Data.IDataView>();

 /// <summary>
 /// Column to use for features
 /// </summary>
 public string FeatureColumn { get; set; } = "Features";

 /// <summary>
 /// Normalize option for the feature column
 /// </summary>
 public Microsoft.ML.Models.NormalizeOption NormalizeFeatures { get; set; } = Microsoft.ML.Models.NormalizeOption.Auto;

 /// <summary>
 /// Whether learner should cache input training data
 /// </summary>
 public Microsoft.ML.Models.CachingOptions Caching { get; set; } = Microsoft.ML.Models.CachingOptions.Auto;


 public sealed class Output : Microsoft.ML.Runtime.EntryPoints.CommonOutputs.IRegressionOutput, Microsoft.ML.Runtime.EntryPoints.CommonOutputs.ITrainerOutput
 {
 /// <summary>
 /// The trained model
 /// </summary>
 public Var<Microsoft.ML.Runtime.EntryPoints.IPredictorModel> PredictorModel { get; set; } = new Var<Microsoft.ML.Runtime.EntryPoints.IPredictorModel>();

 }
 public Var<IDataView> GetInputData() => TrainingData;

 public ILearningPipelineStep ApplyStep(ILearningPipelineStep previousStep, Experiment experiment)
 {
 if (previousStep != null)
 {
 if (!(previousStep is ILearningPipelineDataStep dataStep))
 {
 throw new InvalidOperationException($"{ nameof(OrdinaryLeastSquaresRegressor)} only supports an { nameof(ILearningPipelineDataStep)} as an input.");
 }

 TrainingData = dataStep.Data;
 }
 Output output = experiment.Add(this);
 return new OrdinaryLeastSquaresRegressorPipelineStep(output);
 }

 private class OrdinaryLeastSquaresRegressorPipelineStep : ILearningPipelinePredictorStep
 {
 public OrdinaryLeastSquaresRegressorPipelineStep(Output output)
 {
 Model = output.PredictorModel;
 }

 public Var<IPredictorModel> Model { get; }
 }
 }
 }

 namespace Trainers
 {

Expand Down

1 change: 1 addition & 0 deletions
test/BaselineOutput/Common/EntryPoints/core_ep-list.tsv

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -59,6 +59,7 @@ Trainers.LogisticRegressionBinaryClassifier	Logistic Regression is a method in s
Trainers.LogisticRegressionClassifier	Logistic Regression is a method in statistics used to predict the probability of occurrence of an event and can be used as a classification algorithm. The algorithm predicts the probability of occurrence of an event by fitting data to a logistical function.	Microsoft.ML.Runtime.Learners.LogisticRegression	TrainMultiClass	Microsoft.ML.Runtime.Learners.MulticlassLogisticRegression+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+MulticlassClassificationOutput
Trainers.NaiveBayesClassifier	Train a MultiClassNaiveBayesTrainer.	Microsoft.ML.Runtime.Learners.MultiClassNaiveBayesTrainer	TrainMultiClassNaiveBayesTrainer	Microsoft.ML.Runtime.Learners.MultiClassNaiveBayesTrainer+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+MulticlassClassificationOutput
Trainers.OnlineGradientDescentRegressor	Train a Online gradient descent perceptron.	Microsoft.ML.Runtime.Learners.OnlineGradientDescentTrainer	TrainRegression	Microsoft.ML.Runtime.Learners.OnlineGradientDescentTrainer+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+RegressionOutput
Trainers.OrdinaryLeastSquaresRegressor	Train an OLS regression model.	Microsoft.ML.Runtime.Learners.OlsLinearRegressionTrainer	TrainRegression	Microsoft.ML.Runtime.Learners.OlsLinearRegressionTrainer+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+RegressionOutput
Trainers.PcaAnomalyDetector	Train an PCA Anomaly model.	Microsoft.ML.Runtime.PCA.RandomizedPcaTrainer	TrainPcaAnomaly	Microsoft.ML.Runtime.PCA.RandomizedPcaTrainer+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+AnomalyDetectionOutput
Trainers.PoissonRegressor	Train an Poisson regression model.	Microsoft.ML.Runtime.Learners.PoissonRegression	TrainRegression	Microsoft.ML.Runtime.Learners.PoissonRegression+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+RegressionOutput
Trainers.StochasticDualCoordinateAscentBinaryClassifier	Train an SDCA binary model.	Microsoft.ML.Runtime.Learners.Sdca	TrainBinary	Microsoft.ML.Runtime.Learners.LinearClassificationTrainer+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+BinaryClassificationOutput
Expand Down

143 changes: 143 additions & 0 deletions
test/BaselineOutput/Common/EntryPoints/core_manifest.json

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -14050,6 +14050,149 @@
 "ITrainerOutput"
 ]
 },
 {
 "Name": "Trainers.OrdinaryLeastSquaresRegressor",
 "Desc": "Train an OLS regression model.",
 "FriendlyName": "Ordinary Least Squares (Regression)",
 "ShortName": "ols",
 "Inputs": [
 {
 "Name": "TrainingData",
 "Type": "DataView",
 "Desc": "The data to be used for training",
 "Aliases": [
 "data"
 ],
 "Required": true,
 "SortOrder": 1.0,
 "IsNullable": false
 },
 {
 "Name": "FeatureColumn",
 "Type": "String",
 "Desc": "Column to use for features",
 "Aliases": [
 "feat"
 ],
 "Required": false,
 "SortOrder": 2.0,
 "IsNullable": false,
 "Default": "Features"
 },
 {
 "Name": "LabelColumn",
 "Type": "String",
 "Desc": "Column to use for labels",
 "Aliases": [
 "lab"
 ],
 "Required": false,
 "SortOrder": 3.0,
 "IsNullable": false,
 "Default": "Label"
 },
 {
 "Name": "WeightColumn",
 "Type": "String",
 "Desc": "Column to use for example weight",
 "Aliases": [
 "weight"
 ],
 "Required": false,
 "SortOrder": 4.0,
 "IsNullable": false,
 "Default": "Weight"
 },
 {
 "Name": "NormalizeFeatures",
 "Type": {
 "Kind": "Enum",
 "Values": [
 "No",
 "Warn",
 "Auto",
 "Yes"
 ]
 },
 "Desc": "Normalize option for the feature column",
 "Aliases": [
 "norm"
 ],
 "Required": false,
 "SortOrder": 5.0,
 "IsNullable": false,
 "Default": "Auto"
 },
 {
 "Name": "Caching",
 "Type": {
 "Kind": "Enum",
 "Values": [
 "Auto",
 "Memory",
 "Disk",
 "None"
 ]
 },
 "Desc": "Whether learner should cache input training data",
 "Aliases": [
 "cache"
 ],
 "Required": false,
 "SortOrder": 6.0,
 "IsNullable": false,
 "Default": "Auto"
 },
 {
 "Name": "L2Weight",
 "Type": "Float",
 "Desc": "L2 regularization weight",
 "Aliases": [
 "l2"
 ],
 "Required": false,
 "SortOrder": 50.0,
 "IsNullable": false,
 "Default": 1E-06,
 "SweepRange": {
 "RangeType": "Discrete",
 "Values": [
 1E-06,
 0.1,
 1.0
 ]
 }
 },
 {
 "Name": "PerParameterSignificance",
 "Type": "Bool",
 "Desc": "Whether to calculate per parameter significance statistics",
 "Aliases": [
 "sig"
 ],
 "Required": false,
 "SortOrder": 150.0,
 "IsNullable": false,
 "Default": true
 }
 ],
 "Outputs": [
 {
 "Name": "PredictorModel",
 "Type": "PredictorModel",
 "Desc": "The trained model"
 }
 ],
 "InputKind": [
 "ITrainerInputWithWeight",
 "ITrainerInputWithLabel",
 "ITrainerInput"
 ],
 "OutputKind": [
 "IRegressionOutput",
 "ITrainerOutput"
 ]
 },
 {
 "Name": "Trainers.PcaAnomalyDetector",
 "Desc": "Train an PCA Anomaly model.",
Expand Down

33 changes: 33 additions & 0 deletions
test/BaselineOutput/SingleDebug/OLS/OLS-CV-wine-out.txt

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
maml.exe CV tr=OLS threads=- norm=No dout=%Output% loader=Text{col=Label:R4:11 col=Features:R4:0-10 sep=; header+} data=%Data% seed=1
Not adding a normalizer.
Trainer solving for 12 parameters across 2409 examples
Coefficient of determination R2 = 0.291173667189042, or 0.287920813763543 (adjusted)
Not training a calibrator because it is not needed.
Not adding a normalizer.
Trainer solving for 12 parameters across 2489 examples
Coefficient of determination R2 = 0.280280855195625, or 0.277084686203761 (adjusted)
Not training a calibrator because it is not needed.
L1(avg): 0.586798
L2(avg): 0.573048
RMS(avg): 0.756999
Loss-fn(avg): 0.573048
R Squared: 0.263841
L1(avg): 0.587999
L2(avg): 0.571859
RMS(avg): 0.756214
Loss-fn(avg): 0.571859
R Squared: 0.276072

OVERALL RESULTS
---------------------------------------
L1(avg): 0.587398 (0.0006)
L2(avg): 0.572454 (0.0006)
RMS(avg): 0.756606 (0.0004)
Loss-fn(avg): 0.572454 (0.0006)
R Squared: 0.269956 (0.0061)

---------------------------------------
Physical memory usage(MB): %Number%
Virtual memory usage(MB): %Number%
%DateTime% Time elapsed(s): %Number%

Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix");
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Adding the MlNetMklDeps package, adding the entry point, tests and docs for OLS.#578

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Closed

sfilipi
wants to merge
9
commits into

from

 Closed


Adding the MlNetMklDeps package, adding the entry point, tests and docs for OLS.
#578
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


9 commits


Select commit
Hold shift + click to select a range
9e96175

WIP: adding the MlNetMklDeps package


sfilipi Jul 24, 2018
fa519e8

Fixing the MlNetMklDeps nukpproj


sfilipi Jul 26, 2018
19779fe

reverting test changes to the CSharpCodeGen


sfilipi Jul 26, 2018
94ff6d3

Adding the LoadableClass assembly attribute with the signature:Signat…


sfilipi Jul 26, 2018
dfc2b35

merging from master


sfilipi Jul 26, 2018
cd18b74

following the same naming convention for the doc attributes in tags


sfilipi Jul 27, 2018
80f7bd6

Fixing the path in the OLS doc include


sfilipi Jul 27, 2018
f910923

removing unecessary packing


sfilipi Jul 27, 2018
2965477

In Mac, release only, one of the OLS tests is failing because the act…


sfilipi Jul 27, 2018

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


1 change: 1 addition & 0 deletions
build/Dependencies.props

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -8,5 +8,6 @@
 <SystemReflectionEmitLightweightPackageVersion>4.3.0</SystemReflectionEmitLightweightPackageVersion>
 <PublishSymbolsPackageVersion>1.0.0-beta-62824-02</PublishSymbolsPackageVersion>
 <LightGBMPackageVersion>2.1.2.2</LightGBMPackageVersion>
 <MlNetMklDepsPackageVersion>0.0.0.1</MlNetMklDepsPackageVersion>
 </PropertyGroup>
</Project>

2 changes: 2 additions & 0 deletions
src/Microsoft.ML.StandardLearners/Microsoft.ML.StandardLearners.csproj

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -11,6 +11,8 @@
 <ProjectReference Include="..\Microsoft.ML.CpuMath\Microsoft.ML.CpuMath.csproj" />
 <ProjectReference Include="..\Microsoft.ML.Data\Microsoft.ML.Data.csproj" />
 <ProjectReference Include="..\Microsoft.ML\Microsoft.ML.csproj" />

	<PackageReference Include="MlNetMklDeps" Version="$(MlNetMklDepsPackageVersion)" />
 </ItemGroup>

</Project>

26 changes: 21 additions & 5 deletions
src/Microsoft.ML.StandardLearners/Standard/OlsLinearRegression.cs

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -28,8 +28,11 @@
 "OLS Linear Regression Executor",
 OlsLinearRegressionPredictor.LoaderSignature)]

[assembly: LoadableClass(typeof(void), typeof(OlsLinearRegressionTrainer), null, typeof(SignatureEntryPointModule), OlsLinearRegressionTrainer.LoadNameValue)]

namespace Microsoft.ML.Runtime.Learners
{
 /// <include file='doc.xml' path='doc/members/member[@name="OLS"]/*' />
 public sealed class OlsLinearRegressionTrainer : TrainerBase<OlsLinearRegressionPredictor>
 {
 public sealed class Arguments : LearnerInputBaseWithWeight
Expand All@@ -51,11 +54,6 @@ public sealed class Arguments : LearnerInputBaseWithWeight
 public const string ShortName = "ols";
 internal const string Summary = "The ordinary least square regression fits the target function as a linear function of the numerical features "
 + "that minimizes the square loss function.";
 internal const string Remarks = @"<remarks>
<a href='https://en.wikipedia.org/wiki/Ordinary_least_squares'>Ordinary least squares (OLS)</a> is a parameterized regression method. 
It assumes that the conditional mean of the dependent variable follows a linear function of the dependent variables.
By minimizing the squares of the difference between observed values and the predictions, the parameters of the regressor can be estimated.
</remarks>";

 private readonly Float _l2Weight;
 private readonly bool _perParameterSignificance;
Expand DownExpand Up@@ -463,6 +461,24 @@ public static void Pptri(Layout layout, UpLo uplo, int n, Double[] ap)
 }
 }
 }

 [TlcModule.EntryPoint(Name = "Trainers.OrdinaryLeastSquaresRegressor",
 Desc = "Train an OLS regression model.",
 UserName = UserNameValue,
 ShortName = ShortName,
 XmlInclude = new[] { @"<include file='../Microsoft.ML.StandardLearners/Standard/doc.xml' path='doc/members/member[@name=""OLS""]/*' />" })]
 public static CommonOutputs.RegressionOutput TrainRegression(IHostEnvironment env, Arguments input)
 {
 Contracts.CheckValue(env, nameof(env));
 var host = env.Register("TrainOLS");
 host.CheckValue(input, nameof(input));
 EntryPointUtils.CheckInputArgs(host, input);

 return LearnerEntryPointsUtils.Train<Arguments, CommonOutputs.RegressionOutput>(host, input,
 () => new OlsLinearRegressionTrainer(host, input),
 () => LearnerEntryPointsUtils.FindColumn(host, input.TrainingData.Schema, input.LabelColumn),
 () => LearnerEntryPointsUtils.FindColumn(host, input.TrainingData.Schema, input.WeightColumn));
 }
 }

 /// <summary>
Expand Down

21 changes: 21 additions & 0 deletions
src/Microsoft.ML.StandardLearners/Standard/doc.xml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -68,6 +68,27 @@
 </code>
 </example>
 </example>

 <member name="OLS">
 <summary>
 Train an OLS regression model.
 </summary>
 <remarks>
 <a href='https://en.wikipedia.org/wiki/Ordinary_least_squares'>Ordinary least squares (OLS)</a> is a parameterized regression method. 
 It assumes that the conditional mean of the dependent variable follows a linear function of the dependent variables.
 The parameters of the regressor can be estimated by minimizing the squares of the difference between observed values and the predictions.
 </remarks>
 <example>
 <code language="csharp">
 new OrdinaryLeastSquaresRegressor
 {
 L2Weight = 0.1,
 PerParameterSignificance = false,
 NormalizeFeatures = Microsoft.ML.Models.NormalizeOption.Yes
 }
 </code>
 </example>
 </member>

 </members>
</doc>

99 changes: 99 additions & 0 deletions
src/Microsoft.ML/CSharpApi.cs

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -754,6 +754,18 @@ public void Add(Microsoft.ML.Trainers.OnlineGradientDescentRegressor input, Micr
 _jsonNodes.Add(Serialize("Trainers.OnlineGradientDescentRegressor", input, output));
 }

 public Microsoft.ML.Trainers.OrdinaryLeastSquaresRegressor.Output Add(Microsoft.ML.Trainers.OrdinaryLeastSquaresRegressor input)
 {
 var output = new Microsoft.ML.Trainers.OrdinaryLeastSquaresRegressor.Output();
 Add(input, output);
 return output;
 }

 public void Add(Microsoft.ML.Trainers.OrdinaryLeastSquaresRegressor input, Microsoft.ML.Trainers.OrdinaryLeastSquaresRegressor.Output output)
 {
 _jsonNodes.Add(Serialize("Trainers.OrdinaryLeastSquaresRegressor", input, output));
 }

 public Microsoft.ML.Trainers.PcaAnomalyDetector.Output Add(Microsoft.ML.Trainers.PcaAnomalyDetector input)
 {
 var output = new Microsoft.ML.Trainers.PcaAnomalyDetector.Output();
Expand DownExpand Up@@ -8824,6 +8836,93 @@ public OnlineGradientDescentRegressorPipelineStep(Output output)
 }
 }

 namespace Trainers
 {

 /// <include file='../Microsoft.ML.StandardLearners/Standard/doc.xml' path='doc/members/member[@name="OLS"]/*' />
 public sealed partial class OrdinaryLeastSquaresRegressor : Microsoft.ML.Runtime.EntryPoints.CommonInputs.ITrainerInputWithWeight, Microsoft.ML.Runtime.EntryPoints.CommonInputs.ITrainerInputWithLabel, Microsoft.ML.Runtime.EntryPoints.CommonInputs.ITrainerInput, Microsoft.ML.ILearningPipelineItem
 {


 /// <summary>
 /// L2 regularization weight
 /// </summary>
 [TlcModule.SweepableDiscreteParamAttribute("L2Weight", new object[]{1E-06f, 0.1f, 1f})]
 public float L2Weight { get; set; } = 1E-06f;

 /// <summary>
 /// Whether to calculate per parameter significance statistics
 /// </summary>
 public bool PerParameterSignificance { get; set; } = true;

 /// <summary>
 /// Column to use for example weight
 /// </summary>
 public Microsoft.ML.Runtime.EntryPoints.Optional<string> WeightColumn { get; set; }

 /// <summary>
 /// Column to use for labels
 /// </summary>
 public string LabelColumn { get; set; } = "Label";

 /// <summary>
 /// The data to be used for training
 /// </summary>
 public Var<Microsoft.ML.Runtime.Data.IDataView> TrainingData { get; set; } = new Var<Microsoft.ML.Runtime.Data.IDataView>();

 /// <summary>
 /// Column to use for features
 /// </summary>
 public string FeatureColumn { get; set; } = "Features";

 /// <summary>
 /// Normalize option for the feature column
 /// </summary>
 public Microsoft.ML.Models.NormalizeOption NormalizeFeatures { get; set; } = Microsoft.ML.Models.NormalizeOption.Auto;

 /// <summary>
 /// Whether learner should cache input training data
 /// </summary>
 public Microsoft.ML.Models.CachingOptions Caching { get; set; } = Microsoft.ML.Models.CachingOptions.Auto;


 public sealed class Output : Microsoft.ML.Runtime.EntryPoints.CommonOutputs.IRegressionOutput, Microsoft.ML.Runtime.EntryPoints.CommonOutputs.ITrainerOutput
 {
 /// <summary>
 /// The trained model
 /// </summary>
 public Var<Microsoft.ML.Runtime.EntryPoints.IPredictorModel> PredictorModel { get; set; } = new Var<Microsoft.ML.Runtime.EntryPoints.IPredictorModel>();

 }
 public Var<IDataView> GetInputData() => TrainingData;

 public ILearningPipelineStep ApplyStep(ILearningPipelineStep previousStep, Experiment experiment)
 {
 if (previousStep != null)
 {
 if (!(previousStep is ILearningPipelineDataStep dataStep))
 {
 throw new InvalidOperationException($"{ nameof(OrdinaryLeastSquaresRegressor)} only supports an { nameof(ILearningPipelineDataStep)} as an input.");
 }

 TrainingData = dataStep.Data;
 }
 Output output = experiment.Add(this);
 return new OrdinaryLeastSquaresRegressorPipelineStep(output);
 }

 private class OrdinaryLeastSquaresRegressorPipelineStep : ILearningPipelinePredictorStep
 {
 public OrdinaryLeastSquaresRegressorPipelineStep(Output output)
 {
 Model = output.PredictorModel;
 }

 public Var<IPredictorModel> Model { get; }
 }
 }
 }

 namespace Trainers
 {

Expand Down

1 change: 1 addition & 0 deletions
test/BaselineOutput/Common/EntryPoints/core_ep-list.tsv

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -59,6 +59,7 @@ Trainers.LogisticRegressionBinaryClassifier	Logistic Regression is a method in s
Trainers.LogisticRegressionClassifier	Logistic Regression is a method in statistics used to predict the probability of occurrence of an event and can be used as a classification algorithm. The algorithm predicts the probability of occurrence of an event by fitting data to a logistical function.	Microsoft.ML.Runtime.Learners.LogisticRegression	TrainMultiClass	Microsoft.ML.Runtime.Learners.MulticlassLogisticRegression+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+MulticlassClassificationOutput
Trainers.NaiveBayesClassifier	Train a MultiClassNaiveBayesTrainer.	Microsoft.ML.Runtime.Learners.MultiClassNaiveBayesTrainer	TrainMultiClassNaiveBayesTrainer	Microsoft.ML.Runtime.Learners.MultiClassNaiveBayesTrainer+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+MulticlassClassificationOutput
Trainers.OnlineGradientDescentRegressor	Train a Online gradient descent perceptron.	Microsoft.ML.Runtime.Learners.OnlineGradientDescentTrainer	TrainRegression	Microsoft.ML.Runtime.Learners.OnlineGradientDescentTrainer+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+RegressionOutput
Trainers.OrdinaryLeastSquaresRegressor	Train an OLS regression model.	Microsoft.ML.Runtime.Learners.OlsLinearRegressionTrainer	TrainRegression	Microsoft.ML.Runtime.Learners.OlsLinearRegressionTrainer+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+RegressionOutput
Trainers.PcaAnomalyDetector	Train an PCA Anomaly model.	Microsoft.ML.Runtime.PCA.RandomizedPcaTrainer	TrainPcaAnomaly	Microsoft.ML.Runtime.PCA.RandomizedPcaTrainer+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+AnomalyDetectionOutput
Trainers.PoissonRegressor	Train an Poisson regression model.	Microsoft.ML.Runtime.Learners.PoissonRegression	TrainRegression	Microsoft.ML.Runtime.Learners.PoissonRegression+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+RegressionOutput
Trainers.StochasticDualCoordinateAscentBinaryClassifier	Train an SDCA binary model.	Microsoft.ML.Runtime.Learners.Sdca	TrainBinary	Microsoft.ML.Runtime.Learners.LinearClassificationTrainer+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+BinaryClassificationOutput
Expand Down

143 changes: 143 additions & 0 deletions
test/BaselineOutput/Common/EntryPoints/core_manifest.json

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -14050,6 +14050,149 @@
 "ITrainerOutput"
 ]
 },
 {
 "Name": "Trainers.OrdinaryLeastSquaresRegressor",
 "Desc": "Train an OLS regression model.",
 "FriendlyName": "Ordinary Least Squares (Regression)",
 "ShortName": "ols",
 "Inputs": [
 {
 "Name": "TrainingData",
 "Type": "DataView",
 "Desc": "The data to be used for training",
 "Aliases": [
 "data"
 ],
 "Required": true,
 "SortOrder": 1.0,
 "IsNullable": false
 },
 {
 "Name": "FeatureColumn",
 "Type": "String",
 "Desc": "Column to use for features",
 "Aliases": [
 "feat"
 ],
 "Required": false,
 "SortOrder": 2.0,
 "IsNullable": false,
 "Default": "Features"
 },
 {
 "Name": "LabelColumn",
 "Type": "String",
 "Desc": "Column to use for labels",
 "Aliases": [
 "lab"
 ],
 "Required": false,
 "SortOrder": 3.0,
 "IsNullable": false,
 "Default": "Label"
 },
 {
 "Name": "WeightColumn",
 "Type": "String",
 "Desc": "Column to use for example weight",
 "Aliases": [
 "weight"
 ],
 "Required": false,
 "SortOrder": 4.0,
 "IsNullable": false,
 "Default": "Weight"
 },
 {
 "Name": "NormalizeFeatures",
 "Type": {
 "Kind": "Enum",
 "Values": [
 "No",
 "Warn",
 "Auto",
 "Yes"
 ]
 },
 "Desc": "Normalize option for the feature column",
 "Aliases": [
 "norm"
 ],
 "Required": false,
 "SortOrder": 5.0,
 "IsNullable": false,
 "Default": "Auto"
 },
 {
 "Name": "Caching",
 "Type": {
 "Kind": "Enum",
 "Values": [
 "Auto",
 "Memory",
 "Disk",
 "None"
 ]
 },
 "Desc": "Whether learner should cache input training data",
 "Aliases": [
 "cache"
 ],
 "Required": false,
 "SortOrder": 6.0,
 "IsNullable": false,
 "Default": "Auto"
 },
 {
 "Name": "L2Weight",
 "Type": "Float",
 "Desc": "L2 regularization weight",
 "Aliases": [
 "l2"
 ],
 "Required": false,
 "SortOrder": 50.0,
 "IsNullable": false,
 "Default": 1E-06,
 "SweepRange": {
 "RangeType": "Discrete",
 "Values": [
 1E-06,
 0.1,
 1.0
 ]
 }
 },
 {
 "Name": "PerParameterSignificance",
 "Type": "Bool",
 "Desc": "Whether to calculate per parameter significance statistics",
 "Aliases": [
 "sig"
 ],
 "Required": false,
 "SortOrder": 150.0,
 "IsNullable": false,
 "Default": true
 }
 ],
 "Outputs": [
 {
 "Name": "PredictorModel",
 "Type": "PredictorModel",
 "Desc": "The trained model"
 }
 ],
 "InputKind": [
 "ITrainerInputWithWeight",
 "ITrainerInputWithLabel",
 "ITrainerInput"
 ],
 "OutputKind": [
 "IRegressionOutput",
 "ITrainerOutput"
 ]
 },
 {
 "Name": "Trainers.PcaAnomalyDetector",
 "Desc": "Train an PCA Anomaly model.",
Expand Down

33 changes: 33 additions & 0 deletions
test/BaselineOutput/SingleDebug/OLS/OLS-CV-wine-out.txt

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
maml.exe CV tr=OLS threads=- norm=No dout=%Output% loader=Text{col=Label:R4:11 col=Features:R4:0-10 sep=; header+} data=%Data% seed=1
Not adding a normalizer.
Trainer solving for 12 parameters across 2409 examples
Coefficient of determination R2 = 0.291173667189042, or 0.287920813763543 (adjusted)
Not training a calibrator because it is not needed.
Not adding a normalizer.
Trainer solving for 12 parameters across 2489 examples
Coefficient of determination R2 = 0.280280855195625, or 0.277084686203761 (adjusted)
Not training a calibrator because it is not needed.
L1(avg): 0.586798
L2(avg): 0.573048
RMS(avg): 0.756999
Loss-fn(avg): 0.573048
R Squared: 0.263841
L1(avg): 0.587999
L2(avg): 0.571859
RMS(avg): 0.756214
Loss-fn(avg): 0.571859
R Squared: 0.276072

OVERALL RESULTS
---------------------------------------
L1(avg): 0.587398 (0.0006)
L2(avg): 0.572454 (0.0006)
RMS(avg): 0.756606 (0.0004)
Loss-fn(avg): 0.572454 (0.0006)
R Squared: 0.269956 (0.0061)

---------------------------------------
Physical memory usage(MB): %Number%
Virtual memory usage(MB): %Number%
%DateTime% Time elapsed(s): %Number%

Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms");
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Adding the MlNetMklDeps package, adding the entry point, tests and docs for OLS.#578

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Closed

sfilipi
wants to merge
9
commits into

from

 Closed


Adding the MlNetMklDeps package, adding the entry point, tests and docs for OLS.
#578
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


9 commits


Select commit
Hold shift + click to select a range
9e96175

WIP: adding the MlNetMklDeps package


sfilipi Jul 24, 2018
fa519e8

Fixing the MlNetMklDeps nukpproj


sfilipi Jul 26, 2018
19779fe

reverting test changes to the CSharpCodeGen


sfilipi Jul 26, 2018
94ff6d3

Adding the LoadableClass assembly attribute with the signature:Signat…


sfilipi Jul 26, 2018
dfc2b35

merging from master


sfilipi Jul 26, 2018
cd18b74

following the same naming convention for the doc attributes in tags


sfilipi Jul 27, 2018
80f7bd6

Fixing the path in the OLS doc include


sfilipi Jul 27, 2018
f910923

removing unecessary packing


sfilipi Jul 27, 2018
2965477

In Mac, release only, one of the OLS tests is failing because the act…


sfilipi Jul 27, 2018

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


1 change: 1 addition & 0 deletions
build/Dependencies.props

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -8,5 +8,6 @@
 <SystemReflectionEmitLightweightPackageVersion>4.3.0</SystemReflectionEmitLightweightPackageVersion>
 <PublishSymbolsPackageVersion>1.0.0-beta-62824-02</PublishSymbolsPackageVersion>
 <LightGBMPackageVersion>2.1.2.2</LightGBMPackageVersion>
 <MlNetMklDepsPackageVersion>0.0.0.1</MlNetMklDepsPackageVersion>
 </PropertyGroup>
</Project>

2 changes: 2 additions & 0 deletions
src/Microsoft.ML.StandardLearners/Microsoft.ML.StandardLearners.csproj

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -11,6 +11,8 @@
 <ProjectReference Include="..\Microsoft.ML.CpuMath\Microsoft.ML.CpuMath.csproj" />
 <ProjectReference Include="..\Microsoft.ML.Data\Microsoft.ML.Data.csproj" />
 <ProjectReference Include="..\Microsoft.ML\Microsoft.ML.csproj" />

	<PackageReference Include="MlNetMklDeps" Version="$(MlNetMklDepsPackageVersion)" />
 </ItemGroup>

</Project>

26 changes: 21 additions & 5 deletions
src/Microsoft.ML.StandardLearners/Standard/OlsLinearRegression.cs

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -28,8 +28,11 @@
 "OLS Linear Regression Executor",
 OlsLinearRegressionPredictor.LoaderSignature)]

[assembly: LoadableClass(typeof(void), typeof(OlsLinearRegressionTrainer), null, typeof(SignatureEntryPointModule), OlsLinearRegressionTrainer.LoadNameValue)]

namespace Microsoft.ML.Runtime.Learners
{
 /// <include file='doc.xml' path='doc/members/member[@name="OLS"]/*' />
 public sealed class OlsLinearRegressionTrainer : TrainerBase<OlsLinearRegressionPredictor>
 {
 public sealed class Arguments : LearnerInputBaseWithWeight
Expand All@@ -51,11 +54,6 @@ public sealed class Arguments : LearnerInputBaseWithWeight
 public const string ShortName = "ols";
 internal const string Summary = "The ordinary least square regression fits the target function as a linear function of the numerical features "
 + "that minimizes the square loss function.";
 internal const string Remarks = @"<remarks>
<a href='https://en.wikipedia.org/wiki/Ordinary_least_squares'>Ordinary least squares (OLS)</a> is a parameterized regression method. 
It assumes that the conditional mean of the dependent variable follows a linear function of the dependent variables.
By minimizing the squares of the difference between observed values and the predictions, the parameters of the regressor can be estimated.
</remarks>";

 private readonly Float _l2Weight;
 private readonly bool _perParameterSignificance;
Expand DownExpand Up@@ -463,6 +461,24 @@ public static void Pptri(Layout layout, UpLo uplo, int n, Double[] ap)
 }
 }
 }

 [TlcModule.EntryPoint(Name = "Trainers.OrdinaryLeastSquaresRegressor",
 Desc = "Train an OLS regression model.",
 UserName = UserNameValue,
 ShortName = ShortName,
 XmlInclude = new[] { @"<include file='../Microsoft.ML.StandardLearners/Standard/doc.xml' path='doc/members/member[@name=""OLS""]/*' />" })]
 public static CommonOutputs.RegressionOutput TrainRegression(IHostEnvironment env, Arguments input)
 {
 Contracts.CheckValue(env, nameof(env));
 var host = env.Register("TrainOLS");
 host.CheckValue(input, nameof(input));
 EntryPointUtils.CheckInputArgs(host, input);

 return LearnerEntryPointsUtils.Train<Arguments, CommonOutputs.RegressionOutput>(host, input,
 () => new OlsLinearRegressionTrainer(host, input),
 () => LearnerEntryPointsUtils.FindColumn(host, input.TrainingData.Schema, input.LabelColumn),
 () => LearnerEntryPointsUtils.FindColumn(host, input.TrainingData.Schema, input.WeightColumn));
 }
 }

 /// <summary>
Expand Down

21 changes: 21 additions & 0 deletions
src/Microsoft.ML.StandardLearners/Standard/doc.xml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -68,6 +68,27 @@
 </code>
 </example>
 </example>

 <member name="OLS">
 <summary>
 Train an OLS regression model.
 </summary>
 <remarks>
 <a href='https://en.wikipedia.org/wiki/Ordinary_least_squares'>Ordinary least squares (OLS)</a> is a parameterized regression method. 
 It assumes that the conditional mean of the dependent variable follows a linear function of the dependent variables.
 The parameters of the regressor can be estimated by minimizing the squares of the difference between observed values and the predictions.
 </remarks>
 <example>
 <code language="csharp">
 new OrdinaryLeastSquaresRegressor
 {
 L2Weight = 0.1,
 PerParameterSignificance = false,
 NormalizeFeatures = Microsoft.ML.Models.NormalizeOption.Yes
 }
 </code>
 </example>
 </member>

 </members>
</doc>

99 changes: 99 additions & 0 deletions
src/Microsoft.ML/CSharpApi.cs

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -754,6 +754,18 @@ public void Add(Microsoft.ML.Trainers.OnlineGradientDescentRegressor input, Micr
 _jsonNodes.Add(Serialize("Trainers.OnlineGradientDescentRegressor", input, output));
 }

 public Microsoft.ML.Trainers.OrdinaryLeastSquaresRegressor.Output Add(Microsoft.ML.Trainers.OrdinaryLeastSquaresRegressor input)
 {
 var output = new Microsoft.ML.Trainers.OrdinaryLeastSquaresRegressor.Output();
 Add(input, output);
 return output;
 }

 public void Add(Microsoft.ML.Trainers.OrdinaryLeastSquaresRegressor input, Microsoft.ML.Trainers.OrdinaryLeastSquaresRegressor.Output output)
 {
 _jsonNodes.Add(Serialize("Trainers.OrdinaryLeastSquaresRegressor", input, output));
 }

 public Microsoft.ML.Trainers.PcaAnomalyDetector.Output Add(Microsoft.ML.Trainers.PcaAnomalyDetector input)
 {
 var output = new Microsoft.ML.Trainers.PcaAnomalyDetector.Output();
Expand DownExpand Up@@ -8824,6 +8836,93 @@ public OnlineGradientDescentRegressorPipelineStep(Output output)
 }
 }

 namespace Trainers
 {

 /// <include file='../Microsoft.ML.StandardLearners/Standard/doc.xml' path='doc/members/member[@name="OLS"]/*' />
 public sealed partial class OrdinaryLeastSquaresRegressor : Microsoft.ML.Runtime.EntryPoints.CommonInputs.ITrainerInputWithWeight, Microsoft.ML.Runtime.EntryPoints.CommonInputs.ITrainerInputWithLabel, Microsoft.ML.Runtime.EntryPoints.CommonInputs.ITrainerInput, Microsoft.ML.ILearningPipelineItem
 {


 /// <summary>
 /// L2 regularization weight
 /// </summary>
 [TlcModule.SweepableDiscreteParamAttribute("L2Weight", new object[]{1E-06f, 0.1f, 1f})]
 public float L2Weight { get; set; } = 1E-06f;

 /// <summary>
 /// Whether to calculate per parameter significance statistics
 /// </summary>
 public bool PerParameterSignificance { get; set; } = true;

 /// <summary>
 /// Column to use for example weight
 /// </summary>
 public Microsoft.ML.Runtime.EntryPoints.Optional<string> WeightColumn { get; set; }

 /// <summary>
 /// Column to use for labels
 /// </summary>
 public string LabelColumn { get; set; } = "Label";

 /// <summary>
 /// The data to be used for training
 /// </summary>
 public Var<Microsoft.ML.Runtime.Data.IDataView> TrainingData { get; set; } = new Var<Microsoft.ML.Runtime.Data.IDataView>();

 /// <summary>
 /// Column to use for features
 /// </summary>
 public string FeatureColumn { get; set; } = "Features";

 /// <summary>
 /// Normalize option for the feature column
 /// </summary>
 public Microsoft.ML.Models.NormalizeOption NormalizeFeatures { get; set; } = Microsoft.ML.Models.NormalizeOption.Auto;

 /// <summary>
 /// Whether learner should cache input training data
 /// </summary>
 public Microsoft.ML.Models.CachingOptions Caching { get; set; } = Microsoft.ML.Models.CachingOptions.Auto;


 public sealed class Output : Microsoft.ML.Runtime.EntryPoints.CommonOutputs.IRegressionOutput, Microsoft.ML.Runtime.EntryPoints.CommonOutputs.ITrainerOutput
 {
 /// <summary>
 /// The trained model
 /// </summary>
 public Var<Microsoft.ML.Runtime.EntryPoints.IPredictorModel> PredictorModel { get; set; } = new Var<Microsoft.ML.Runtime.EntryPoints.IPredictorModel>();

 }
 public Var<IDataView> GetInputData() => TrainingData;

 public ILearningPipelineStep ApplyStep(ILearningPipelineStep previousStep, Experiment experiment)
 {
 if (previousStep != null)
 {
 if (!(previousStep is ILearningPipelineDataStep dataStep))
 {
 throw new InvalidOperationException($"{ nameof(OrdinaryLeastSquaresRegressor)} only supports an { nameof(ILearningPipelineDataStep)} as an input.");
 }

 TrainingData = dataStep.Data;
 }
 Output output = experiment.Add(this);
 return new OrdinaryLeastSquaresRegressorPipelineStep(output);
 }

 private class OrdinaryLeastSquaresRegressorPipelineStep : ILearningPipelinePredictorStep
 {
 public OrdinaryLeastSquaresRegressorPipelineStep(Output output)
 {
 Model = output.PredictorModel;
 }

 public Var<IPredictorModel> Model { get; }
 }
 }
 }

 namespace Trainers
 {

Expand Down

1 change: 1 addition & 0 deletions
test/BaselineOutput/Common/EntryPoints/core_ep-list.tsv

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -59,6 +59,7 @@ Trainers.LogisticRegressionBinaryClassifier	Logistic Regression is a method in s
Trainers.LogisticRegressionClassifier	Logistic Regression is a method in statistics used to predict the probability of occurrence of an event and can be used as a classification algorithm. The algorithm predicts the probability of occurrence of an event by fitting data to a logistical function.	Microsoft.ML.Runtime.Learners.LogisticRegression	TrainMultiClass	Microsoft.ML.Runtime.Learners.MulticlassLogisticRegression+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+MulticlassClassificationOutput
Trainers.NaiveBayesClassifier	Train a MultiClassNaiveBayesTrainer.	Microsoft.ML.Runtime.Learners.MultiClassNaiveBayesTrainer	TrainMultiClassNaiveBayesTrainer	Microsoft.ML.Runtime.Learners.MultiClassNaiveBayesTrainer+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+MulticlassClassificationOutput
Trainers.OnlineGradientDescentRegressor	Train a Online gradient descent perceptron.	Microsoft.ML.Runtime.Learners.OnlineGradientDescentTrainer	TrainRegression	Microsoft.ML.Runtime.Learners.OnlineGradientDescentTrainer+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+RegressionOutput
Trainers.OrdinaryLeastSquaresRegressor	Train an OLS regression model.	Microsoft.ML.Runtime.Learners.OlsLinearRegressionTrainer	TrainRegression	Microsoft.ML.Runtime.Learners.OlsLinearRegressionTrainer+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+RegressionOutput
Trainers.PcaAnomalyDetector	Train an PCA Anomaly model.	Microsoft.ML.Runtime.PCA.RandomizedPcaTrainer	TrainPcaAnomaly	Microsoft.ML.Runtime.PCA.RandomizedPcaTrainer+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+AnomalyDetectionOutput
Trainers.PoissonRegressor	Train an Poisson regression model.	Microsoft.ML.Runtime.Learners.PoissonRegression	TrainRegression	Microsoft.ML.Runtime.Learners.PoissonRegression+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+RegressionOutput
Trainers.StochasticDualCoordinateAscentBinaryClassifier	Train an SDCA binary model.	Microsoft.ML.Runtime.Learners.Sdca	TrainBinary	Microsoft.ML.Runtime.Learners.LinearClassificationTrainer+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+BinaryClassificationOutput
Expand Down

143 changes: 143 additions & 0 deletions
test/BaselineOutput/Common/EntryPoints/core_manifest.json

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -14050,6 +14050,149 @@
 "ITrainerOutput"
 ]
 },
 {
 "Name": "Trainers.OrdinaryLeastSquaresRegressor",
 "Desc": "Train an OLS regression model.",
 "FriendlyName": "Ordinary Least Squares (Regression)",
 "ShortName": "ols",
 "Inputs": [
 {
 "Name": "TrainingData",
 "Type": "DataView",
 "Desc": "The data to be used for training",
 "Aliases": [
 "data"
 ],
 "Required": true,
 "SortOrder": 1.0,
 "IsNullable": false
 },
 {
 "Name": "FeatureColumn",
 "Type": "String",
 "Desc": "Column to use for features",
 "Aliases": [
 "feat"
 ],
 "Required": false,
 "SortOrder": 2.0,
 "IsNullable": false,
 "Default": "Features"
 },
 {
 "Name": "LabelColumn",
 "Type": "String",
 "Desc": "Column to use for labels",
 "Aliases": [
 "lab"
 ],
 "Required": false,
 "SortOrder": 3.0,
 "IsNullable": false,
 "Default": "Label"
 },
 {
 "Name": "WeightColumn",
 "Type": "String",
 "Desc": "Column to use for example weight",
 "Aliases": [
 "weight"
 ],
 "Required": false,
 "SortOrder": 4.0,
 "IsNullable": false,
 "Default": "Weight"
 },
 {
 "Name": "NormalizeFeatures",
 "Type": {
 "Kind": "Enum",
 "Values": [
 "No",
 "Warn",
 "Auto",
 "Yes"
 ]
 },
 "Desc": "Normalize option for the feature column",
 "Aliases": [
 "norm"
 ],
 "Required": false,
 "SortOrder": 5.0,
 "IsNullable": false,
 "Default": "Auto"
 },
 {
 "Name": "Caching",
 "Type": {
 "Kind": "Enum",
 "Values": [
 "Auto",
 "Memory",
 "Disk",
 "None"
 ]
 },
 "Desc": "Whether learner should cache input training data",
 "Aliases": [
 "cache"
 ],
 "Required": false,
 "SortOrder": 6.0,
 "IsNullable": false,
 "Default": "Auto"
 },
 {
 "Name": "L2Weight",
 "Type": "Float",
 "Desc": "L2 regularization weight",
 "Aliases": [
 "l2"
 ],
 "Required": false,
 "SortOrder": 50.0,
 "IsNullable": false,
 "Default": 1E-06,
 "SweepRange": {
 "RangeType": "Discrete",
 "Values": [
 1E-06,
 0.1,
 1.0
 ]
 }
 },
 {
 "Name": "PerParameterSignificance",
 "Type": "Bool",
 "Desc": "Whether to calculate per parameter significance statistics",
 "Aliases": [
 "sig"
 ],
 "Required": false,
 "SortOrder": 150.0,
 "IsNullable": false,
 "Default": true
 }
 ],
 "Outputs": [
 {
 "Name": "PredictorModel",
 "Type": "PredictorModel",
 "Desc": "The trained model"
 }
 ],
 "InputKind": [
 "ITrainerInputWithWeight",
 "ITrainerInputWithLabel",
 "ITrainerInput"
 ],
 "OutputKind": [
 "IRegressionOutput",
 "ITrainerOutput"
 ]
 },
 {
 "Name": "Trainers.PcaAnomalyDetector",
 "Desc": "Train an PCA Anomaly model.",
Expand Down

33 changes: 33 additions & 0 deletions
test/BaselineOutput/SingleDebug/OLS/OLS-CV-wine-out.txt

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
maml.exe CV tr=OLS threads=- norm=No dout=%Output% loader=Text{col=Label:R4:11 col=Features:R4:0-10 sep=; header+} data=%Data% seed=1
Not adding a normalizer.
Trainer solving for 12 parameters across 2409 examples
Coefficient of determination R2 = 0.291173667189042, or 0.287920813763543 (adjusted)
Not training a calibrator because it is not needed.
Not adding a normalizer.
Trainer solving for 12 parameters across 2489 examples
Coefficient of determination R2 = 0.280280855195625, or 0.277084686203761 (adjusted)
Not training a calibrator because it is not needed.
L1(avg): 0.586798
L2(avg): 0.573048
RMS(avg): 0.756999
Loss-fn(avg): 0.573048
R Squared: 0.263841
L1(avg): 0.587999
L2(avg): 0.571859
RMS(avg): 0.756214
Loss-fn(avg): 0.571859
R Squared: 0.276072

OVERALL RESULTS
---------------------------------------
L1(avg): 0.587398 (0.0006)
L2(avg): 0.572454 (0.0006)
RMS(avg): 0.756606 (0.0004)
Loss-fn(avg): 0.572454 (0.0006)
R Squared: 0.269956 (0.0061)

---------------------------------------
Physical memory usage(MB): %Number%
Virtual memory usage(MB): %Number%
%DateTime% Time elapsed(s): %Number%

Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links");
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '

Skip to content
Adding the MlNetMklDeps package, adding the entry point, tests and docs for OLS.#578

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Closed

sfilipi
wants to merge
9
commits into

from

 Closed


Adding the MlNetMklDeps package, adding the entry point, tests and docs for OLS.
#578
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


9 commits


Select commit
Hold shift + click to select a range
9e96175

WIP: adding the MlNetMklDeps package


sfilipi Jul 24, 2018
fa519e8

Fixing the MlNetMklDeps nukpproj


sfilipi Jul 26, 2018
19779fe

reverting test changes to the CSharpCodeGen


sfilipi Jul 26, 2018
94ff6d3

Adding the LoadableClass assembly attribute with the signature:Signat…


sfilipi Jul 26, 2018
dfc2b35

merging from master


sfilipi Jul 26, 2018
cd18b74

following the same naming convention for the doc attributes in tags


sfilipi Jul 27, 2018
80f7bd6

Fixing the path in the OLS doc include


sfilipi Jul 27, 2018
f910923

removing unecessary packing


sfilipi Jul 27, 2018
2965477

In Mac, release only, one of the OLS tests is failing because the act…


sfilipi Jul 27, 2018

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


1 change: 1 addition & 0 deletions
build/Dependencies.props

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -8,5 +8,6 @@
 <SystemReflectionEmitLightweightPackageVersion>4.3.0</SystemReflectionEmitLightweightPackageVersion>
 <PublishSymbolsPackageVersion>1.0.0-beta-62824-02</PublishSymbolsPackageVersion>
 <LightGBMPackageVersion>2.1.2.2</LightGBMPackageVersion>
 <MlNetMklDepsPackageVersion>0.0.0.1</MlNetMklDepsPackageVersion>
 </PropertyGroup>
</Project>

2 changes: 2 additions & 0 deletions
src/Microsoft.ML.StandardLearners/Microsoft.ML.StandardLearners.csproj

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -11,6 +11,8 @@
 <ProjectReference Include="..\Microsoft.ML.CpuMath\Microsoft.ML.CpuMath.csproj" />
 <ProjectReference Include="..\Microsoft.ML.Data\Microsoft.ML.Data.csproj" />
 <ProjectReference Include="..\Microsoft.ML\Microsoft.ML.csproj" />

	<PackageReference Include="MlNetMklDeps" Version="$(MlNetMklDepsPackageVersion)" />
 </ItemGroup>

</Project>

26 changes: 21 additions & 5 deletions
src/Microsoft.ML.StandardLearners/Standard/OlsLinearRegression.cs

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -28,8 +28,11 @@
 "OLS Linear Regression Executor",
 OlsLinearRegressionPredictor.LoaderSignature)]

[assembly: LoadableClass(typeof(void), typeof(OlsLinearRegressionTrainer), null, typeof(SignatureEntryPointModule), OlsLinearRegressionTrainer.LoadNameValue)]

namespace Microsoft.ML.Runtime.Learners
{
 /// <include file='doc.xml' path='doc/members/member[@name="OLS"]/*' />
 public sealed class OlsLinearRegressionTrainer : TrainerBase<OlsLinearRegressionPredictor>
 {
 public sealed class Arguments : LearnerInputBaseWithWeight
Expand All@@ -51,11 +54,6 @@ public sealed class Arguments : LearnerInputBaseWithWeight
 public const string ShortName = "ols";
 internal const string Summary = "The ordinary least square regression fits the target function as a linear function of the numerical features "
 + "that minimizes the square loss function.";
 internal const string Remarks = @"<remarks>
<a href='https://en.wikipedia.org/wiki/Ordinary_least_squares'>Ordinary least squares (OLS)</a> is a parameterized regression method. 
It assumes that the conditional mean of the dependent variable follows a linear function of the dependent variables.
By minimizing the squares of the difference between observed values and the predictions, the parameters of the regressor can be estimated.
</remarks>";

 private readonly Float _l2Weight;
 private readonly bool _perParameterSignificance;
Expand DownExpand Up@@ -463,6 +461,24 @@ public static void Pptri(Layout layout, UpLo uplo, int n, Double[] ap)
 }
 }
 }

 [TlcModule.EntryPoint(Name = "Trainers.OrdinaryLeastSquaresRegressor",
 Desc = "Train an OLS regression model.",
 UserName = UserNameValue,
 ShortName = ShortName,
 XmlInclude = new[] { @"<include file='../Microsoft.ML.StandardLearners/Standard/doc.xml' path='doc/members/member[@name=""OLS""]/*' />" })]
 public static CommonOutputs.RegressionOutput TrainRegression(IHostEnvironment env, Arguments input)
 {
 Contracts.CheckValue(env, nameof(env));
 var host = env.Register("TrainOLS");
 host.CheckValue(input, nameof(input));
 EntryPointUtils.CheckInputArgs(host, input);

 return LearnerEntryPointsUtils.Train<Arguments, CommonOutputs.RegressionOutput>(host, input,
 () => new OlsLinearRegressionTrainer(host, input),
 () => LearnerEntryPointsUtils.FindColumn(host, input.TrainingData.Schema, input.LabelColumn),
 () => LearnerEntryPointsUtils.FindColumn(host, input.TrainingData.Schema, input.WeightColumn));
 }
 }

 /// <summary>
Expand Down

21 changes: 21 additions & 0 deletions
src/Microsoft.ML.StandardLearners/Standard/doc.xml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -68,6 +68,27 @@
 </code>
 </example>
 </example>

 <member name="OLS">
 <summary>
 Train an OLS regression model.
 </summary>
 <remarks>
 <a href='https://en.wikipedia.org/wiki/Ordinary_least_squares'>Ordinary least squares (OLS)</a> is a parameterized regression method. 
 It assumes that the conditional mean of the dependent variable follows a linear function of the dependent variables.
 The parameters of the regressor can be estimated by minimizing the squares of the difference between observed values and the predictions.
 </remarks>
 <example>
 <code language="csharp">
 new OrdinaryLeastSquaresRegressor
 {
 L2Weight = 0.1,
 PerParameterSignificance = false,
 NormalizeFeatures = Microsoft.ML.Models.NormalizeOption.Yes
 }
 </code>
 </example>
 </member>

 </members>
</doc>

99 changes: 99 additions & 0 deletions
src/Microsoft.ML/CSharpApi.cs

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -754,6 +754,18 @@ public void Add(Microsoft.ML.Trainers.OnlineGradientDescentRegressor input, Micr
 _jsonNodes.Add(Serialize("Trainers.OnlineGradientDescentRegressor", input, output));
 }

 public Microsoft.ML.Trainers.OrdinaryLeastSquaresRegressor.Output Add(Microsoft.ML.Trainers.OrdinaryLeastSquaresRegressor input)
 {
 var output = new Microsoft.ML.Trainers.OrdinaryLeastSquaresRegressor.Output();
 Add(input, output);
 return output;
 }

 public void Add(Microsoft.ML.Trainers.OrdinaryLeastSquaresRegressor input, Microsoft.ML.Trainers.OrdinaryLeastSquaresRegressor.Output output)
 {
 _jsonNodes.Add(Serialize("Trainers.OrdinaryLeastSquaresRegressor", input, output));
 }

 public Microsoft.ML.Trainers.PcaAnomalyDetector.Output Add(Microsoft.ML.Trainers.PcaAnomalyDetector input)
 {
 var output = new Microsoft.ML.Trainers.PcaAnomalyDetector.Output();
Expand DownExpand Up@@ -8824,6 +8836,93 @@ public OnlineGradientDescentRegressorPipelineStep(Output output)
 }
 }

 namespace Trainers
 {

 /// <include file='../Microsoft.ML.StandardLearners/Standard/doc.xml' path='doc/members/member[@name="OLS"]/*' />
 public sealed partial class OrdinaryLeastSquaresRegressor : Microsoft.ML.Runtime.EntryPoints.CommonInputs.ITrainerInputWithWeight, Microsoft.ML.Runtime.EntryPoints.CommonInputs.ITrainerInputWithLabel, Microsoft.ML.Runtime.EntryPoints.CommonInputs.ITrainerInput, Microsoft.ML.ILearningPipelineItem
 {


 /// <summary>
 /// L2 regularization weight
 /// </summary>
 [TlcModule.SweepableDiscreteParamAttribute("L2Weight", new object[]{1E-06f, 0.1f, 1f})]
 public float L2Weight { get; set; } = 1E-06f;

 /// <summary>
 /// Whether to calculate per parameter significance statistics
 /// </summary>
 public bool PerParameterSignificance { get; set; } = true;

 /// <summary>
 /// Column to use for example weight
 /// </summary>
 public Microsoft.ML.Runtime.EntryPoints.Optional<string> WeightColumn { get; set; }

 /// <summary>
 /// Column to use for labels
 /// </summary>
 public string LabelColumn { get; set; } = "Label";

 /// <summary>
 /// The data to be used for training
 /// </summary>
 public Var<Microsoft.ML.Runtime.Data.IDataView> TrainingData { get; set; } = new Var<Microsoft.ML.Runtime.Data.IDataView>();

 /// <summary>
 /// Column to use for features
 /// </summary>
 public string FeatureColumn { get; set; } = "Features";

 /// <summary>
 /// Normalize option for the feature column
 /// </summary>
 public Microsoft.ML.Models.NormalizeOption NormalizeFeatures { get; set; } = Microsoft.ML.Models.NormalizeOption.Auto;

 /// <summary>
 /// Whether learner should cache input training data
 /// </summary>
 public Microsoft.ML.Models.CachingOptions Caching { get; set; } = Microsoft.ML.Models.CachingOptions.Auto;


 public sealed class Output : Microsoft.ML.Runtime.EntryPoints.CommonOutputs.IRegressionOutput, Microsoft.ML.Runtime.EntryPoints.CommonOutputs.ITrainerOutput
 {
 /// <summary>
 /// The trained model
 /// </summary>
 public Var<Microsoft.ML.Runtime.EntryPoints.IPredictorModel> PredictorModel { get; set; } = new Var<Microsoft.ML.Runtime.EntryPoints.IPredictorModel>();

 }
 public Var<IDataView> GetInputData() => TrainingData;

 public ILearningPipelineStep ApplyStep(ILearningPipelineStep previousStep, Experiment experiment)
 {
 if (previousStep != null)
 {
 if (!(previousStep is ILearningPipelineDataStep dataStep))
 {
 throw new InvalidOperationException($"{ nameof(OrdinaryLeastSquaresRegressor)} only supports an { nameof(ILearningPipelineDataStep)} as an input.");
 }

 TrainingData = dataStep.Data;
 }
 Output output = experiment.Add(this);
 return new OrdinaryLeastSquaresRegressorPipelineStep(output);
 }

 private class OrdinaryLeastSquaresRegressorPipelineStep : ILearningPipelinePredictorStep
 {
 public OrdinaryLeastSquaresRegressorPipelineStep(Output output)
 {
 Model = output.PredictorModel;
 }

 public Var<IPredictorModel> Model { get; }
 }
 }
 }

 namespace Trainers
 {

Expand Down

1 change: 1 addition & 0 deletions
test/BaselineOutput/Common/EntryPoints/core_ep-list.tsv

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -59,6 +59,7 @@ Trainers.LogisticRegressionBinaryClassifier	Logistic Regression is a method in s
Trainers.LogisticRegressionClassifier	Logistic Regression is a method in statistics used to predict the probability of occurrence of an event and can be used as a classification algorithm. The algorithm predicts the probability of occurrence of an event by fitting data to a logistical function.	Microsoft.ML.Runtime.Learners.LogisticRegression	TrainMultiClass	Microsoft.ML.Runtime.Learners.MulticlassLogisticRegression+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+MulticlassClassificationOutput
Trainers.NaiveBayesClassifier	Train a MultiClassNaiveBayesTrainer.	Microsoft.ML.Runtime.Learners.MultiClassNaiveBayesTrainer	TrainMultiClassNaiveBayesTrainer	Microsoft.ML.Runtime.Learners.MultiClassNaiveBayesTrainer+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+MulticlassClassificationOutput
Trainers.OnlineGradientDescentRegressor	Train a Online gradient descent perceptron.	Microsoft.ML.Runtime.Learners.OnlineGradientDescentTrainer	TrainRegression	Microsoft.ML.Runtime.Learners.OnlineGradientDescentTrainer+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+RegressionOutput
Trainers.OrdinaryLeastSquaresRegressor	Train an OLS regression model.	Microsoft.ML.Runtime.Learners.OlsLinearRegressionTrainer	TrainRegression	Microsoft.ML.Runtime.Learners.OlsLinearRegressionTrainer+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+RegressionOutput
Trainers.PcaAnomalyDetector	Train an PCA Anomaly model.	Microsoft.ML.Runtime.PCA.RandomizedPcaTrainer	TrainPcaAnomaly	Microsoft.ML.Runtime.PCA.RandomizedPcaTrainer+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+AnomalyDetectionOutput
Trainers.PoissonRegressor	Train an Poisson regression model.	Microsoft.ML.Runtime.Learners.PoissonRegression	TrainRegression	Microsoft.ML.Runtime.Learners.PoissonRegression+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+RegressionOutput
Trainers.StochasticDualCoordinateAscentBinaryClassifier	Train an SDCA binary model.	Microsoft.ML.Runtime.Learners.Sdca	TrainBinary	Microsoft.ML.Runtime.Learners.LinearClassificationTrainer+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+BinaryClassificationOutput
Expand Down

143 changes: 143 additions & 0 deletions
test/BaselineOutput/Common/EntryPoints/core_manifest.json

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -14050,6 +14050,149 @@
 "ITrainerOutput"
 ]
 },
 {
 "Name": "Trainers.OrdinaryLeastSquaresRegressor",
 "Desc": "Train an OLS regression model.",
 "FriendlyName": "Ordinary Least Squares (Regression)",
 "ShortName": "ols",
 "Inputs": [
 {
 "Name": "TrainingData",
 "Type": "DataView",
 "Desc": "The data to be used for training",
 "Aliases": [
 "data"
 ],
 "Required": true,
 "SortOrder": 1.0,
 "IsNullable": false
 },
 {
 "Name": "FeatureColumn",
 "Type": "String",
 "Desc": "Column to use for features",
 "Aliases": [
 "feat"
 ],
 "Required": false,
 "SortOrder": 2.0,
 "IsNullable": false,
 "Default": "Features"
 },
 {
 "Name": "LabelColumn",
 "Type": "String",
 "Desc": "Column to use for labels",
 "Aliases": [
 "lab"
 ],
 "Required": false,
 "SortOrder": 3.0,
 "IsNullable": false,
 "Default": "Label"
 },
 {
 "Name": "WeightColumn",
 "Type": "String",
 "Desc": "Column to use for example weight",
 "Aliases": [
 "weight"
 ],
 "Required": false,
 "SortOrder": 4.0,
 "IsNullable": false,
 "Default": "Weight"
 },
 {
 "Name": "NormalizeFeatures",
 "Type": {
 "Kind": "Enum",
 "Values": [
 "No",
 "Warn",
 "Auto",
 "Yes"
 ]
 },
 "Desc": "Normalize option for the feature column",
 "Aliases": [
 "norm"
 ],
 "Required": false,
 "SortOrder": 5.0,
 "IsNullable": false,
 "Default": "Auto"
 },
 {
 "Name": "Caching",
 "Type": {
 "Kind": "Enum",
 "Values": [
 "Auto",
 "Memory",
 "Disk",
 "None"
 ]
 },
 "Desc": "Whether learner should cache input training data",
 "Aliases": [
 "cache"
 ],
 "Required": false,
 "SortOrder": 6.0,
 "IsNullable": false,
 "Default": "Auto"
 },
 {
 "Name": "L2Weight",
 "Type": "Float",
 "Desc": "L2 regularization weight",
 "Aliases": [
 "l2"
 ],
 "Required": false,
 "SortOrder": 50.0,
 "IsNullable": false,
 "Default": 1E-06,
 "SweepRange": {
 "RangeType": "Discrete",
 "Values": [
 1E-06,
 0.1,
 1.0
 ]
 }
 },
 {
 "Name": "PerParameterSignificance",
 "Type": "Bool",
 "Desc": "Whether to calculate per parameter significance statistics",
 "Aliases": [
 "sig"
 ],
 "Required": false,
 "SortOrder": 150.0,
 "IsNullable": false,
 "Default": true
 }
 ],
 "Outputs": [
 {
 "Name": "PredictorModel",
 "Type": "PredictorModel",
 "Desc": "The trained model"
 }
 ],
 "InputKind": [
 "ITrainerInputWithWeight",
 "ITrainerInputWithLabel",
 "ITrainerInput"
 ],
 "OutputKind": [
 "IRegressionOutput",
 "ITrainerOutput"
 ]
 },
 {
 "Name": "Trainers.PcaAnomalyDetector",
 "Desc": "Train an PCA Anomaly model.",
Expand Down

33 changes: 33 additions & 0 deletions
test/BaselineOutput/SingleDebug/OLS/OLS-CV-wine-out.txt

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
maml.exe CV tr=OLS threads=- norm=No dout=%Output% loader=Text{col=Label:R4:11 col=Features:R4:0-10 sep=; header+} data=%Data% seed=1
Not adding a normalizer.
Trainer solving for 12 parameters across 2409 examples
Coefficient of determination R2 = 0.291173667189042, or 0.287920813763543 (adjusted)
Not training a calibrator because it is not needed.
Not adding a normalizer.
Trainer solving for 12 parameters across 2489 examples
Coefficient of determination R2 = 0.280280855195625, or 0.277084686203761 (adjusted)
Not training a calibrator because it is not needed.
L1(avg): 0.586798
L2(avg): 0.573048
RMS(avg): 0.756999
Loss-fn(avg): 0.573048
R Squared: 0.263841
L1(avg): 0.587999
L2(avg): 0.571859
RMS(avg): 0.756214
Loss-fn(avg): 0.571859
R Squared: 0.276072

OVERALL RESULTS
---------------------------------------
L1(avg): 0.587398 (0.0006)
L2(avg): 0.572454 (0.0006)
RMS(avg): 0.756606 (0.0004)
Loss-fn(avg): 0.572454 (0.0006)
R Squared: 0.269956 (0.0061)

---------------------------------------
Physical memory usage(MB): %Number%
Virtual memory usage(MB): %Number%
%DateTime% Time elapsed(s): %Number%

Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default");
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Adding the MlNetMklDeps package, adding the entry point, tests and docs for OLS.#578

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Closed

sfilipi
wants to merge
9
commits into

from

 Closed


Adding the MlNetMklDeps package, adding the entry point, tests and docs for OLS.
#578
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


9 commits


Select commit
Hold shift + click to select a range
9e96175

WIP: adding the MlNetMklDeps package


sfilipi Jul 24, 2018
fa519e8

Fixing the MlNetMklDeps nukpproj


sfilipi Jul 26, 2018
19779fe

reverting test changes to the CSharpCodeGen


sfilipi Jul 26, 2018
94ff6d3

Adding the LoadableClass assembly attribute with the signature:Signat…


sfilipi Jul 26, 2018
dfc2b35

merging from master


sfilipi Jul 26, 2018
cd18b74

following the same naming convention for the doc attributes in tags


sfilipi Jul 27, 2018
80f7bd6

Fixing the path in the OLS doc include


sfilipi Jul 27, 2018
f910923

removing unecessary packing


sfilipi Jul 27, 2018
2965477

In Mac, release only, one of the OLS tests is failing because the act…


sfilipi Jul 27, 2018

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


1 change: 1 addition & 0 deletions
build/Dependencies.props

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -8,5 +8,6 @@
 <SystemReflectionEmitLightweightPackageVersion>4.3.0</SystemReflectionEmitLightweightPackageVersion>
 <PublishSymbolsPackageVersion>1.0.0-beta-62824-02</PublishSymbolsPackageVersion>
 <LightGBMPackageVersion>2.1.2.2</LightGBMPackageVersion>
 <MlNetMklDepsPackageVersion>0.0.0.1</MlNetMklDepsPackageVersion>
 </PropertyGroup>
</Project>

2 changes: 2 additions & 0 deletions
src/Microsoft.ML.StandardLearners/Microsoft.ML.StandardLearners.csproj

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -11,6 +11,8 @@
 <ProjectReference Include="..\Microsoft.ML.CpuMath\Microsoft.ML.CpuMath.csproj" />
 <ProjectReference Include="..\Microsoft.ML.Data\Microsoft.ML.Data.csproj" />
 <ProjectReference Include="..\Microsoft.ML\Microsoft.ML.csproj" />

	<PackageReference Include="MlNetMklDeps" Version="$(MlNetMklDepsPackageVersion)" />
 </ItemGroup>

</Project>

26 changes: 21 additions & 5 deletions
src/Microsoft.ML.StandardLearners/Standard/OlsLinearRegression.cs

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -28,8 +28,11 @@
 "OLS Linear Regression Executor",
 OlsLinearRegressionPredictor.LoaderSignature)]

[assembly: LoadableClass(typeof(void), typeof(OlsLinearRegressionTrainer), null, typeof(SignatureEntryPointModule), OlsLinearRegressionTrainer.LoadNameValue)]

namespace Microsoft.ML.Runtime.Learners
{
 /// <include file='doc.xml' path='doc/members/member[@name="OLS"]/*' />
 public sealed class OlsLinearRegressionTrainer : TrainerBase<OlsLinearRegressionPredictor>
 {
 public sealed class Arguments : LearnerInputBaseWithWeight
Expand All@@ -51,11 +54,6 @@ public sealed class Arguments : LearnerInputBaseWithWeight
 public const string ShortName = "ols";
 internal const string Summary = "The ordinary least square regression fits the target function as a linear function of the numerical features "
 + "that minimizes the square loss function.";
 internal const string Remarks = @"<remarks>
<a href='https://en.wikipedia.org/wiki/Ordinary_least_squares'>Ordinary least squares (OLS)</a> is a parameterized regression method. 
It assumes that the conditional mean of the dependent variable follows a linear function of the dependent variables.
By minimizing the squares of the difference between observed values and the predictions, the parameters of the regressor can be estimated.
</remarks>";

 private readonly Float _l2Weight;
 private readonly bool _perParameterSignificance;
Expand DownExpand Up@@ -463,6 +461,24 @@ public static void Pptri(Layout layout, UpLo uplo, int n, Double[] ap)
 }
 }
 }

 [TlcModule.EntryPoint(Name = "Trainers.OrdinaryLeastSquaresRegressor",
 Desc = "Train an OLS regression model.",
 UserName = UserNameValue,
 ShortName = ShortName,
 XmlInclude = new[] { @"<include file='../Microsoft.ML.StandardLearners/Standard/doc.xml' path='doc/members/member[@name=""OLS""]/*' />" })]
 public static CommonOutputs.RegressionOutput TrainRegression(IHostEnvironment env, Arguments input)
 {
 Contracts.CheckValue(env, nameof(env));
 var host = env.Register("TrainOLS");
 host.CheckValue(input, nameof(input));
 EntryPointUtils.CheckInputArgs(host, input);

 return LearnerEntryPointsUtils.Train<Arguments, CommonOutputs.RegressionOutput>(host, input,
 () => new OlsLinearRegressionTrainer(host, input),
 () => LearnerEntryPointsUtils.FindColumn(host, input.TrainingData.Schema, input.LabelColumn),
 () => LearnerEntryPointsUtils.FindColumn(host, input.TrainingData.Schema, input.WeightColumn));
 }
 }

 /// <summary>
Expand Down

21 changes: 21 additions & 0 deletions
src/Microsoft.ML.StandardLearners/Standard/doc.xml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -68,6 +68,27 @@
 </code>
 </example>
 </example>

 <member name="OLS">
 <summary>
 Train an OLS regression model.
 </summary>
 <remarks>
 <a href='https://en.wikipedia.org/wiki/Ordinary_least_squares'>Ordinary least squares (OLS)</a> is a parameterized regression method. 
 It assumes that the conditional mean of the dependent variable follows a linear function of the dependent variables.
 The parameters of the regressor can be estimated by minimizing the squares of the difference between observed values and the predictions.
 </remarks>
 <example>
 <code language="csharp">
 new OrdinaryLeastSquaresRegressor
 {
 L2Weight = 0.1,
 PerParameterSignificance = false,
 NormalizeFeatures = Microsoft.ML.Models.NormalizeOption.Yes
 }
 </code>
 </example>
 </member>

 </members>
</doc>

99 changes: 99 additions & 0 deletions
src/Microsoft.ML/CSharpApi.cs

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -754,6 +754,18 @@ public void Add(Microsoft.ML.Trainers.OnlineGradientDescentRegressor input, Micr
 _jsonNodes.Add(Serialize("Trainers.OnlineGradientDescentRegressor", input, output));
 }

 public Microsoft.ML.Trainers.OrdinaryLeastSquaresRegressor.Output Add(Microsoft.ML.Trainers.OrdinaryLeastSquaresRegressor input)
 {
 var output = new Microsoft.ML.Trainers.OrdinaryLeastSquaresRegressor.Output();
 Add(input, output);
 return output;
 }

 public void Add(Microsoft.ML.Trainers.OrdinaryLeastSquaresRegressor input, Microsoft.ML.Trainers.OrdinaryLeastSquaresRegressor.Output output)
 {
 _jsonNodes.Add(Serialize("Trainers.OrdinaryLeastSquaresRegressor", input, output));
 }

 public Microsoft.ML.Trainers.PcaAnomalyDetector.Output Add(Microsoft.ML.Trainers.PcaAnomalyDetector input)
 {
 var output = new Microsoft.ML.Trainers.PcaAnomalyDetector.Output();
Expand DownExpand Up@@ -8824,6 +8836,93 @@ public OnlineGradientDescentRegressorPipelineStep(Output output)
 }
 }

 namespace Trainers
 {

 /// <include file='../Microsoft.ML.StandardLearners/Standard/doc.xml' path='doc/members/member[@name="OLS"]/*' />
 public sealed partial class OrdinaryLeastSquaresRegressor : Microsoft.ML.Runtime.EntryPoints.CommonInputs.ITrainerInputWithWeight, Microsoft.ML.Runtime.EntryPoints.CommonInputs.ITrainerInputWithLabel, Microsoft.ML.Runtime.EntryPoints.CommonInputs.ITrainerInput, Microsoft.ML.ILearningPipelineItem
 {


 /// <summary>
 /// L2 regularization weight
 /// </summary>
 [TlcModule.SweepableDiscreteParamAttribute("L2Weight", new object[]{1E-06f, 0.1f, 1f})]
 public float L2Weight { get; set; } = 1E-06f;

 /// <summary>
 /// Whether to calculate per parameter significance statistics
 /// </summary>
 public bool PerParameterSignificance { get; set; } = true;

 /// <summary>
 /// Column to use for example weight
 /// </summary>
 public Microsoft.ML.Runtime.EntryPoints.Optional<string> WeightColumn { get; set; }

 /// <summary>
 /// Column to use for labels
 /// </summary>
 public string LabelColumn { get; set; } = "Label";

 /// <summary>
 /// The data to be used for training
 /// </summary>
 public Var<Microsoft.ML.Runtime.Data.IDataView> TrainingData { get; set; } = new Var<Microsoft.ML.Runtime.Data.IDataView>();

 /// <summary>
 /// Column to use for features
 /// </summary>
 public string FeatureColumn { get; set; } = "Features";

 /// <summary>
 /// Normalize option for the feature column
 /// </summary>
 public Microsoft.ML.Models.NormalizeOption NormalizeFeatures { get; set; } = Microsoft.ML.Models.NormalizeOption.Auto;

 /// <summary>
 /// Whether learner should cache input training data
 /// </summary>
 public Microsoft.ML.Models.CachingOptions Caching { get; set; } = Microsoft.ML.Models.CachingOptions.Auto;


 public sealed class Output : Microsoft.ML.Runtime.EntryPoints.CommonOutputs.IRegressionOutput, Microsoft.ML.Runtime.EntryPoints.CommonOutputs.ITrainerOutput
 {
 /// <summary>
 /// The trained model
 /// </summary>
 public Var<Microsoft.ML.Runtime.EntryPoints.IPredictorModel> PredictorModel { get; set; } = new Var<Microsoft.ML.Runtime.EntryPoints.IPredictorModel>();

 }
 public Var<IDataView> GetInputData() => TrainingData;

 public ILearningPipelineStep ApplyStep(ILearningPipelineStep previousStep, Experiment experiment)
 {
 if (previousStep != null)
 {
 if (!(previousStep is ILearningPipelineDataStep dataStep))
 {
 throw new InvalidOperationException($"{ nameof(OrdinaryLeastSquaresRegressor)} only supports an { nameof(ILearningPipelineDataStep)} as an input.");
 }

 TrainingData = dataStep.Data;
 }
 Output output = experiment.Add(this);
 return new OrdinaryLeastSquaresRegressorPipelineStep(output);
 }

 private class OrdinaryLeastSquaresRegressorPipelineStep : ILearningPipelinePredictorStep
 {
 public OrdinaryLeastSquaresRegressorPipelineStep(Output output)
 {
 Model = output.PredictorModel;
 }

 public Var<IPredictorModel> Model { get; }
 }
 }
 }

 namespace Trainers
 {

Expand Down

1 change: 1 addition & 0 deletions
test/BaselineOutput/Common/EntryPoints/core_ep-list.tsv

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -59,6 +59,7 @@ Trainers.LogisticRegressionBinaryClassifier	Logistic Regression is a method in s
Trainers.LogisticRegressionClassifier	Logistic Regression is a method in statistics used to predict the probability of occurrence of an event and can be used as a classification algorithm. The algorithm predicts the probability of occurrence of an event by fitting data to a logistical function.	Microsoft.ML.Runtime.Learners.LogisticRegression	TrainMultiClass	Microsoft.ML.Runtime.Learners.MulticlassLogisticRegression+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+MulticlassClassificationOutput
Trainers.NaiveBayesClassifier	Train a MultiClassNaiveBayesTrainer.	Microsoft.ML.Runtime.Learners.MultiClassNaiveBayesTrainer	TrainMultiClassNaiveBayesTrainer	Microsoft.ML.Runtime.Learners.MultiClassNaiveBayesTrainer+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+MulticlassClassificationOutput
Trainers.OnlineGradientDescentRegressor	Train a Online gradient descent perceptron.	Microsoft.ML.Runtime.Learners.OnlineGradientDescentTrainer	TrainRegression	Microsoft.ML.Runtime.Learners.OnlineGradientDescentTrainer+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+RegressionOutput
Trainers.OrdinaryLeastSquaresRegressor	Train an OLS regression model.	Microsoft.ML.Runtime.Learners.OlsLinearRegressionTrainer	TrainRegression	Microsoft.ML.Runtime.Learners.OlsLinearRegressionTrainer+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+RegressionOutput
Trainers.PcaAnomalyDetector	Train an PCA Anomaly model.	Microsoft.ML.Runtime.PCA.RandomizedPcaTrainer	TrainPcaAnomaly	Microsoft.ML.Runtime.PCA.RandomizedPcaTrainer+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+AnomalyDetectionOutput
Trainers.PoissonRegressor	Train an Poisson regression model.	Microsoft.ML.Runtime.Learners.PoissonRegression	TrainRegression	Microsoft.ML.Runtime.Learners.PoissonRegression+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+RegressionOutput
Trainers.StochasticDualCoordinateAscentBinaryClassifier	Train an SDCA binary model.	Microsoft.ML.Runtime.Learners.Sdca	TrainBinary	Microsoft.ML.Runtime.Learners.LinearClassificationTrainer+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+BinaryClassificationOutput
Expand Down

143 changes: 143 additions & 0 deletions
test/BaselineOutput/Common/EntryPoints/core_manifest.json

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -14050,6 +14050,149 @@
 "ITrainerOutput"
 ]
 },
 {
 "Name": "Trainers.OrdinaryLeastSquaresRegressor",
 "Desc": "Train an OLS regression model.",
 "FriendlyName": "Ordinary Least Squares (Regression)",
 "ShortName": "ols",
 "Inputs": [
 {
 "Name": "TrainingData",
 "Type": "DataView",
 "Desc": "The data to be used for training",
 "Aliases": [
 "data"
 ],
 "Required": true,
 "SortOrder": 1.0,
 "IsNullable": false
 },
 {
 "Name": "FeatureColumn",
 "Type": "String",
 "Desc": "Column to use for features",
 "Aliases": [
 "feat"
 ],
 "Required": false,
 "SortOrder": 2.0,
 "IsNullable": false,
 "Default": "Features"
 },
 {
 "Name": "LabelColumn",
 "Type": "String",
 "Desc": "Column to use for labels",
 "Aliases": [
 "lab"
 ],
 "Required": false,
 "SortOrder": 3.0,
 "IsNullable": false,
 "Default": "Label"
 },
 {
 "Name": "WeightColumn",
 "Type": "String",
 "Desc": "Column to use for example weight",
 "Aliases": [
 "weight"
 ],
 "Required": false,
 "SortOrder": 4.0,
 "IsNullable": false,
 "Default": "Weight"
 },
 {
 "Name": "NormalizeFeatures",
 "Type": {
 "Kind": "Enum",
 "Values": [
 "No",
 "Warn",
 "Auto",
 "Yes"
 ]
 },
 "Desc": "Normalize option for the feature column",
 "Aliases": [
 "norm"
 ],
 "Required": false,
 "SortOrder": 5.0,
 "IsNullable": false,
 "Default": "Auto"
 },
 {
 "Name": "Caching",
 "Type": {
 "Kind": "Enum",
 "Values": [
 "Auto",
 "Memory",
 "Disk",
 "None"
 ]
 },
 "Desc": "Whether learner should cache input training data",
 "Aliases": [
 "cache"
 ],
 "Required": false,
 "SortOrder": 6.0,
 "IsNullable": false,
 "Default": "Auto"
 },
 {
 "Name": "L2Weight",
 "Type": "Float",
 "Desc": "L2 regularization weight",
 "Aliases": [
 "l2"
 ],
 "Required": false,
 "SortOrder": 50.0,
 "IsNullable": false,
 "Default": 1E-06,
 "SweepRange": {
 "RangeType": "Discrete",
 "Values": [
 1E-06,
 0.1,
 1.0
 ]
 }
 },
 {
 "Name": "PerParameterSignificance",
 "Type": "Bool",
 "Desc": "Whether to calculate per parameter significance statistics",
 "Aliases": [
 "sig"
 ],
 "Required": false,
 "SortOrder": 150.0,
 "IsNullable": false,
 "Default": true
 }
 ],
 "Outputs": [
 {
 "Name": "PredictorModel",
 "Type": "PredictorModel",
 "Desc": "The trained model"
 }
 ],
 "InputKind": [
 "ITrainerInputWithWeight",
 "ITrainerInputWithLabel",
 "ITrainerInput"
 ],
 "OutputKind": [
 "IRegressionOutput",
 "ITrainerOutput"
 ]
 },
 {
 "Name": "Trainers.PcaAnomalyDetector",
 "Desc": "Train an PCA Anomaly model.",
Expand Down

33 changes: 33 additions & 0 deletions
test/BaselineOutput/SingleDebug/OLS/OLS-CV-wine-out.txt

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
maml.exe CV tr=OLS threads=- norm=No dout=%Output% loader=Text{col=Label:R4:11 col=Features:R4:0-10 sep=; header+} data=%Data% seed=1
Not adding a normalizer.
Trainer solving for 12 parameters across 2409 examples
Coefficient of determination R2 = 0.291173667189042, or 0.287920813763543 (adjusted)
Not training a calibrator because it is not needed.
Not adding a normalizer.
Trainer solving for 12 parameters across 2489 examples
Coefficient of determination R2 = 0.280280855195625, or 0.277084686203761 (adjusted)
Not training a calibrator because it is not needed.
L1(avg): 0.586798
L2(avg): 0.573048
RMS(avg): 0.756999
Loss-fn(avg): 0.573048
R Squared: 0.263841
L1(avg): 0.587999
L2(avg): 0.571859
RMS(avg): 0.756214
Loss-fn(avg): 0.571859
R Squared: 0.276072

OVERALL RESULTS
---------------------------------------
L1(avg): 0.587398 (0.0006)
L2(avg): 0.572454 (0.0006)
RMS(avg): 0.756606 (0.0004)
Loss-fn(avg): 0.572454 (0.0006)
R Squared: 0.269956 (0.0061)

---------------------------------------
Physical memory usage(MB): %Number%
Virtual memory usage(MB): %Number%
%DateTime% Time elapsed(s): %Number%

Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers");
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Adding the MlNetMklDeps package, adding the entry point, tests and docs for OLS.#578

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Closed

sfilipi
wants to merge
9
commits into

from

 Closed


Adding the MlNetMklDeps package, adding the entry point, tests and docs for OLS.
#578
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


9 commits


Select commit
Hold shift + click to select a range
9e96175

WIP: adding the MlNetMklDeps package


sfilipi Jul 24, 2018
fa519e8

Fixing the MlNetMklDeps nukpproj


sfilipi Jul 26, 2018
19779fe

reverting test changes to the CSharpCodeGen


sfilipi Jul 26, 2018
94ff6d3

Adding the LoadableClass assembly attribute with the signature:Signat…


sfilipi Jul 26, 2018
dfc2b35

merging from master


sfilipi Jul 26, 2018
cd18b74

following the same naming convention for the doc attributes in tags


sfilipi Jul 27, 2018
80f7bd6

Fixing the path in the OLS doc include


sfilipi Jul 27, 2018
f910923

removing unecessary packing


sfilipi Jul 27, 2018
2965477

In Mac, release only, one of the OLS tests is failing because the act…


sfilipi Jul 27, 2018

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


1 change: 1 addition & 0 deletions
build/Dependencies.props

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -8,5 +8,6 @@
 <SystemReflectionEmitLightweightPackageVersion>4.3.0</SystemReflectionEmitLightweightPackageVersion>
 <PublishSymbolsPackageVersion>1.0.0-beta-62824-02</PublishSymbolsPackageVersion>
 <LightGBMPackageVersion>2.1.2.2</LightGBMPackageVersion>
 <MlNetMklDepsPackageVersion>0.0.0.1</MlNetMklDepsPackageVersion>
 </PropertyGroup>
</Project>

2 changes: 2 additions & 0 deletions
src/Microsoft.ML.StandardLearners/Microsoft.ML.StandardLearners.csproj

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -11,6 +11,8 @@
 <ProjectReference Include="..\Microsoft.ML.CpuMath\Microsoft.ML.CpuMath.csproj" />
 <ProjectReference Include="..\Microsoft.ML.Data\Microsoft.ML.Data.csproj" />
 <ProjectReference Include="..\Microsoft.ML\Microsoft.ML.csproj" />

	<PackageReference Include="MlNetMklDeps" Version="$(MlNetMklDepsPackageVersion)" />
 </ItemGroup>

</Project>

26 changes: 21 additions & 5 deletions
src/Microsoft.ML.StandardLearners/Standard/OlsLinearRegression.cs

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -28,8 +28,11 @@
 "OLS Linear Regression Executor",
 OlsLinearRegressionPredictor.LoaderSignature)]

[assembly: LoadableClass(typeof(void), typeof(OlsLinearRegressionTrainer), null, typeof(SignatureEntryPointModule), OlsLinearRegressionTrainer.LoadNameValue)]

namespace Microsoft.ML.Runtime.Learners
{
 /// <include file='doc.xml' path='doc/members/member[@name="OLS"]/*' />
 public sealed class OlsLinearRegressionTrainer : TrainerBase<OlsLinearRegressionPredictor>
 {
 public sealed class Arguments : LearnerInputBaseWithWeight
Expand All@@ -51,11 +54,6 @@ public sealed class Arguments : LearnerInputBaseWithWeight
 public const string ShortName = "ols";
 internal const string Summary = "The ordinary least square regression fits the target function as a linear function of the numerical features "
 + "that minimizes the square loss function.";
 internal const string Remarks = @"<remarks>
<a href='https://en.wikipedia.org/wiki/Ordinary_least_squares'>Ordinary least squares (OLS)</a> is a parameterized regression method. 
It assumes that the conditional mean of the dependent variable follows a linear function of the dependent variables.
By minimizing the squares of the difference between observed values and the predictions, the parameters of the regressor can be estimated.
</remarks>";

 private readonly Float _l2Weight;
 private readonly bool _perParameterSignificance;
Expand DownExpand Up@@ -463,6 +461,24 @@ public static void Pptri(Layout layout, UpLo uplo, int n, Double[] ap)
 }
 }
 }

 [TlcModule.EntryPoint(Name = "Trainers.OrdinaryLeastSquaresRegressor",
 Desc = "Train an OLS regression model.",
 UserName = UserNameValue,
 ShortName = ShortName,
 XmlInclude = new[] { @"<include file='../Microsoft.ML.StandardLearners/Standard/doc.xml' path='doc/members/member[@name=""OLS""]/*' />" })]
 public static CommonOutputs.RegressionOutput TrainRegression(IHostEnvironment env, Arguments input)
 {
 Contracts.CheckValue(env, nameof(env));
 var host = env.Register("TrainOLS");
 host.CheckValue(input, nameof(input));
 EntryPointUtils.CheckInputArgs(host, input);

 return LearnerEntryPointsUtils.Train<Arguments, CommonOutputs.RegressionOutput>(host, input,
 () => new OlsLinearRegressionTrainer(host, input),
 () => LearnerEntryPointsUtils.FindColumn(host, input.TrainingData.Schema, input.LabelColumn),
 () => LearnerEntryPointsUtils.FindColumn(host, input.TrainingData.Schema, input.WeightColumn));
 }
 }

 /// <summary>
Expand Down

21 changes: 21 additions & 0 deletions
src/Microsoft.ML.StandardLearners/Standard/doc.xml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -68,6 +68,27 @@
 </code>
 </example>
 </example>

 <member name="OLS">
 <summary>
 Train an OLS regression model.
 </summary>
 <remarks>
 <a href='https://en.wikipedia.org/wiki/Ordinary_least_squares'>Ordinary least squares (OLS)</a> is a parameterized regression method. 
 It assumes that the conditional mean of the dependent variable follows a linear function of the dependent variables.
 The parameters of the regressor can be estimated by minimizing the squares of the difference between observed values and the predictions.
 </remarks>
 <example>
 <code language="csharp">
 new OrdinaryLeastSquaresRegressor
 {
 L2Weight = 0.1,
 PerParameterSignificance = false,
 NormalizeFeatures = Microsoft.ML.Models.NormalizeOption.Yes
 }
 </code>
 </example>
 </member>

 </members>
</doc>

99 changes: 99 additions & 0 deletions
src/Microsoft.ML/CSharpApi.cs

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -754,6 +754,18 @@ public void Add(Microsoft.ML.Trainers.OnlineGradientDescentRegressor input, Micr
 _jsonNodes.Add(Serialize("Trainers.OnlineGradientDescentRegressor", input, output));
 }

 public Microsoft.ML.Trainers.OrdinaryLeastSquaresRegressor.Output Add(Microsoft.ML.Trainers.OrdinaryLeastSquaresRegressor input)
 {
 var output = new Microsoft.ML.Trainers.OrdinaryLeastSquaresRegressor.Output();
 Add(input, output);
 return output;
 }

 public void Add(Microsoft.ML.Trainers.OrdinaryLeastSquaresRegressor input, Microsoft.ML.Trainers.OrdinaryLeastSquaresRegressor.Output output)
 {
 _jsonNodes.Add(Serialize("Trainers.OrdinaryLeastSquaresRegressor", input, output));
 }

 public Microsoft.ML.Trainers.PcaAnomalyDetector.Output Add(Microsoft.ML.Trainers.PcaAnomalyDetector input)
 {
 var output = new Microsoft.ML.Trainers.PcaAnomalyDetector.Output();
Expand DownExpand Up@@ -8824,6 +8836,93 @@ public OnlineGradientDescentRegressorPipelineStep(Output output)
 }
 }

 namespace Trainers
 {

 /// <include file='../Microsoft.ML.StandardLearners/Standard/doc.xml' path='doc/members/member[@name="OLS"]/*' />
 public sealed partial class OrdinaryLeastSquaresRegressor : Microsoft.ML.Runtime.EntryPoints.CommonInputs.ITrainerInputWithWeight, Microsoft.ML.Runtime.EntryPoints.CommonInputs.ITrainerInputWithLabel, Microsoft.ML.Runtime.EntryPoints.CommonInputs.ITrainerInput, Microsoft.ML.ILearningPipelineItem
 {


 /// <summary>
 /// L2 regularization weight
 /// </summary>
 [TlcModule.SweepableDiscreteParamAttribute("L2Weight", new object[]{1E-06f, 0.1f, 1f})]
 public float L2Weight { get; set; } = 1E-06f;

 /// <summary>
 /// Whether to calculate per parameter significance statistics
 /// </summary>
 public bool PerParameterSignificance { get; set; } = true;

 /// <summary>
 /// Column to use for example weight
 /// </summary>
 public Microsoft.ML.Runtime.EntryPoints.Optional<string> WeightColumn { get; set; }

 /// <summary>
 /// Column to use for labels
 /// </summary>
 public string LabelColumn { get; set; } = "Label";

 /// <summary>
 /// The data to be used for training
 /// </summary>
 public Var<Microsoft.ML.Runtime.Data.IDataView> TrainingData { get; set; } = new Var<Microsoft.ML.Runtime.Data.IDataView>();

 /// <summary>
 /// Column to use for features
 /// </summary>
 public string FeatureColumn { get; set; } = "Features";

 /// <summary>
 /// Normalize option for the feature column
 /// </summary>
 public Microsoft.ML.Models.NormalizeOption NormalizeFeatures { get; set; } = Microsoft.ML.Models.NormalizeOption.Auto;

 /// <summary>
 /// Whether learner should cache input training data
 /// </summary>
 public Microsoft.ML.Models.CachingOptions Caching { get; set; } = Microsoft.ML.Models.CachingOptions.Auto;


 public sealed class Output : Microsoft.ML.Runtime.EntryPoints.CommonOutputs.IRegressionOutput, Microsoft.ML.Runtime.EntryPoints.CommonOutputs.ITrainerOutput
 {
 /// <summary>
 /// The trained model
 /// </summary>
 public Var<Microsoft.ML.Runtime.EntryPoints.IPredictorModel> PredictorModel { get; set; } = new Var<Microsoft.ML.Runtime.EntryPoints.IPredictorModel>();

 }
 public Var<IDataView> GetInputData() => TrainingData;

 public ILearningPipelineStep ApplyStep(ILearningPipelineStep previousStep, Experiment experiment)
 {
 if (previousStep != null)
 {
 if (!(previousStep is ILearningPipelineDataStep dataStep))
 {
 throw new InvalidOperationException($"{ nameof(OrdinaryLeastSquaresRegressor)} only supports an { nameof(ILearningPipelineDataStep)} as an input.");
 }

 TrainingData = dataStep.Data;
 }
 Output output = experiment.Add(this);
 return new OrdinaryLeastSquaresRegressorPipelineStep(output);
 }

 private class OrdinaryLeastSquaresRegressorPipelineStep : ILearningPipelinePredictorStep
 {
 public OrdinaryLeastSquaresRegressorPipelineStep(Output output)
 {
 Model = output.PredictorModel;
 }

 public Var<IPredictorModel> Model { get; }
 }
 }
 }

 namespace Trainers
 {

Expand Down

1 change: 1 addition & 0 deletions
test/BaselineOutput/Common/EntryPoints/core_ep-list.tsv

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -59,6 +59,7 @@ Trainers.LogisticRegressionBinaryClassifier	Logistic Regression is a method in s
Trainers.LogisticRegressionClassifier	Logistic Regression is a method in statistics used to predict the probability of occurrence of an event and can be used as a classification algorithm. The algorithm predicts the probability of occurrence of an event by fitting data to a logistical function.	Microsoft.ML.Runtime.Learners.LogisticRegression	TrainMultiClass	Microsoft.ML.Runtime.Learners.MulticlassLogisticRegression+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+MulticlassClassificationOutput
Trainers.NaiveBayesClassifier	Train a MultiClassNaiveBayesTrainer.	Microsoft.ML.Runtime.Learners.MultiClassNaiveBayesTrainer	TrainMultiClassNaiveBayesTrainer	Microsoft.ML.Runtime.Learners.MultiClassNaiveBayesTrainer+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+MulticlassClassificationOutput
Trainers.OnlineGradientDescentRegressor	Train a Online gradient descent perceptron.	Microsoft.ML.Runtime.Learners.OnlineGradientDescentTrainer	TrainRegression	Microsoft.ML.Runtime.Learners.OnlineGradientDescentTrainer+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+RegressionOutput
Trainers.OrdinaryLeastSquaresRegressor	Train an OLS regression model.	Microsoft.ML.Runtime.Learners.OlsLinearRegressionTrainer	TrainRegression	Microsoft.ML.Runtime.Learners.OlsLinearRegressionTrainer+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+RegressionOutput
Trainers.PcaAnomalyDetector	Train an PCA Anomaly model.	Microsoft.ML.Runtime.PCA.RandomizedPcaTrainer	TrainPcaAnomaly	Microsoft.ML.Runtime.PCA.RandomizedPcaTrainer+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+AnomalyDetectionOutput
Trainers.PoissonRegressor	Train an Poisson regression model.	Microsoft.ML.Runtime.Learners.PoissonRegression	TrainRegression	Microsoft.ML.Runtime.Learners.PoissonRegression+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+RegressionOutput
Trainers.StochasticDualCoordinateAscentBinaryClassifier	Train an SDCA binary model.	Microsoft.ML.Runtime.Learners.Sdca	TrainBinary	Microsoft.ML.Runtime.Learners.LinearClassificationTrainer+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+BinaryClassificationOutput
Expand Down

143 changes: 143 additions & 0 deletions
test/BaselineOutput/Common/EntryPoints/core_manifest.json

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -14050,6 +14050,149 @@
 "ITrainerOutput"
 ]
 },
 {
 "Name": "Trainers.OrdinaryLeastSquaresRegressor",
 "Desc": "Train an OLS regression model.",
 "FriendlyName": "Ordinary Least Squares (Regression)",
 "ShortName": "ols",
 "Inputs": [
 {
 "Name": "TrainingData",
 "Type": "DataView",
 "Desc": "The data to be used for training",
 "Aliases": [
 "data"
 ],
 "Required": true,
 "SortOrder": 1.0,
 "IsNullable": false
 },
 {
 "Name": "FeatureColumn",
 "Type": "String",
 "Desc": "Column to use for features",
 "Aliases": [
 "feat"
 ],
 "Required": false,
 "SortOrder": 2.0,
 "IsNullable": false,
 "Default": "Features"
 },
 {
 "Name": "LabelColumn",
 "Type": "String",
 "Desc": "Column to use for labels",
 "Aliases": [
 "lab"
 ],
 "Required": false,
 "SortOrder": 3.0,
 "IsNullable": false,
 "Default": "Label"
 },
 {
 "Name": "WeightColumn",
 "Type": "String",
 "Desc": "Column to use for example weight",
 "Aliases": [
 "weight"
 ],
 "Required": false,
 "SortOrder": 4.0,
 "IsNullable": false,
 "Default": "Weight"
 },
 {
 "Name": "NormalizeFeatures",
 "Type": {
 "Kind": "Enum",
 "Values": [
 "No",
 "Warn",
 "Auto",
 "Yes"
 ]
 },
 "Desc": "Normalize option for the feature column",
 "Aliases": [
 "norm"
 ],
 "Required": false,
 "SortOrder": 5.0,
 "IsNullable": false,
 "Default": "Auto"
 },
 {
 "Name": "Caching",
 "Type": {
 "Kind": "Enum",
 "Values": [
 "Auto",
 "Memory",
 "Disk",
 "None"
 ]
 },
 "Desc": "Whether learner should cache input training data",
 "Aliases": [
 "cache"
 ],
 "Required": false,
 "SortOrder": 6.0,
 "IsNullable": false,
 "Default": "Auto"
 },
 {
 "Name": "L2Weight",
 "Type": "Float",
 "Desc": "L2 regularization weight",
 "Aliases": [
 "l2"
 ],
 "Required": false,
 "SortOrder": 50.0,
 "IsNullable": false,
 "Default": 1E-06,
 "SweepRange": {
 "RangeType": "Discrete",
 "Values": [
 1E-06,
 0.1,
 1.0
 ]
 }
 },
 {
 "Name": "PerParameterSignificance",
 "Type": "Bool",
 "Desc": "Whether to calculate per parameter significance statistics",
 "Aliases": [
 "sig"
 ],
 "Required": false,
 "SortOrder": 150.0,
 "IsNullable": false,
 "Default": true
 }
 ],
 "Outputs": [
 {
 "Name": "PredictorModel",
 "Type": "PredictorModel",
 "Desc": "The trained model"
 }
 ],
 "InputKind": [
 "ITrainerInputWithWeight",
 "ITrainerInputWithLabel",
 "ITrainerInput"
 ],
 "OutputKind": [
 "IRegressionOutput",
 "ITrainerOutput"
 ]
 },
 {
 "Name": "Trainers.PcaAnomalyDetector",
 "Desc": "Train an PCA Anomaly model.",
Expand Down

33 changes: 33 additions & 0 deletions
test/BaselineOutput/SingleDebug/OLS/OLS-CV-wine-out.txt

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
maml.exe CV tr=OLS threads=- norm=No dout=%Output% loader=Text{col=Label:R4:11 col=Features:R4:0-10 sep=; header+} data=%Data% seed=1
Not adding a normalizer.
Trainer solving for 12 parameters across 2409 examples
Coefficient of determination R2 = 0.291173667189042, or 0.287920813763543 (adjusted)
Not training a calibrator because it is not needed.
Not adding a normalizer.
Trainer solving for 12 parameters across 2489 examples
Coefficient of determination R2 = 0.280280855195625, or 0.277084686203761 (adjusted)
Not training a calibrator because it is not needed.
L1(avg): 0.586798
L2(avg): 0.573048
RMS(avg): 0.756999
Loss-fn(avg): 0.573048
R Squared: 0.263841
L1(avg): 0.587999
L2(avg): 0.571859
RMS(avg): 0.756214
Loss-fn(avg): 0.571859
R Squared: 0.276072

OVERALL RESULTS
---------------------------------------
L1(avg): 0.587398 (0.0006)
L2(avg): 0.572454 (0.0006)
RMS(avg): 0.756606 (0.0004)
Loss-fn(avg): 0.572454 (0.0006)
R Squared: 0.269956 (0.0061)

---------------------------------------
Physical memory usage(MB): %Number%
Virtual memory usage(MB): %Number%
%DateTime% Time elapsed(s): %Number%

Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode");
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();

Skip to content
Adding the MlNetMklDeps package, adding the entry point, tests and docs for OLS.#578

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Closed

sfilipi
wants to merge
9
commits into

from

 Closed


Adding the MlNetMklDeps package, adding the entry point, tests and docs for OLS.
#578
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


9 commits


Select commit
Hold shift + click to select a range
9e96175

WIP: adding the MlNetMklDeps package


sfilipi Jul 24, 2018
fa519e8

Fixing the MlNetMklDeps nukpproj


sfilipi Jul 26, 2018
19779fe

reverting test changes to the CSharpCodeGen


sfilipi Jul 26, 2018
94ff6d3

Adding the LoadableClass assembly attribute with the signature:Signat…


sfilipi Jul 26, 2018
dfc2b35

merging from master


sfilipi Jul 26, 2018
cd18b74

following the same naming convention for the doc attributes in tags


sfilipi Jul 27, 2018
80f7bd6

Fixing the path in the OLS doc include


sfilipi Jul 27, 2018
f910923

removing unecessary packing


sfilipi Jul 27, 2018
2965477

In Mac, release only, one of the OLS tests is failing because the act…


sfilipi Jul 27, 2018

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


1 change: 1 addition & 0 deletions
build/Dependencies.props

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -8,5 +8,6 @@
 <SystemReflectionEmitLightweightPackageVersion>4.3.0</SystemReflectionEmitLightweightPackageVersion>
 <PublishSymbolsPackageVersion>1.0.0-beta-62824-02</PublishSymbolsPackageVersion>
 <LightGBMPackageVersion>2.1.2.2</LightGBMPackageVersion>
 <MlNetMklDepsPackageVersion>0.0.0.1</MlNetMklDepsPackageVersion>
 </PropertyGroup>
</Project>

2 changes: 2 additions & 0 deletions
src/Microsoft.ML.StandardLearners/Microsoft.ML.StandardLearners.csproj

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -11,6 +11,8 @@
 <ProjectReference Include="..\Microsoft.ML.CpuMath\Microsoft.ML.CpuMath.csproj" />
 <ProjectReference Include="..\Microsoft.ML.Data\Microsoft.ML.Data.csproj" />
 <ProjectReference Include="..\Microsoft.ML\Microsoft.ML.csproj" />

	<PackageReference Include="MlNetMklDeps" Version="$(MlNetMklDepsPackageVersion)" />
 </ItemGroup>

</Project>

26 changes: 21 additions & 5 deletions
src/Microsoft.ML.StandardLearners/Standard/OlsLinearRegression.cs

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -28,8 +28,11 @@
 "OLS Linear Regression Executor",
 OlsLinearRegressionPredictor.LoaderSignature)]

[assembly: LoadableClass(typeof(void), typeof(OlsLinearRegressionTrainer), null, typeof(SignatureEntryPointModule), OlsLinearRegressionTrainer.LoadNameValue)]

namespace Microsoft.ML.Runtime.Learners
{
 /// <include file='doc.xml' path='doc/members/member[@name="OLS"]/*' />
 public sealed class OlsLinearRegressionTrainer : TrainerBase<OlsLinearRegressionPredictor>
 {
 public sealed class Arguments : LearnerInputBaseWithWeight
Expand All@@ -51,11 +54,6 @@ public sealed class Arguments : LearnerInputBaseWithWeight
 public const string ShortName = "ols";
 internal const string Summary = "The ordinary least square regression fits the target function as a linear function of the numerical features "
 + "that minimizes the square loss function.";
 internal const string Remarks = @"<remarks>
<a href='https://en.wikipedia.org/wiki/Ordinary_least_squares'>Ordinary least squares (OLS)</a> is a parameterized regression method. 
It assumes that the conditional mean of the dependent variable follows a linear function of the dependent variables.
By minimizing the squares of the difference between observed values and the predictions, the parameters of the regressor can be estimated.
</remarks>";

 private readonly Float _l2Weight;
 private readonly bool _perParameterSignificance;
Expand DownExpand Up@@ -463,6 +461,24 @@ public static void Pptri(Layout layout, UpLo uplo, int n, Double[] ap)
 }
 }
 }

 [TlcModule.EntryPoint(Name = "Trainers.OrdinaryLeastSquaresRegressor",
 Desc = "Train an OLS regression model.",
 UserName = UserNameValue,
 ShortName = ShortName,
 XmlInclude = new[] { @"<include file='../Microsoft.ML.StandardLearners/Standard/doc.xml' path='doc/members/member[@name=""OLS""]/*' />" })]
 public static CommonOutputs.RegressionOutput TrainRegression(IHostEnvironment env, Arguments input)
 {
 Contracts.CheckValue(env, nameof(env));
 var host = env.Register("TrainOLS");
 host.CheckValue(input, nameof(input));
 EntryPointUtils.CheckInputArgs(host, input);

 return LearnerEntryPointsUtils.Train<Arguments, CommonOutputs.RegressionOutput>(host, input,
 () => new OlsLinearRegressionTrainer(host, input),
 () => LearnerEntryPointsUtils.FindColumn(host, input.TrainingData.Schema, input.LabelColumn),
 () => LearnerEntryPointsUtils.FindColumn(host, input.TrainingData.Schema, input.WeightColumn));
 }
 }

 /// <summary>
Expand Down

21 changes: 21 additions & 0 deletions
src/Microsoft.ML.StandardLearners/Standard/doc.xml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -68,6 +68,27 @@
 </code>
 </example>
 </example>

 <member name="OLS">
 <summary>
 Train an OLS regression model.
 </summary>
 <remarks>
 <a href='https://en.wikipedia.org/wiki/Ordinary_least_squares'>Ordinary least squares (OLS)</a> is a parameterized regression method. 
 It assumes that the conditional mean of the dependent variable follows a linear function of the dependent variables.
 The parameters of the regressor can be estimated by minimizing the squares of the difference between observed values and the predictions.
 </remarks>
 <example>
 <code language="csharp">
 new OrdinaryLeastSquaresRegressor
 {
 L2Weight = 0.1,
 PerParameterSignificance = false,
 NormalizeFeatures = Microsoft.ML.Models.NormalizeOption.Yes
 }
 </code>
 </example>
 </member>

 </members>
</doc>

99 changes: 99 additions & 0 deletions
src/Microsoft.ML/CSharpApi.cs

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -754,6 +754,18 @@ public void Add(Microsoft.ML.Trainers.OnlineGradientDescentRegressor input, Micr
 _jsonNodes.Add(Serialize("Trainers.OnlineGradientDescentRegressor", input, output));
 }

 public Microsoft.ML.Trainers.OrdinaryLeastSquaresRegressor.Output Add(Microsoft.ML.Trainers.OrdinaryLeastSquaresRegressor input)
 {
 var output = new Microsoft.ML.Trainers.OrdinaryLeastSquaresRegressor.Output();
 Add(input, output);
 return output;
 }

 public void Add(Microsoft.ML.Trainers.OrdinaryLeastSquaresRegressor input, Microsoft.ML.Trainers.OrdinaryLeastSquaresRegressor.Output output)
 {
 _jsonNodes.Add(Serialize("Trainers.OrdinaryLeastSquaresRegressor", input, output));
 }

 public Microsoft.ML.Trainers.PcaAnomalyDetector.Output Add(Microsoft.ML.Trainers.PcaAnomalyDetector input)
 {
 var output = new Microsoft.ML.Trainers.PcaAnomalyDetector.Output();
Expand DownExpand Up@@ -8824,6 +8836,93 @@ public OnlineGradientDescentRegressorPipelineStep(Output output)
 }
 }

 namespace Trainers
 {

 /// <include file='../Microsoft.ML.StandardLearners/Standard/doc.xml' path='doc/members/member[@name="OLS"]/*' />
 public sealed partial class OrdinaryLeastSquaresRegressor : Microsoft.ML.Runtime.EntryPoints.CommonInputs.ITrainerInputWithWeight, Microsoft.ML.Runtime.EntryPoints.CommonInputs.ITrainerInputWithLabel, Microsoft.ML.Runtime.EntryPoints.CommonInputs.ITrainerInput, Microsoft.ML.ILearningPipelineItem
 {


 /// <summary>
 /// L2 regularization weight
 /// </summary>
 [TlcModule.SweepableDiscreteParamAttribute("L2Weight", new object[]{1E-06f, 0.1f, 1f})]
 public float L2Weight { get; set; } = 1E-06f;

 /// <summary>
 /// Whether to calculate per parameter significance statistics
 /// </summary>
 public bool PerParameterSignificance { get; set; } = true;

 /// <summary>
 /// Column to use for example weight
 /// </summary>
 public Microsoft.ML.Runtime.EntryPoints.Optional<string> WeightColumn { get; set; }

 /// <summary>
 /// Column to use for labels
 /// </summary>
 public string LabelColumn { get; set; } = "Label";

 /// <summary>
 /// The data to be used for training
 /// </summary>
 public Var<Microsoft.ML.Runtime.Data.IDataView> TrainingData { get; set; } = new Var<Microsoft.ML.Runtime.Data.IDataView>();

 /// <summary>
 /// Column to use for features
 /// </summary>
 public string FeatureColumn { get; set; } = "Features";

 /// <summary>
 /// Normalize option for the feature column
 /// </summary>
 public Microsoft.ML.Models.NormalizeOption NormalizeFeatures { get; set; } = Microsoft.ML.Models.NormalizeOption.Auto;

 /// <summary>
 /// Whether learner should cache input training data
 /// </summary>
 public Microsoft.ML.Models.CachingOptions Caching { get; set; } = Microsoft.ML.Models.CachingOptions.Auto;


 public sealed class Output : Microsoft.ML.Runtime.EntryPoints.CommonOutputs.IRegressionOutput, Microsoft.ML.Runtime.EntryPoints.CommonOutputs.ITrainerOutput
 {
 /// <summary>
 /// The trained model
 /// </summary>
 public Var<Microsoft.ML.Runtime.EntryPoints.IPredictorModel> PredictorModel { get; set; } = new Var<Microsoft.ML.Runtime.EntryPoints.IPredictorModel>();

 }
 public Var<IDataView> GetInputData() => TrainingData;

 public ILearningPipelineStep ApplyStep(ILearningPipelineStep previousStep, Experiment experiment)
 {
 if (previousStep != null)
 {
 if (!(previousStep is ILearningPipelineDataStep dataStep))
 {
 throw new InvalidOperationException($"{ nameof(OrdinaryLeastSquaresRegressor)} only supports an { nameof(ILearningPipelineDataStep)} as an input.");
 }

 TrainingData = dataStep.Data;
 }
 Output output = experiment.Add(this);
 return new OrdinaryLeastSquaresRegressorPipelineStep(output);
 }

 private class OrdinaryLeastSquaresRegressorPipelineStep : ILearningPipelinePredictorStep
 {
 public OrdinaryLeastSquaresRegressorPipelineStep(Output output)
 {
 Model = output.PredictorModel;
 }

 public Var<IPredictorModel> Model { get; }
 }
 }
 }

 namespace Trainers
 {

Expand Down

1 change: 1 addition & 0 deletions
test/BaselineOutput/Common/EntryPoints/core_ep-list.tsv

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -59,6 +59,7 @@ Trainers.LogisticRegressionBinaryClassifier	Logistic Regression is a method in s
Trainers.LogisticRegressionClassifier	Logistic Regression is a method in statistics used to predict the probability of occurrence of an event and can be used as a classification algorithm. The algorithm predicts the probability of occurrence of an event by fitting data to a logistical function.	Microsoft.ML.Runtime.Learners.LogisticRegression	TrainMultiClass	Microsoft.ML.Runtime.Learners.MulticlassLogisticRegression+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+MulticlassClassificationOutput
Trainers.NaiveBayesClassifier	Train a MultiClassNaiveBayesTrainer.	Microsoft.ML.Runtime.Learners.MultiClassNaiveBayesTrainer	TrainMultiClassNaiveBayesTrainer	Microsoft.ML.Runtime.Learners.MultiClassNaiveBayesTrainer+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+MulticlassClassificationOutput
Trainers.OnlineGradientDescentRegressor	Train a Online gradient descent perceptron.	Microsoft.ML.Runtime.Learners.OnlineGradientDescentTrainer	TrainRegression	Microsoft.ML.Runtime.Learners.OnlineGradientDescentTrainer+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+RegressionOutput
Trainers.OrdinaryLeastSquaresRegressor	Train an OLS regression model.	Microsoft.ML.Runtime.Learners.OlsLinearRegressionTrainer	TrainRegression	Microsoft.ML.Runtime.Learners.OlsLinearRegressionTrainer+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+RegressionOutput
Trainers.PcaAnomalyDetector	Train an PCA Anomaly model.	Microsoft.ML.Runtime.PCA.RandomizedPcaTrainer	TrainPcaAnomaly	Microsoft.ML.Runtime.PCA.RandomizedPcaTrainer+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+AnomalyDetectionOutput
Trainers.PoissonRegressor	Train an Poisson regression model.	Microsoft.ML.Runtime.Learners.PoissonRegression	TrainRegression	Microsoft.ML.Runtime.Learners.PoissonRegression+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+RegressionOutput
Trainers.StochasticDualCoordinateAscentBinaryClassifier	Train an SDCA binary model.	Microsoft.ML.Runtime.Learners.Sdca	TrainBinary	Microsoft.ML.Runtime.Learners.LinearClassificationTrainer+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+BinaryClassificationOutput
Expand Down

143 changes: 143 additions & 0 deletions
test/BaselineOutput/Common/EntryPoints/core_manifest.json

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -14050,6 +14050,149 @@
 "ITrainerOutput"
 ]
 },
 {
 "Name": "Trainers.OrdinaryLeastSquaresRegressor",
 "Desc": "Train an OLS regression model.",
 "FriendlyName": "Ordinary Least Squares (Regression)",
 "ShortName": "ols",
 "Inputs": [
 {
 "Name": "TrainingData",
 "Type": "DataView",
 "Desc": "The data to be used for training",
 "Aliases": [
 "data"
 ],
 "Required": true,
 "SortOrder": 1.0,
 "IsNullable": false
 },
 {
 "Name": "FeatureColumn",
 "Type": "String",
 "Desc": "Column to use for features",
 "Aliases": [
 "feat"
 ],
 "Required": false,
 "SortOrder": 2.0,
 "IsNullable": false,
 "Default": "Features"
 },
 {
 "Name": "LabelColumn",
 "Type": "String",
 "Desc": "Column to use for labels",
 "Aliases": [
 "lab"
 ],
 "Required": false,
 "SortOrder": 3.0,
 "IsNullable": false,
 "Default": "Label"
 },
 {
 "Name": "WeightColumn",
 "Type": "String",
 "Desc": "Column to use for example weight",
 "Aliases": [
 "weight"
 ],
 "Required": false,
 "SortOrder": 4.0,
 "IsNullable": false,
 "Default": "Weight"
 },
 {
 "Name": "NormalizeFeatures",
 "Type": {
 "Kind": "Enum",
 "Values": [
 "No",
 "Warn",
 "Auto",
 "Yes"
 ]
 },
 "Desc": "Normalize option for the feature column",
 "Aliases": [
 "norm"
 ],
 "Required": false,
 "SortOrder": 5.0,
 "IsNullable": false,
 "Default": "Auto"
 },
 {
 "Name": "Caching",
 "Type": {
 "Kind": "Enum",
 "Values": [
 "Auto",
 "Memory",
 "Disk",
 "None"
 ]
 },
 "Desc": "Whether learner should cache input training data",
 "Aliases": [
 "cache"
 ],
 "Required": false,
 "SortOrder": 6.0,
 "IsNullable": false,
 "Default": "Auto"
 },
 {
 "Name": "L2Weight",
 "Type": "Float",
 "Desc": "L2 regularization weight",
 "Aliases": [
 "l2"
 ],
 "Required": false,
 "SortOrder": 50.0,
 "IsNullable": false,
 "Default": 1E-06,
 "SweepRange": {
 "RangeType": "Discrete",
 "Values": [
 1E-06,
 0.1,
 1.0
 ]
 }
 },
 {
 "Name": "PerParameterSignificance",
 "Type": "Bool",
 "Desc": "Whether to calculate per parameter significance statistics",
 "Aliases": [
 "sig"
 ],
 "Required": false,
 "SortOrder": 150.0,
 "IsNullable": false,
 "Default": true
 }
 ],
 "Outputs": [
 {
 "Name": "PredictorModel",
 "Type": "PredictorModel",
 "Desc": "The trained model"
 }
 ],
 "InputKind": [
 "ITrainerInputWithWeight",
 "ITrainerInputWithLabel",
 "ITrainerInput"
 ],
 "OutputKind": [
 "IRegressionOutput",
 "ITrainerOutput"
 ]
 },
 {
 "Name": "Trainers.PcaAnomalyDetector",
 "Desc": "Train an PCA Anomaly model.",
Expand Down

33 changes: 33 additions & 0 deletions
test/BaselineOutput/SingleDebug/OLS/OLS-CV-wine-out.txt

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
maml.exe CV tr=OLS threads=- norm=No dout=%Output% loader=Text{col=Label:R4:11 col=Features:R4:0-10 sep=; header+} data=%Data% seed=1
Not adding a normalizer.
Trainer solving for 12 parameters across 2409 examples
Coefficient of determination R2 = 0.291173667189042, or 0.287920813763543 (adjusted)
Not training a calibrator because it is not needed.
Not adding a normalizer.
Trainer solving for 12 parameters across 2489 examples
Coefficient of determination R2 = 0.280280855195625, or 0.277084686203761 (adjusted)
Not training a calibrator because it is not needed.
L1(avg): 0.586798
L2(avg): 0.573048
RMS(avg): 0.756999
Loss-fn(avg): 0.573048
R Squared: 0.263841
L1(avg): 0.587999
L2(avg): 0.571859
RMS(avg): 0.756214
Loss-fn(avg): 0.571859
R Squared: 0.276072

OVERALL RESULTS
---------------------------------------
L1(avg): 0.587398 (0.0006)
L2(avg): 0.572454 (0.0006)
RMS(avg): 0.756606 (0.0004)
Loss-fn(avg): 0.572454 (0.0006)
R Squared: 0.269956 (0.0061)

---------------------------------------
Physical memory usage(MB): %Number%
Virtual memory usage(MB): %Number%
%DateTime% Time elapsed(s): %Number%

Loading