Update MacroUtils to map trainer kinds to the correct suffix of trainer entry point names#113

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
yaeldekel merged 8 commits into from
May 14, 2018
Merged

Update MacroUtils to map trainer kinds to the correct suffix of trainer entry point names #113

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
02f72c9
Update suffix of trainer entry point names by trainer kind.
yaeldMS May 10, 2018
ff6736d
Address PR comments.
yaeldMS May 10, 2018
1d01831
Add unit test.
yaeldMS May 10, 2018
f1bbeda
Merge remote-tracking branch 'upstream/master' into macroutils
yaeldMS May 11, 2018
db8492d
Update C# API
yaeldMS May 11, 2018
4267412
Move unit test to TestAutoInference and fix EntryPointCatalog test.
yaeldMS May 11, 2018
9bb6c89
Trigger build.
yaeldMS May 11, 2018
be06047
Add reference to the test project to make the sweeper entry point vis…
yaeldMS May 11, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions ZBaselines/Common/EntryPoints/core_ep-list.tsv
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -15,11 +15,13 @@ Models.NaiveCalibrator Apply a Naive calibrator to an input model Microsoft.ML.R
Models.OneVersusAll One-vs-All macro (OVA) Microsoft.ML.Runtime.EntryPoints.OneVersusAllMacro OVA Microsoft.ML.Runtime.EntryPoints.OneVersusAllMacro+Arguments Microsoft.ML.Runtime.EntryPoints.CommonOutputs+MacroOutput`1[Microsoft.ML.Runtime.EntryPoints.OneVersusAllMacro+Output]
Models.OvaModelCombiner Combines a sequence of PredictorModels into a single model Microsoft.ML.Runtime.Learners.OvaPredictor CombineOvaModels Microsoft.ML.Runtime.EntryPoints.ModelOperations+CombineOvaPredictorModelsInput Microsoft.ML.Runtime.EntryPoints.ModelOperations+PredictorModelOutput
Models.PAVCalibrator Apply a PAV calibrator to an input model Microsoft.ML.Runtime.Internal.Calibration.Calibrate Pav Microsoft.ML.Runtime.Internal.Calibration.Calibrate+NoArgumentsInput Microsoft.ML.Runtime.EntryPoints.CommonOutputs+CalibratorOutput
Models.PipelineSweeper AutoML pipeline sweeping optimzation macro. Microsoft.ML.Runtime.EntryPoints.PipelineSweeperMacro PipelineSweep Microsoft.ML.Runtime.EntryPoints.PipelineSweeperMacro+Arguments Microsoft.ML.Runtime.EntryPoints.CommonOutputs+MacroOutput`1[Microsoft.ML.Runtime.EntryPoints.PipelineSweeperMacro+Output]
Models.PlattCalibrator Apply a Platt calibrator to an input model Microsoft.ML.Runtime.Internal.Calibration.Calibrate Platt Microsoft.ML.Runtime.Internal.Calibration.Calibrate+NoArgumentsInput Microsoft.ML.Runtime.EntryPoints.CommonOutputs+CalibratorOutput
Models.QuantileRegressionEvaluator Evaluates a quantile regression scored dataset. Microsoft.ML.Runtime.Data.Evaluate QuantileRegression Microsoft.ML.Runtime.Data.QuantileRegressionMamlEvaluator+Arguments Microsoft.ML.Runtime.EntryPoints.CommonOutputs+CommonEvaluateOutput
Models.RankerEvaluator Evaluates a ranking scored dataset. Microsoft.ML.Runtime.Data.Evaluate Ranking Microsoft.ML.Runtime.Data.RankerMamlEvaluator+Arguments Microsoft.ML.Runtime.EntryPoints.CommonOutputs+CommonEvaluateOutput
Models.RegressionEvaluator Evaluates a regression scored dataset. Microsoft.ML.Runtime.Data.Evaluate Regression Microsoft.ML.Runtime.Data.RegressionMamlEvaluator+Arguments Microsoft.ML.Runtime.EntryPoints.CommonOutputs+CommonEvaluateOutput
Models.Summarizer Summarize a linear regression predictor. Microsoft.ML.Runtime.EntryPoints.SummarizePredictor Summarize Microsoft.ML.Runtime.EntryPoints.SummarizePredictor+Input Microsoft.ML.Runtime.EntryPoints.CommonOutputs+SummaryOutput
Models.SweepResultExtractor Extracts the sweep result. Microsoft.ML.Runtime.EntryPoints.PipelineSweeperMacro ExtractSweepResult Microsoft.ML.Runtime.EntryPoints.PipelineSweeperMacro+ResultInput Microsoft.ML.Runtime.EntryPoints.PipelineSweeperMacro+Output
Models.TrainTestBinaryEvaluator Train test for binary classification Microsoft.ML.Runtime.EntryPoints.TrainTestBinaryMacro TrainTestBinary Microsoft.ML.Runtime.EntryPoints.TrainTestBinaryMacro+Arguments Microsoft.ML.Runtime.EntryPoints.CommonOutputs+MacroOutput`1[Microsoft.ML.Runtime.EntryPoints.TrainTestBinaryMacro+Output]
Models.TrainTestEvaluator General train test for any supported evaluator Microsoft.ML.Runtime.EntryPoints.TrainTestMacro TrainTest Microsoft.ML.Runtime.EntryPoints.TrainTestMacro+Arguments Microsoft.ML.Runtime.EntryPoints.CommonOutputs+MacroOutput`1[Microsoft.ML.Runtime.EntryPoints.TrainTestMacro+Output]
Trainers.AveragedPerceptronBinaryClassifier Train a Average perceptron. Microsoft.ML.Runtime.Learners.AveragedPerceptronTrainer TrainBinary Microsoft.ML.Runtime.Learners.AveragedPerceptronTrainer+Arguments Microsoft.ML.Runtime.EntryPoints.CommonOutputs+BinaryClassificationOutput
Expand Down
358 changes: 358 additions & 0 deletions ZBaselines/Common/EntryPoints/core_manifest.json
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -1723,6 +1723,105 @@
"ITrainerOutput"
]
},
{
"Name": "Models.PipelineSweeper",
"Desc": "AutoML pipeline sweeping optimzation macro.",
"FriendlyName": null,
"ShortName": null,
"Inputs": [
{
"Name": "TrainingData",
"Type": "DataView",
"Desc": "The data to be used for training.",
"Aliases": [
"train"
],
"Required": true,
"SortOrder": 1.0,
"IsNullable": false
},
{
"Name": "TestingData",
"Type": "DataView",
"Desc": "The data to be used for testing.",
"Aliases": [
"test"
],
"Required": true,
"SortOrder": 2.0,
"IsNullable": false
},
{
"Name": "StateArguments",
"Type": {
"Kind": "Component",
"ComponentKind": "AutoMlStateBase"
},
"Desc": "The arguments for creating an AutoMlState component.",
"Aliases": [
"args"
],
"Required": false,
"SortOrder": 3.0,
"IsNullable": false,
"Default": null
},
{
"Name": "State",
"Type": {
"Kind": "C# Object",
"ItemType": "Microsoft.ML.Runtime.EntryPoints.IMlState"
},
"Desc": "The stateful object conducting of the autoML search.",
"Aliases": [
"state"
],
"Required": false,
"SortOrder": 3.0,
"IsNullable": false,
"Default": null
},
{
"Name": "BatchSize",
"Type": "Int",
"Desc": "Number of candidate pipelines to retrieve each round.",
"Aliases": [
"bsize"
],
"Required": true,
"SortOrder": 4.0,
"IsNullable": false,
"Default": 0
},
{
"Name": "CandidateOutputs",
"Type": {
"Kind": "Array",
"ItemType": "DataView"
},
"Desc": "Output datasets from previous iteration of sweep.",
"Required": false,
"SortOrder": 7.0,
"IsNullable": false,
"Default": null
}
],
"Outputs": [
{
"Name": "State",
"Type": {
"Kind": "C# Object",
"ItemType": "Microsoft.ML.Runtime.EntryPoints.IMlState"
},
"Desc": "Stateful autoML object, keeps track of where the search in progress."
},
{
"Name": "Results",
"Type": "DataView",
"Desc": "Results of the sweep, including pipelines (as graph strings), IDs, and metric values."
}
]
},
{
"Name": "Models.PlattCalibrator",
"Desc": "Apply a Platt calibrator to an input model",
Expand DownExpand Up@@ -2209,6 +2308,44 @@
}
]
},
{
"Name": "Models.SweepResultExtractor",
"Desc": "Extracts the sweep result.",
"FriendlyName": null,
"ShortName": null,
"Inputs": [
{
"Name": "State",
"Type": {
"Kind": "C# Object",
"ItemType": "Microsoft.ML.Runtime.EntryPoints.IMlState"
},
"Desc": "The stateful object conducting of the autoML search.",
"Aliases": [
"state"
],
"Required": false,
"SortOrder": 1.0,
"IsNullable": false,
"Default": null
}
],
"Outputs": [
{
"Name": "State",
"Type": {
"Kind": "C# Object",
"ItemType": "Microsoft.ML.Runtime.EntryPoints.IMlState"
},
"Desc": "Stateful autoML object, keeps track of where the search in progress."
},
{
"Name": "Results",
"Type": "DataView",
"Desc": "Results of the sweep, including pipelines (as graph strings), IDs, and metric values."
}
]
},
{
"Name": "Models.TrainTestBinaryEvaluator",
"Desc": "Train test for binary classification",
Expand DownExpand Up@@ -17200,6 +17337,203 @@
}
],
"Components": [
{
"Kind": "AutoMlEngine",
"Components": [
{
"Name": "Defaults",
"Desc": "AutoML engine that returns learners with default settings.",
"FriendlyName": "Defaults Engine",
"Settings": []
},
{
"Name": "Rocket",
"Desc": "AutoML engine that consists of distinct, hierarchical stages of operation.",
"FriendlyName": "Rocket Engine",
"Settings": [
{
"Name": "TopKLearners",
"Type": "Int",
"Desc": "Number of learners to retain for second stage.",
"Aliases": [
"topk"
],
"Required": false,
"SortOrder": 1.0,
"IsNullable": false,
"Default": 2
},
{
"Name": "SecondRoundTrialsPerLearner",
"Type": "Int",
"Desc": "Number of trials for retained second stage learners.",
"Aliases": [
"stage2num"
],
"Required": false,
"SortOrder": 2.0,
"IsNullable": false,
"Default": 5
},
{
"Name": "RandomInitialization",
"Type": "Bool",
"Desc": "Use random initialization only.",
"Aliases": [
"randinit"
],
"Required": false,
"SortOrder": 3.0,
"IsNullable": false,
"Default": false
},
{
"Name": "NumInitializationPipelines",
"Type": "Int",
"Desc": "Number of initilization pipelines, used for random initialization only.",
"Aliases": [
"numinitseeds"
],
"Required": false,
"SortOrder": 4.0,
"IsNullable": false,
"Default": 20
}
]
},
{
"Name": "UniformRandom",
"Desc": "AutoML engine using uniform random sampling.",
"FriendlyName": "Uniform Random Engine",
"Settings": []
}
]
},
{
"Kind": "AutoMlStateBase",
"Components": [
{
"Name": "AutoMlState",
"Desc": "State of an AutoML search and search space.",
"FriendlyName": "AutoML State",
"Aliases": [
"automlst"
],
"Settings": [
{
"Name": "Metric",
"Type": {
"Kind": "Enum",
"Values": [
"Auc",
"AccuracyMicro",
"AccuracyMacro",
"L2",
"F1",
"AuPrc",
"TopKAccuracy",
"Rms",
"LossFn",
"RSquared",
"LogLoss",
"LogLossReduction",
"Ndcg",
"Dcg",
"PositivePrecision",
"PositiveRecall",
"NegativePrecision",
"NegativeRecall",
"DrAtK",
"DrAtPFpr",
"DrAtNumPos",
"NumAnomalies",
"ThreshAtK",
"ThreshAtP",
"ThreshAtNumPos",
"Nmi",
"AvgMinScore",
"Dbi"
]
},
"Desc": "Supported metric for evaluator.",
"Aliases": [
"metric"
],
"Required": true,
"SortOrder": 150.0,
"IsNullable": false,
"Default": "Auc"
},
{
"Name": "Engine",
"Type": {
"Kind": "Component",
"ComponentKind": "AutoMlEngine"
},
"Desc": "AutoML engine (pipeline optimizer) that generates next candidates.",
"Aliases": [
"engine"
],
"Required": true,
"SortOrder": 150.0,
"IsNullable": false
},
{
"Name": "TrainerKind",
"Type": {
"Kind": "Enum",
"Values": [
"SignatureBinaryClassifierTrainer",
"SignatureMultiClassClassifierTrainer",
"SignatureRankerTrainer",
"SignatureRegressorTrainer",
"SignatureMultiOutputRegressorTrainer",
"SignatureAnomalyDetectorTrainer",
"SignatureClusteringTrainer"
]
},
"Desc": "Kind of trainer for task, such as binary classification trainer, multiclass trainer, etc.",
"Aliases": [
"tk"
],
"Required": true,
"SortOrder": 150.0,
"IsNullable": false,
"Default": "SignatureBinaryClassifierTrainer"
},
{
"Name": "TerminatorArgs",
"Type": {
"Kind": "Component",
"ComponentKind": "SearchTerminator"
},
"Desc": "Arguments for creating terminator, which determines when to stop search.",
"Aliases": [
"term"
],
"Required": true,
"SortOrder": 150.0,
"IsNullable": false
},
{
"Name": "RequestedLearners",
"Type": {
"Kind": "Array",
"ItemType": "String"
},
"Desc": "Learner set to sweep over (if available).",
"Aliases": [
"learners"
],
"Required": false,
"SortOrder": 150.0,
"IsNullable": false,
"Default": null
}
]
}
]
},
{
"Kind": "CalibratorTrainer",
"Components": [
Expand DownExpand Up@@ -21441,6 +21775,30 @@
}
]
},
{
"Kind": "SearchTerminator",
"Components": [
{
"Name": "IterationLimited",
"Desc": "Terminators a sweep based on total number of iterations.",
"FriendlyName": "Pipeline Sweep Iteration Terminator",
"Settings": [
{
"Name": "FinalHistoryLength",
"Type": "Int",
"Desc": "Total number of iterations.",
"Aliases": [
"length"
],
"Required": true,
"SortOrder": 150.0,
"IsNullable": false,
"Default": 0
}
]
}
]
},
{
"Kind": "StopWordsRemover",
"Components": [
Expand Down
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all 
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '

Skip to content
Update MacroUtils to map trainer kinds to the correct suffix of trainer entry point names#113

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

yaeldekel
merged 8 commits into

from

May 14, 2018
 Merged


Update MacroUtils to map trainer kinds to the correct suffix of trainer entry point names
#113
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


8 commits


Select commit
Hold shift + click to select a range
02f72c9

Update suffix of trainer entry point names by trainer kind.


yaeldMS May 10, 2018
ff6736d

Address PR comments.


yaeldMS May 10, 2018
1d01831

Add unit test.


yaeldMS May 10, 2018
f1bbeda

Merge remote-tracking branch 'upstream/master' into macroutils


yaeldMS May 11, 2018
db8492d

Update C# API


yaeldMS May 11, 2018
4267412

Move unit test to TestAutoInference and fix EntryPointCatalog test.


yaeldMS May 11, 2018
9bb6c89

Trigger build.


yaeldMS May 11, 2018
be06047

Add reference to the test project to make the sweeper entry point vis…


yaeldMS May 11, 2018

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


2 changes: 2 additions & 0 deletions
ZBaselines/Common/EntryPoints/core_ep-list.tsv

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -15,11 +15,13 @@ Models.NaiveCalibrator	Apply a Naive calibrator to an input model	Microsoft.ML.R
Models.OneVersusAll	One-vs-All macro (OVA)	Microsoft.ML.Runtime.EntryPoints.OneVersusAllMacro	OVA	Microsoft.ML.Runtime.EntryPoints.OneVersusAllMacro+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+MacroOutput`1[Microsoft.ML.Runtime.EntryPoints.OneVersusAllMacro+Output]
Models.OvaModelCombiner	Combines a sequence of PredictorModels into a single model	Microsoft.ML.Runtime.Learners.OvaPredictor	CombineOvaModels	Microsoft.ML.Runtime.EntryPoints.ModelOperations+CombineOvaPredictorModelsInput	Microsoft.ML.Runtime.EntryPoints.ModelOperations+PredictorModelOutput
Models.PAVCalibrator	Apply a PAV calibrator to an input model	Microsoft.ML.Runtime.Internal.Calibration.Calibrate	Pav	Microsoft.ML.Runtime.Internal.Calibration.Calibrate+NoArgumentsInput	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+CalibratorOutput
Models.PipelineSweeper	AutoML pipeline sweeping optimzation macro.	Microsoft.ML.Runtime.EntryPoints.PipelineSweeperMacro	PipelineSweep	Microsoft.ML.Runtime.EntryPoints.PipelineSweeperMacro+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+MacroOutput`1[Microsoft.ML.Runtime.EntryPoints.PipelineSweeperMacro+Output]
Models.PlattCalibrator	Apply a Platt calibrator to an input model	Microsoft.ML.Runtime.Internal.Calibration.Calibrate	Platt	Microsoft.ML.Runtime.Internal.Calibration.Calibrate+NoArgumentsInput	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+CalibratorOutput
Models.QuantileRegressionEvaluator	Evaluates a quantile regression scored dataset.	Microsoft.ML.Runtime.Data.Evaluate	QuantileRegression	Microsoft.ML.Runtime.Data.QuantileRegressionMamlEvaluator+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+CommonEvaluateOutput
Models.RankerEvaluator	Evaluates a ranking scored dataset.	Microsoft.ML.Runtime.Data.Evaluate	Ranking	Microsoft.ML.Runtime.Data.RankerMamlEvaluator+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+CommonEvaluateOutput
Models.RegressionEvaluator	Evaluates a regression scored dataset.	Microsoft.ML.Runtime.Data.Evaluate	Regression	Microsoft.ML.Runtime.Data.RegressionMamlEvaluator+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+CommonEvaluateOutput
Models.Summarizer	Summarize a linear regression predictor.	Microsoft.ML.Runtime.EntryPoints.SummarizePredictor	Summarize	Microsoft.ML.Runtime.EntryPoints.SummarizePredictor+Input	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+SummaryOutput
Models.SweepResultExtractor	Extracts the sweep result.	Microsoft.ML.Runtime.EntryPoints.PipelineSweeperMacro	ExtractSweepResult	Microsoft.ML.Runtime.EntryPoints.PipelineSweeperMacro+ResultInput	Microsoft.ML.Runtime.EntryPoints.PipelineSweeperMacro+Output
Models.TrainTestBinaryEvaluator	Train test for binary classification	Microsoft.ML.Runtime.EntryPoints.TrainTestBinaryMacro	TrainTestBinary	Microsoft.ML.Runtime.EntryPoints.TrainTestBinaryMacro+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+MacroOutput`1[Microsoft.ML.Runtime.EntryPoints.TrainTestBinaryMacro+Output]
Models.TrainTestEvaluator	General train test for any supported evaluator	Microsoft.ML.Runtime.EntryPoints.TrainTestMacro	TrainTest	Microsoft.ML.Runtime.EntryPoints.TrainTestMacro+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+MacroOutput`1[Microsoft.ML.Runtime.EntryPoints.TrainTestMacro+Output]
Trainers.AveragedPerceptronBinaryClassifier	Train a Average perceptron.	Microsoft.ML.Runtime.Learners.AveragedPerceptronTrainer	TrainBinary	Microsoft.ML.Runtime.Learners.AveragedPerceptronTrainer+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+BinaryClassificationOutput
Expand Down

358 changes: 358 additions & 0 deletions
ZBaselines/Common/EntryPoints/core_manifest.json

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -1723,6 +1723,105 @@
 "ITrainerOutput"
 ]
 },
 {
 "Name": "Models.PipelineSweeper",
 "Desc": "AutoML pipeline sweeping optimzation macro.",
 "FriendlyName": null,
 "ShortName": null,
 "Inputs": [
 {
 "Name": "TrainingData",
 "Type": "DataView",
 "Desc": "The data to be used for training.",
 "Aliases": [
 "train"
 ],
 "Required": true,
 "SortOrder": 1.0,
 "IsNullable": false
 },
 {
 "Name": "TestingData",
 "Type": "DataView",
 "Desc": "The data to be used for testing.",
 "Aliases": [
 "test"
 ],
 "Required": true,
 "SortOrder": 2.0,
 "IsNullable": false
 },
 {
 "Name": "StateArguments",
 "Type": {
 "Kind": "Component",
 "ComponentKind": "AutoMlStateBase"
 },
 "Desc": "The arguments for creating an AutoMlState component.",
 "Aliases": [
 "args"
 ],
 "Required": false,
 "SortOrder": 3.0,
 "IsNullable": false,
 "Default": null
 },
 {
 "Name": "State",
 "Type": {
 "Kind": "C# Object",
 "ItemType": "Microsoft.ML.Runtime.EntryPoints.IMlState"
 },
 "Desc": "The stateful object conducting of the autoML search.",
 "Aliases": [
 "state"
 ],
 "Required": false,
 "SortOrder": 3.0,
 "IsNullable": false,
 "Default": null
 },
 {
 "Name": "BatchSize",
 "Type": "Int",
 "Desc": "Number of candidate pipelines to retrieve each round.",
 "Aliases": [
 "bsize"
 ],
 "Required": true,
 "SortOrder": 4.0,
 "IsNullable": false,
 "Default": 0
 },
 {
 "Name": "CandidateOutputs",
 "Type": {
 "Kind": "Array",
 "ItemType": "DataView"
 },
 "Desc": "Output datasets from previous iteration of sweep.",
 "Required": false,
 "SortOrder": 7.0,
 "IsNullable": false,
 "Default": null
 }
 ],
 "Outputs": [
 {
 "Name": "State",
 "Type": {
 "Kind": "C# Object",
 "ItemType": "Microsoft.ML.Runtime.EntryPoints.IMlState"
 },
 "Desc": "Stateful autoML object, keeps track of where the search in progress."
 },
 {
 "Name": "Results",
 "Type": "DataView",
 "Desc": "Results of the sweep, including pipelines (as graph strings), IDs, and metric values."
 }
 ]
 },
 {
 "Name": "Models.PlattCalibrator",
 "Desc": "Apply a Platt calibrator to an input model",
Expand DownExpand Up@@ -2209,6 +2308,44 @@
 }
 ]
 },
 {
 "Name": "Models.SweepResultExtractor",
 "Desc": "Extracts the sweep result.",
 "FriendlyName": null,
 "ShortName": null,
 "Inputs": [
 {
 "Name": "State",
 "Type": {
 "Kind": "C# Object",
 "ItemType": "Microsoft.ML.Runtime.EntryPoints.IMlState"
 },
 "Desc": "The stateful object conducting of the autoML search.",
 "Aliases": [
 "state"
 ],
 "Required": false,
 "SortOrder": 1.0,
 "IsNullable": false,
 "Default": null
 }
 ],
 "Outputs": [
 {
 "Name": "State",
 "Type": {
 "Kind": "C# Object",
 "ItemType": "Microsoft.ML.Runtime.EntryPoints.IMlState"
 },
 "Desc": "Stateful autoML object, keeps track of where the search in progress."
 },
 {
 "Name": "Results",
 "Type": "DataView",
 "Desc": "Results of the sweep, including pipelines (as graph strings), IDs, and metric values."
 }
 ]
 },
 {
 "Name": "Models.TrainTestBinaryEvaluator",
 "Desc": "Train test for binary classification",
Expand DownExpand Up@@ -17200,6 +17337,203 @@
 }
 ],
 "Components": [
 {
 "Kind": "AutoMlEngine",
 "Components": [
 {
 "Name": "Defaults",
 "Desc": "AutoML engine that returns learners with default settings.",
 "FriendlyName": "Defaults Engine",
 "Settings": []
 },
 {
 "Name": "Rocket",
 "Desc": "AutoML engine that consists of distinct, hierarchical stages of operation.",
 "FriendlyName": "Rocket Engine",
 "Settings": [
 {
 "Name": "TopKLearners",
 "Type": "Int",
 "Desc": "Number of learners to retain for second stage.",
 "Aliases": [
 "topk"
 ],
 "Required": false,
 "SortOrder": 1.0,
 "IsNullable": false,
 "Default": 2
 },
 {
 "Name": "SecondRoundTrialsPerLearner",
 "Type": "Int",
 "Desc": "Number of trials for retained second stage learners.",
 "Aliases": [
 "stage2num"
 ],
 "Required": false,
 "SortOrder": 2.0,
 "IsNullable": false,
 "Default": 5
 },
 {
 "Name": "RandomInitialization",
 "Type": "Bool",
 "Desc": "Use random initialization only.",
 "Aliases": [
 "randinit"
 ],
 "Required": false,
 "SortOrder": 3.0,
 "IsNullable": false,
 "Default": false
 },
 {
 "Name": "NumInitializationPipelines",
 "Type": "Int",
 "Desc": "Number of initilization pipelines, used for random initialization only.",
 "Aliases": [
 "numinitseeds"
 ],
 "Required": false,
 "SortOrder": 4.0,
 "IsNullable": false,
 "Default": 20
 }
 ]
 },
 {
 "Name": "UniformRandom",
 "Desc": "AutoML engine using uniform random sampling.",
 "FriendlyName": "Uniform Random Engine",
 "Settings": []
 }
 ]
 },
 {
 "Kind": "AutoMlStateBase",
 "Components": [
 {
 "Name": "AutoMlState",
 "Desc": "State of an AutoML search and search space.",
 "FriendlyName": "AutoML State",
 "Aliases": [
 "automlst"
 ],
 "Settings": [
 {
 "Name": "Metric",
 "Type": {
 "Kind": "Enum",
 "Values": [
 "Auc",
 "AccuracyMicro",
 "AccuracyMacro",
 "L2",
 "F1",
 "AuPrc",
 "TopKAccuracy",
 "Rms",
 "LossFn",
 "RSquared",
 "LogLoss",
 "LogLossReduction",
 "Ndcg",
 "Dcg",
 "PositivePrecision",
 "PositiveRecall",
 "NegativePrecision",
 "NegativeRecall",
 "DrAtK",
 "DrAtPFpr",
 "DrAtNumPos",
 "NumAnomalies",
 "ThreshAtK",
 "ThreshAtP",
 "ThreshAtNumPos",
 "Nmi",
 "AvgMinScore",
 "Dbi"
 ]
 },
 "Desc": "Supported metric for evaluator.",
 "Aliases": [
 "metric"
 ],
 "Required": true,
 "SortOrder": 150.0,
 "IsNullable": false,
 "Default": "Auc"
 },
 {
 "Name": "Engine",
 "Type": {
 "Kind": "Component",
 "ComponentKind": "AutoMlEngine"
 },
 "Desc": "AutoML engine (pipeline optimizer) that generates next candidates.",
 "Aliases": [
 "engine"
 ],
 "Required": true,
 "SortOrder": 150.0,
 "IsNullable": false
 },
 {
 "Name": "TrainerKind",
 "Type": {
 "Kind": "Enum",
 "Values": [
 "SignatureBinaryClassifierTrainer",
 "SignatureMultiClassClassifierTrainer",
 "SignatureRankerTrainer",
 "SignatureRegressorTrainer",
 "SignatureMultiOutputRegressorTrainer",
 "SignatureAnomalyDetectorTrainer",
 "SignatureClusteringTrainer"
 ]
 },
 "Desc": "Kind of trainer for task, such as binary classification trainer, multiclass trainer, etc.",
 "Aliases": [
 "tk"
 ],
 "Required": true,
 "SortOrder": 150.0,
 "IsNullable": false,
 "Default": "SignatureBinaryClassifierTrainer"
 },
 {
 "Name": "TerminatorArgs",
 "Type": {
 "Kind": "Component",
 "ComponentKind": "SearchTerminator"
 },
 "Desc": "Arguments for creating terminator, which determines when to stop search.",
 "Aliases": [
 "term"
 ],
 "Required": true,
 "SortOrder": 150.0,
 "IsNullable": false
 },
 {
 "Name": "RequestedLearners",
 "Type": {
 "Kind": "Array",
 "ItemType": "String"
 },
 "Desc": "Learner set to sweep over (if available).",
 "Aliases": [
 "learners"
 ],
 "Required": false,
 "SortOrder": 150.0,
 "IsNullable": false,
 "Default": null
 }
 ]
 }
 ]
 },
 {
 "Kind": "CalibratorTrainer",
 "Components": [
Expand DownExpand Up@@ -21441,6 +21775,30 @@
 }
 ]
 },
 {
 "Kind": "SearchTerminator",
 "Components": [
 {
 "Name": "IterationLimited",
 "Desc": "Terminators a sweep based on total number of iterations.",
 "FriendlyName": "Pipeline Sweep Iteration Terminator",
 "Settings": [
 {
 "Name": "FinalHistoryLength",
 "Type": "Int",
 "Desc": "Total number of iterations.",
 "Aliases": [
 "length"
 ],
 "Required": true,
 "SortOrder": 150.0,
 "IsNullable": false,
 "Default": 0
 }
 ]
 }
 ]
 },
 {
 "Kind": "StopWordsRemover",
 "Components": [
Expand Down
Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix");
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Update MacroUtils to map trainer kinds to the correct suffix of trainer entry point names#113

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

yaeldekel
merged 8 commits into

from

May 14, 2018
 Merged


Update MacroUtils to map trainer kinds to the correct suffix of trainer entry point names
#113
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


8 commits


Select commit
Hold shift + click to select a range
02f72c9

Update suffix of trainer entry point names by trainer kind.


yaeldMS May 10, 2018
ff6736d

Address PR comments.


yaeldMS May 10, 2018
1d01831

Add unit test.


yaeldMS May 10, 2018
f1bbeda

Merge remote-tracking branch 'upstream/master' into macroutils


yaeldMS May 11, 2018
db8492d

Update C# API


yaeldMS May 11, 2018
4267412

Move unit test to TestAutoInference and fix EntryPointCatalog test.


yaeldMS May 11, 2018
9bb6c89

Trigger build.


yaeldMS May 11, 2018
be06047

Add reference to the test project to make the sweeper entry point vis…


yaeldMS May 11, 2018

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


2 changes: 2 additions & 0 deletions
ZBaselines/Common/EntryPoints/core_ep-list.tsv

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -15,11 +15,13 @@ Models.NaiveCalibrator	Apply a Naive calibrator to an input model	Microsoft.ML.R
Models.OneVersusAll	One-vs-All macro (OVA)	Microsoft.ML.Runtime.EntryPoints.OneVersusAllMacro	OVA	Microsoft.ML.Runtime.EntryPoints.OneVersusAllMacro+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+MacroOutput`1[Microsoft.ML.Runtime.EntryPoints.OneVersusAllMacro+Output]
Models.OvaModelCombiner	Combines a sequence of PredictorModels into a single model	Microsoft.ML.Runtime.Learners.OvaPredictor	CombineOvaModels	Microsoft.ML.Runtime.EntryPoints.ModelOperations+CombineOvaPredictorModelsInput	Microsoft.ML.Runtime.EntryPoints.ModelOperations+PredictorModelOutput
Models.PAVCalibrator	Apply a PAV calibrator to an input model	Microsoft.ML.Runtime.Internal.Calibration.Calibrate	Pav	Microsoft.ML.Runtime.Internal.Calibration.Calibrate+NoArgumentsInput	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+CalibratorOutput
Models.PipelineSweeper	AutoML pipeline sweeping optimzation macro.	Microsoft.ML.Runtime.EntryPoints.PipelineSweeperMacro	PipelineSweep	Microsoft.ML.Runtime.EntryPoints.PipelineSweeperMacro+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+MacroOutput`1[Microsoft.ML.Runtime.EntryPoints.PipelineSweeperMacro+Output]
Models.PlattCalibrator	Apply a Platt calibrator to an input model	Microsoft.ML.Runtime.Internal.Calibration.Calibrate	Platt	Microsoft.ML.Runtime.Internal.Calibration.Calibrate+NoArgumentsInput	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+CalibratorOutput
Models.QuantileRegressionEvaluator	Evaluates a quantile regression scored dataset.	Microsoft.ML.Runtime.Data.Evaluate	QuantileRegression	Microsoft.ML.Runtime.Data.QuantileRegressionMamlEvaluator+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+CommonEvaluateOutput
Models.RankerEvaluator	Evaluates a ranking scored dataset.	Microsoft.ML.Runtime.Data.Evaluate	Ranking	Microsoft.ML.Runtime.Data.RankerMamlEvaluator+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+CommonEvaluateOutput
Models.RegressionEvaluator	Evaluates a regression scored dataset.	Microsoft.ML.Runtime.Data.Evaluate	Regression	Microsoft.ML.Runtime.Data.RegressionMamlEvaluator+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+CommonEvaluateOutput
Models.Summarizer	Summarize a linear regression predictor.	Microsoft.ML.Runtime.EntryPoints.SummarizePredictor	Summarize	Microsoft.ML.Runtime.EntryPoints.SummarizePredictor+Input	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+SummaryOutput
Models.SweepResultExtractor	Extracts the sweep result.	Microsoft.ML.Runtime.EntryPoints.PipelineSweeperMacro	ExtractSweepResult	Microsoft.ML.Runtime.EntryPoints.PipelineSweeperMacro+ResultInput	Microsoft.ML.Runtime.EntryPoints.PipelineSweeperMacro+Output
Models.TrainTestBinaryEvaluator	Train test for binary classification	Microsoft.ML.Runtime.EntryPoints.TrainTestBinaryMacro	TrainTestBinary	Microsoft.ML.Runtime.EntryPoints.TrainTestBinaryMacro+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+MacroOutput`1[Microsoft.ML.Runtime.EntryPoints.TrainTestBinaryMacro+Output]
Models.TrainTestEvaluator	General train test for any supported evaluator	Microsoft.ML.Runtime.EntryPoints.TrainTestMacro	TrainTest	Microsoft.ML.Runtime.EntryPoints.TrainTestMacro+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+MacroOutput`1[Microsoft.ML.Runtime.EntryPoints.TrainTestMacro+Output]
Trainers.AveragedPerceptronBinaryClassifier	Train a Average perceptron.	Microsoft.ML.Runtime.Learners.AveragedPerceptronTrainer	TrainBinary	Microsoft.ML.Runtime.Learners.AveragedPerceptronTrainer+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+BinaryClassificationOutput
Expand Down

358 changes: 358 additions & 0 deletions
ZBaselines/Common/EntryPoints/core_manifest.json

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -1723,6 +1723,105 @@
 "ITrainerOutput"
 ]
 },
 {
 "Name": "Models.PipelineSweeper",
 "Desc": "AutoML pipeline sweeping optimzation macro.",
 "FriendlyName": null,
 "ShortName": null,
 "Inputs": [
 {
 "Name": "TrainingData",
 "Type": "DataView",
 "Desc": "The data to be used for training.",
 "Aliases": [
 "train"
 ],
 "Required": true,
 "SortOrder": 1.0,
 "IsNullable": false
 },
 {
 "Name": "TestingData",
 "Type": "DataView",
 "Desc": "The data to be used for testing.",
 "Aliases": [
 "test"
 ],
 "Required": true,
 "SortOrder": 2.0,
 "IsNullable": false
 },
 {
 "Name": "StateArguments",
 "Type": {
 "Kind": "Component",
 "ComponentKind": "AutoMlStateBase"
 },
 "Desc": "The arguments for creating an AutoMlState component.",
 "Aliases": [
 "args"
 ],
 "Required": false,
 "SortOrder": 3.0,
 "IsNullable": false,
 "Default": null
 },
 {
 "Name": "State",
 "Type": {
 "Kind": "C# Object",
 "ItemType": "Microsoft.ML.Runtime.EntryPoints.IMlState"
 },
 "Desc": "The stateful object conducting of the autoML search.",
 "Aliases": [
 "state"
 ],
 "Required": false,
 "SortOrder": 3.0,
 "IsNullable": false,
 "Default": null
 },
 {
 "Name": "BatchSize",
 "Type": "Int",
 "Desc": "Number of candidate pipelines to retrieve each round.",
 "Aliases": [
 "bsize"
 ],
 "Required": true,
 "SortOrder": 4.0,
 "IsNullable": false,
 "Default": 0
 },
 {
 "Name": "CandidateOutputs",
 "Type": {
 "Kind": "Array",
 "ItemType": "DataView"
 },
 "Desc": "Output datasets from previous iteration of sweep.",
 "Required": false,
 "SortOrder": 7.0,
 "IsNullable": false,
 "Default": null
 }
 ],
 "Outputs": [
 {
 "Name": "State",
 "Type": {
 "Kind": "C# Object",
 "ItemType": "Microsoft.ML.Runtime.EntryPoints.IMlState"
 },
 "Desc": "Stateful autoML object, keeps track of where the search in progress."
 },
 {
 "Name": "Results",
 "Type": "DataView",
 "Desc": "Results of the sweep, including pipelines (as graph strings), IDs, and metric values."
 }
 ]
 },
 {
 "Name": "Models.PlattCalibrator",
 "Desc": "Apply a Platt calibrator to an input model",
Expand DownExpand Up@@ -2209,6 +2308,44 @@
 }
 ]
 },
 {
 "Name": "Models.SweepResultExtractor",
 "Desc": "Extracts the sweep result.",
 "FriendlyName": null,
 "ShortName": null,
 "Inputs": [
 {
 "Name": "State",
 "Type": {
 "Kind": "C# Object",
 "ItemType": "Microsoft.ML.Runtime.EntryPoints.IMlState"
 },
 "Desc": "The stateful object conducting of the autoML search.",
 "Aliases": [
 "state"
 ],
 "Required": false,
 "SortOrder": 1.0,
 "IsNullable": false,
 "Default": null
 }
 ],
 "Outputs": [
 {
 "Name": "State",
 "Type": {
 "Kind": "C# Object",
 "ItemType": "Microsoft.ML.Runtime.EntryPoints.IMlState"
 },
 "Desc": "Stateful autoML object, keeps track of where the search in progress."
 },
 {
 "Name": "Results",
 "Type": "DataView",
 "Desc": "Results of the sweep, including pipelines (as graph strings), IDs, and metric values."
 }
 ]
 },
 {
 "Name": "Models.TrainTestBinaryEvaluator",
 "Desc": "Train test for binary classification",
Expand DownExpand Up@@ -17200,6 +17337,203 @@
 }
 ],
 "Components": [
 {
 "Kind": "AutoMlEngine",
 "Components": [
 {
 "Name": "Defaults",
 "Desc": "AutoML engine that returns learners with default settings.",
 "FriendlyName": "Defaults Engine",
 "Settings": []
 },
 {
 "Name": "Rocket",
 "Desc": "AutoML engine that consists of distinct, hierarchical stages of operation.",
 "FriendlyName": "Rocket Engine",
 "Settings": [
 {
 "Name": "TopKLearners",
 "Type": "Int",
 "Desc": "Number of learners to retain for second stage.",
 "Aliases": [
 "topk"
 ],
 "Required": false,
 "SortOrder": 1.0,
 "IsNullable": false,
 "Default": 2
 },
 {
 "Name": "SecondRoundTrialsPerLearner",
 "Type": "Int",
 "Desc": "Number of trials for retained second stage learners.",
 "Aliases": [
 "stage2num"
 ],
 "Required": false,
 "SortOrder": 2.0,
 "IsNullable": false,
 "Default": 5
 },
 {
 "Name": "RandomInitialization",
 "Type": "Bool",
 "Desc": "Use random initialization only.",
 "Aliases": [
 "randinit"
 ],
 "Required": false,
 "SortOrder": 3.0,
 "IsNullable": false,
 "Default": false
 },
 {
 "Name": "NumInitializationPipelines",
 "Type": "Int",
 "Desc": "Number of initilization pipelines, used for random initialization only.",
 "Aliases": [
 "numinitseeds"
 ],
 "Required": false,
 "SortOrder": 4.0,
 "IsNullable": false,
 "Default": 20
 }
 ]
 },
 {
 "Name": "UniformRandom",
 "Desc": "AutoML engine using uniform random sampling.",
 "FriendlyName": "Uniform Random Engine",
 "Settings": []
 }
 ]
 },
 {
 "Kind": "AutoMlStateBase",
 "Components": [
 {
 "Name": "AutoMlState",
 "Desc": "State of an AutoML search and search space.",
 "FriendlyName": "AutoML State",
 "Aliases": [
 "automlst"
 ],
 "Settings": [
 {
 "Name": "Metric",
 "Type": {
 "Kind": "Enum",
 "Values": [
 "Auc",
 "AccuracyMicro",
 "AccuracyMacro",
 "L2",
 "F1",
 "AuPrc",
 "TopKAccuracy",
 "Rms",
 "LossFn",
 "RSquared",
 "LogLoss",
 "LogLossReduction",
 "Ndcg",
 "Dcg",
 "PositivePrecision",
 "PositiveRecall",
 "NegativePrecision",
 "NegativeRecall",
 "DrAtK",
 "DrAtPFpr",
 "DrAtNumPos",
 "NumAnomalies",
 "ThreshAtK",
 "ThreshAtP",
 "ThreshAtNumPos",
 "Nmi",
 "AvgMinScore",
 "Dbi"
 ]
 },
 "Desc": "Supported metric for evaluator.",
 "Aliases": [
 "metric"
 ],
 "Required": true,
 "SortOrder": 150.0,
 "IsNullable": false,
 "Default": "Auc"
 },
 {
 "Name": "Engine",
 "Type": {
 "Kind": "Component",
 "ComponentKind": "AutoMlEngine"
 },
 "Desc": "AutoML engine (pipeline optimizer) that generates next candidates.",
 "Aliases": [
 "engine"
 ],
 "Required": true,
 "SortOrder": 150.0,
 "IsNullable": false
 },
 {
 "Name": "TrainerKind",
 "Type": {
 "Kind": "Enum",
 "Values": [
 "SignatureBinaryClassifierTrainer",
 "SignatureMultiClassClassifierTrainer",
 "SignatureRankerTrainer",
 "SignatureRegressorTrainer",
 "SignatureMultiOutputRegressorTrainer",
 "SignatureAnomalyDetectorTrainer",
 "SignatureClusteringTrainer"
 ]
 },
 "Desc": "Kind of trainer for task, such as binary classification trainer, multiclass trainer, etc.",
 "Aliases": [
 "tk"
 ],
 "Required": true,
 "SortOrder": 150.0,
 "IsNullable": false,
 "Default": "SignatureBinaryClassifierTrainer"
 },
 {
 "Name": "TerminatorArgs",
 "Type": {
 "Kind": "Component",
 "ComponentKind": "SearchTerminator"
 },
 "Desc": "Arguments for creating terminator, which determines when to stop search.",
 "Aliases": [
 "term"
 ],
 "Required": true,
 "SortOrder": 150.0,
 "IsNullable": false
 },
 {
 "Name": "RequestedLearners",
 "Type": {
 "Kind": "Array",
 "ItemType": "String"
 },
 "Desc": "Learner set to sweep over (if available).",
 "Aliases": [
 "learners"
 ],
 "Required": false,
 "SortOrder": 150.0,
 "IsNullable": false,
 "Default": null
 }
 ]
 }
 ]
 },
 {
 "Kind": "CalibratorTrainer",
 "Components": [
Expand DownExpand Up@@ -21441,6 +21775,30 @@
 }
 ]
 },
 {
 "Kind": "SearchTerminator",
 "Components": [
 {
 "Name": "IterationLimited",
 "Desc": "Terminators a sweep based on total number of iterations.",
 "FriendlyName": "Pipeline Sweep Iteration Terminator",
 "Settings": [
 {
 "Name": "FinalHistoryLength",
 "Type": "Int",
 "Desc": "Total number of iterations.",
 "Aliases": [
 "length"
 ],
 "Required": true,
 "SortOrder": 150.0,
 "IsNullable": false,
 "Default": 0
 }
 ]
 }
 ]
 },
 {
 "Kind": "StopWordsRemover",
 "Components": [
Expand Down
Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms");
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Update MacroUtils to map trainer kinds to the correct suffix of trainer entry point names#113

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

yaeldekel
merged 8 commits into

from

May 14, 2018
 Merged


Update MacroUtils to map trainer kinds to the correct suffix of trainer entry point names
#113
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


8 commits


Select commit
Hold shift + click to select a range
02f72c9

Update suffix of trainer entry point names by trainer kind.


yaeldMS May 10, 2018
ff6736d

Address PR comments.


yaeldMS May 10, 2018
1d01831

Add unit test.


yaeldMS May 10, 2018
f1bbeda

Merge remote-tracking branch 'upstream/master' into macroutils


yaeldMS May 11, 2018
db8492d

Update C# API


yaeldMS May 11, 2018
4267412

Move unit test to TestAutoInference and fix EntryPointCatalog test.


yaeldMS May 11, 2018
9bb6c89

Trigger build.


yaeldMS May 11, 2018
be06047

Add reference to the test project to make the sweeper entry point vis…


yaeldMS May 11, 2018

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


2 changes: 2 additions & 0 deletions
ZBaselines/Common/EntryPoints/core_ep-list.tsv

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -15,11 +15,13 @@ Models.NaiveCalibrator	Apply a Naive calibrator to an input model	Microsoft.ML.R
Models.OneVersusAll	One-vs-All macro (OVA)	Microsoft.ML.Runtime.EntryPoints.OneVersusAllMacro	OVA	Microsoft.ML.Runtime.EntryPoints.OneVersusAllMacro+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+MacroOutput`1[Microsoft.ML.Runtime.EntryPoints.OneVersusAllMacro+Output]
Models.OvaModelCombiner	Combines a sequence of PredictorModels into a single model	Microsoft.ML.Runtime.Learners.OvaPredictor	CombineOvaModels	Microsoft.ML.Runtime.EntryPoints.ModelOperations+CombineOvaPredictorModelsInput	Microsoft.ML.Runtime.EntryPoints.ModelOperations+PredictorModelOutput
Models.PAVCalibrator	Apply a PAV calibrator to an input model	Microsoft.ML.Runtime.Internal.Calibration.Calibrate	Pav	Microsoft.ML.Runtime.Internal.Calibration.Calibrate+NoArgumentsInput	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+CalibratorOutput
Models.PipelineSweeper	AutoML pipeline sweeping optimzation macro.	Microsoft.ML.Runtime.EntryPoints.PipelineSweeperMacro	PipelineSweep	Microsoft.ML.Runtime.EntryPoints.PipelineSweeperMacro+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+MacroOutput`1[Microsoft.ML.Runtime.EntryPoints.PipelineSweeperMacro+Output]
Models.PlattCalibrator	Apply a Platt calibrator to an input model	Microsoft.ML.Runtime.Internal.Calibration.Calibrate	Platt	Microsoft.ML.Runtime.Internal.Calibration.Calibrate+NoArgumentsInput	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+CalibratorOutput
Models.QuantileRegressionEvaluator	Evaluates a quantile regression scored dataset.	Microsoft.ML.Runtime.Data.Evaluate	QuantileRegression	Microsoft.ML.Runtime.Data.QuantileRegressionMamlEvaluator+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+CommonEvaluateOutput
Models.RankerEvaluator	Evaluates a ranking scored dataset.	Microsoft.ML.Runtime.Data.Evaluate	Ranking	Microsoft.ML.Runtime.Data.RankerMamlEvaluator+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+CommonEvaluateOutput
Models.RegressionEvaluator	Evaluates a regression scored dataset.	Microsoft.ML.Runtime.Data.Evaluate	Regression	Microsoft.ML.Runtime.Data.RegressionMamlEvaluator+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+CommonEvaluateOutput
Models.Summarizer	Summarize a linear regression predictor.	Microsoft.ML.Runtime.EntryPoints.SummarizePredictor	Summarize	Microsoft.ML.Runtime.EntryPoints.SummarizePredictor+Input	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+SummaryOutput
Models.SweepResultExtractor	Extracts the sweep result.	Microsoft.ML.Runtime.EntryPoints.PipelineSweeperMacro	ExtractSweepResult	Microsoft.ML.Runtime.EntryPoints.PipelineSweeperMacro+ResultInput	Microsoft.ML.Runtime.EntryPoints.PipelineSweeperMacro+Output
Models.TrainTestBinaryEvaluator	Train test for binary classification	Microsoft.ML.Runtime.EntryPoints.TrainTestBinaryMacro	TrainTestBinary	Microsoft.ML.Runtime.EntryPoints.TrainTestBinaryMacro+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+MacroOutput`1[Microsoft.ML.Runtime.EntryPoints.TrainTestBinaryMacro+Output]
Models.TrainTestEvaluator	General train test for any supported evaluator	Microsoft.ML.Runtime.EntryPoints.TrainTestMacro	TrainTest	Microsoft.ML.Runtime.EntryPoints.TrainTestMacro+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+MacroOutput`1[Microsoft.ML.Runtime.EntryPoints.TrainTestMacro+Output]
Trainers.AveragedPerceptronBinaryClassifier	Train a Average perceptron.	Microsoft.ML.Runtime.Learners.AveragedPerceptronTrainer	TrainBinary	Microsoft.ML.Runtime.Learners.AveragedPerceptronTrainer+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+BinaryClassificationOutput
Expand Down

358 changes: 358 additions & 0 deletions
ZBaselines/Common/EntryPoints/core_manifest.json

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -1723,6 +1723,105 @@
 "ITrainerOutput"
 ]
 },
 {
 "Name": "Models.PipelineSweeper",
 "Desc": "AutoML pipeline sweeping optimzation macro.",
 "FriendlyName": null,
 "ShortName": null,
 "Inputs": [
 {
 "Name": "TrainingData",
 "Type": "DataView",
 "Desc": "The data to be used for training.",
 "Aliases": [
 "train"
 ],
 "Required": true,
 "SortOrder": 1.0,
 "IsNullable": false
 },
 {
 "Name": "TestingData",
 "Type": "DataView",
 "Desc": "The data to be used for testing.",
 "Aliases": [
 "test"
 ],
 "Required": true,
 "SortOrder": 2.0,
 "IsNullable": false
 },
 {
 "Name": "StateArguments",
 "Type": {
 "Kind": "Component",
 "ComponentKind": "AutoMlStateBase"
 },
 "Desc": "The arguments for creating an AutoMlState component.",
 "Aliases": [
 "args"
 ],
 "Required": false,
 "SortOrder": 3.0,
 "IsNullable": false,
 "Default": null
 },
 {
 "Name": "State",
 "Type": {
 "Kind": "C# Object",
 "ItemType": "Microsoft.ML.Runtime.EntryPoints.IMlState"
 },
 "Desc": "The stateful object conducting of the autoML search.",
 "Aliases": [
 "state"
 ],
 "Required": false,
 "SortOrder": 3.0,
 "IsNullable": false,
 "Default": null
 },
 {
 "Name": "BatchSize",
 "Type": "Int",
 "Desc": "Number of candidate pipelines to retrieve each round.",
 "Aliases": [
 "bsize"
 ],
 "Required": true,
 "SortOrder": 4.0,
 "IsNullable": false,
 "Default": 0
 },
 {
 "Name": "CandidateOutputs",
 "Type": {
 "Kind": "Array",
 "ItemType": "DataView"
 },
 "Desc": "Output datasets from previous iteration of sweep.",
 "Required": false,
 "SortOrder": 7.0,
 "IsNullable": false,
 "Default": null
 }
 ],
 "Outputs": [
 {
 "Name": "State",
 "Type": {
 "Kind": "C# Object",
 "ItemType": "Microsoft.ML.Runtime.EntryPoints.IMlState"
 },
 "Desc": "Stateful autoML object, keeps track of where the search in progress."
 },
 {
 "Name": "Results",
 "Type": "DataView",
 "Desc": "Results of the sweep, including pipelines (as graph strings), IDs, and metric values."
 }
 ]
 },
 {
 "Name": "Models.PlattCalibrator",
 "Desc": "Apply a Platt calibrator to an input model",
Expand DownExpand Up@@ -2209,6 +2308,44 @@
 }
 ]
 },
 {
 "Name": "Models.SweepResultExtractor",
 "Desc": "Extracts the sweep result.",
 "FriendlyName": null,
 "ShortName": null,
 "Inputs": [
 {
 "Name": "State",
 "Type": {
 "Kind": "C# Object",
 "ItemType": "Microsoft.ML.Runtime.EntryPoints.IMlState"
 },
 "Desc": "The stateful object conducting of the autoML search.",
 "Aliases": [
 "state"
 ],
 "Required": false,
 "SortOrder": 1.0,
 "IsNullable": false,
 "Default": null
 }
 ],
 "Outputs": [
 {
 "Name": "State",
 "Type": {
 "Kind": "C# Object",
 "ItemType": "Microsoft.ML.Runtime.EntryPoints.IMlState"
 },
 "Desc": "Stateful autoML object, keeps track of where the search in progress."
 },
 {
 "Name": "Results",
 "Type": "DataView",
 "Desc": "Results of the sweep, including pipelines (as graph strings), IDs, and metric values."
 }
 ]
 },
 {
 "Name": "Models.TrainTestBinaryEvaluator",
 "Desc": "Train test for binary classification",
Expand DownExpand Up@@ -17200,6 +17337,203 @@
 }
 ],
 "Components": [
 {
 "Kind": "AutoMlEngine",
 "Components": [
 {
 "Name": "Defaults",
 "Desc": "AutoML engine that returns learners with default settings.",
 "FriendlyName": "Defaults Engine",
 "Settings": []
 },
 {
 "Name": "Rocket",
 "Desc": "AutoML engine that consists of distinct, hierarchical stages of operation.",
 "FriendlyName": "Rocket Engine",
 "Settings": [
 {
 "Name": "TopKLearners",
 "Type": "Int",
 "Desc": "Number of learners to retain for second stage.",
 "Aliases": [
 "topk"
 ],
 "Required": false,
 "SortOrder": 1.0,
 "IsNullable": false,
 "Default": 2
 },
 {
 "Name": "SecondRoundTrialsPerLearner",
 "Type": "Int",
 "Desc": "Number of trials for retained second stage learners.",
 "Aliases": [
 "stage2num"
 ],
 "Required": false,
 "SortOrder": 2.0,
 "IsNullable": false,
 "Default": 5
 },
 {
 "Name": "RandomInitialization",
 "Type": "Bool",
 "Desc": "Use random initialization only.",
 "Aliases": [
 "randinit"
 ],
 "Required": false,
 "SortOrder": 3.0,
 "IsNullable": false,
 "Default": false
 },
 {
 "Name": "NumInitializationPipelines",
 "Type": "Int",
 "Desc": "Number of initilization pipelines, used for random initialization only.",
 "Aliases": [
 "numinitseeds"
 ],
 "Required": false,
 "SortOrder": 4.0,
 "IsNullable": false,
 "Default": 20
 }
 ]
 },
 {
 "Name": "UniformRandom",
 "Desc": "AutoML engine using uniform random sampling.",
 "FriendlyName": "Uniform Random Engine",
 "Settings": []
 }
 ]
 },
 {
 "Kind": "AutoMlStateBase",
 "Components": [
 {
 "Name": "AutoMlState",
 "Desc": "State of an AutoML search and search space.",
 "FriendlyName": "AutoML State",
 "Aliases": [
 "automlst"
 ],
 "Settings": [
 {
 "Name": "Metric",
 "Type": {
 "Kind": "Enum",
 "Values": [
 "Auc",
 "AccuracyMicro",
 "AccuracyMacro",
 "L2",
 "F1",
 "AuPrc",
 "TopKAccuracy",
 "Rms",
 "LossFn",
 "RSquared",
 "LogLoss",
 "LogLossReduction",
 "Ndcg",
 "Dcg",
 "PositivePrecision",
 "PositiveRecall",
 "NegativePrecision",
 "NegativeRecall",
 "DrAtK",
 "DrAtPFpr",
 "DrAtNumPos",
 "NumAnomalies",
 "ThreshAtK",
 "ThreshAtP",
 "ThreshAtNumPos",
 "Nmi",
 "AvgMinScore",
 "Dbi"
 ]
 },
 "Desc": "Supported metric for evaluator.",
 "Aliases": [
 "metric"
 ],
 "Required": true,
 "SortOrder": 150.0,
 "IsNullable": false,
 "Default": "Auc"
 },
 {
 "Name": "Engine",
 "Type": {
 "Kind": "Component",
 "ComponentKind": "AutoMlEngine"
 },
 "Desc": "AutoML engine (pipeline optimizer) that generates next candidates.",
 "Aliases": [
 "engine"
 ],
 "Required": true,
 "SortOrder": 150.0,
 "IsNullable": false
 },
 {
 "Name": "TrainerKind",
 "Type": {
 "Kind": "Enum",
 "Values": [
 "SignatureBinaryClassifierTrainer",
 "SignatureMultiClassClassifierTrainer",
 "SignatureRankerTrainer",
 "SignatureRegressorTrainer",
 "SignatureMultiOutputRegressorTrainer",
 "SignatureAnomalyDetectorTrainer",
 "SignatureClusteringTrainer"
 ]
 },
 "Desc": "Kind of trainer for task, such as binary classification trainer, multiclass trainer, etc.",
 "Aliases": [
 "tk"
 ],
 "Required": true,
 "SortOrder": 150.0,
 "IsNullable": false,
 "Default": "SignatureBinaryClassifierTrainer"
 },
 {
 "Name": "TerminatorArgs",
 "Type": {
 "Kind": "Component",
 "ComponentKind": "SearchTerminator"
 },
 "Desc": "Arguments for creating terminator, which determines when to stop search.",
 "Aliases": [
 "term"
 ],
 "Required": true,
 "SortOrder": 150.0,
 "IsNullable": false
 },
 {
 "Name": "RequestedLearners",
 "Type": {
 "Kind": "Array",
 "ItemType": "String"
 },
 "Desc": "Learner set to sweep over (if available).",
 "Aliases": [
 "learners"
 ],
 "Required": false,
 "SortOrder": 150.0,
 "IsNullable": false,
 "Default": null
 }
 ]
 }
 ]
 },
 {
 "Kind": "CalibratorTrainer",
 "Components": [
Expand DownExpand Up@@ -21441,6 +21775,30 @@
 }
 ]
 },
 {
 "Kind": "SearchTerminator",
 "Components": [
 {
 "Name": "IterationLimited",
 "Desc": "Terminators a sweep based on total number of iterations.",
 "FriendlyName": "Pipeline Sweep Iteration Terminator",
 "Settings": [
 {
 "Name": "FinalHistoryLength",
 "Type": "Int",
 "Desc": "Total number of iterations.",
 "Aliases": [
 "length"
 ],
 "Required": true,
 "SortOrder": 150.0,
 "IsNullable": false,
 "Default": 0
 }
 ]
 }
 ]
 },
 {
 "Kind": "StopWordsRemover",
 "Components": [
Expand Down
Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links");
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '

Skip to content
Update MacroUtils to map trainer kinds to the correct suffix of trainer entry point names#113

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

yaeldekel
merged 8 commits into

from

May 14, 2018
 Merged


Update MacroUtils to map trainer kinds to the correct suffix of trainer entry point names
#113
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


8 commits


Select commit
Hold shift + click to select a range
02f72c9

Update suffix of trainer entry point names by trainer kind.


yaeldMS May 10, 2018
ff6736d

Address PR comments.


yaeldMS May 10, 2018
1d01831

Add unit test.


yaeldMS May 10, 2018
f1bbeda

Merge remote-tracking branch 'upstream/master' into macroutils


yaeldMS May 11, 2018
db8492d

Update C# API


yaeldMS May 11, 2018
4267412

Move unit test to TestAutoInference and fix EntryPointCatalog test.


yaeldMS May 11, 2018
9bb6c89

Trigger build.


yaeldMS May 11, 2018
be06047

Add reference to the test project to make the sweeper entry point vis…


yaeldMS May 11, 2018

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


2 changes: 2 additions & 0 deletions
ZBaselines/Common/EntryPoints/core_ep-list.tsv

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -15,11 +15,13 @@ Models.NaiveCalibrator	Apply a Naive calibrator to an input model	Microsoft.ML.R
Models.OneVersusAll	One-vs-All macro (OVA)	Microsoft.ML.Runtime.EntryPoints.OneVersusAllMacro	OVA	Microsoft.ML.Runtime.EntryPoints.OneVersusAllMacro+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+MacroOutput`1[Microsoft.ML.Runtime.EntryPoints.OneVersusAllMacro+Output]
Models.OvaModelCombiner	Combines a sequence of PredictorModels into a single model	Microsoft.ML.Runtime.Learners.OvaPredictor	CombineOvaModels	Microsoft.ML.Runtime.EntryPoints.ModelOperations+CombineOvaPredictorModelsInput	Microsoft.ML.Runtime.EntryPoints.ModelOperations+PredictorModelOutput
Models.PAVCalibrator	Apply a PAV calibrator to an input model	Microsoft.ML.Runtime.Internal.Calibration.Calibrate	Pav	Microsoft.ML.Runtime.Internal.Calibration.Calibrate+NoArgumentsInput	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+CalibratorOutput
Models.PipelineSweeper	AutoML pipeline sweeping optimzation macro.	Microsoft.ML.Runtime.EntryPoints.PipelineSweeperMacro	PipelineSweep	Microsoft.ML.Runtime.EntryPoints.PipelineSweeperMacro+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+MacroOutput`1[Microsoft.ML.Runtime.EntryPoints.PipelineSweeperMacro+Output]
Models.PlattCalibrator	Apply a Platt calibrator to an input model	Microsoft.ML.Runtime.Internal.Calibration.Calibrate	Platt	Microsoft.ML.Runtime.Internal.Calibration.Calibrate+NoArgumentsInput	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+CalibratorOutput
Models.QuantileRegressionEvaluator	Evaluates a quantile regression scored dataset.	Microsoft.ML.Runtime.Data.Evaluate	QuantileRegression	Microsoft.ML.Runtime.Data.QuantileRegressionMamlEvaluator+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+CommonEvaluateOutput
Models.RankerEvaluator	Evaluates a ranking scored dataset.	Microsoft.ML.Runtime.Data.Evaluate	Ranking	Microsoft.ML.Runtime.Data.RankerMamlEvaluator+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+CommonEvaluateOutput
Models.RegressionEvaluator	Evaluates a regression scored dataset.	Microsoft.ML.Runtime.Data.Evaluate	Regression	Microsoft.ML.Runtime.Data.RegressionMamlEvaluator+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+CommonEvaluateOutput
Models.Summarizer	Summarize a linear regression predictor.	Microsoft.ML.Runtime.EntryPoints.SummarizePredictor	Summarize	Microsoft.ML.Runtime.EntryPoints.SummarizePredictor+Input	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+SummaryOutput
Models.SweepResultExtractor	Extracts the sweep result.	Microsoft.ML.Runtime.EntryPoints.PipelineSweeperMacro	ExtractSweepResult	Microsoft.ML.Runtime.EntryPoints.PipelineSweeperMacro+ResultInput	Microsoft.ML.Runtime.EntryPoints.PipelineSweeperMacro+Output
Models.TrainTestBinaryEvaluator	Train test for binary classification	Microsoft.ML.Runtime.EntryPoints.TrainTestBinaryMacro	TrainTestBinary	Microsoft.ML.Runtime.EntryPoints.TrainTestBinaryMacro+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+MacroOutput`1[Microsoft.ML.Runtime.EntryPoints.TrainTestBinaryMacro+Output]
Models.TrainTestEvaluator	General train test for any supported evaluator	Microsoft.ML.Runtime.EntryPoints.TrainTestMacro	TrainTest	Microsoft.ML.Runtime.EntryPoints.TrainTestMacro+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+MacroOutput`1[Microsoft.ML.Runtime.EntryPoints.TrainTestMacro+Output]
Trainers.AveragedPerceptronBinaryClassifier	Train a Average perceptron.	Microsoft.ML.Runtime.Learners.AveragedPerceptronTrainer	TrainBinary	Microsoft.ML.Runtime.Learners.AveragedPerceptronTrainer+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+BinaryClassificationOutput
Expand Down

358 changes: 358 additions & 0 deletions
ZBaselines/Common/EntryPoints/core_manifest.json

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -1723,6 +1723,105 @@
 "ITrainerOutput"
 ]
 },
 {
 "Name": "Models.PipelineSweeper",
 "Desc": "AutoML pipeline sweeping optimzation macro.",
 "FriendlyName": null,
 "ShortName": null,
 "Inputs": [
 {
 "Name": "TrainingData",
 "Type": "DataView",
 "Desc": "The data to be used for training.",
 "Aliases": [
 "train"
 ],
 "Required": true,
 "SortOrder": 1.0,
 "IsNullable": false
 },
 {
 "Name": "TestingData",
 "Type": "DataView",
 "Desc": "The data to be used for testing.",
 "Aliases": [
 "test"
 ],
 "Required": true,
 "SortOrder": 2.0,
 "IsNullable": false
 },
 {
 "Name": "StateArguments",
 "Type": {
 "Kind": "Component",
 "ComponentKind": "AutoMlStateBase"
 },
 "Desc": "The arguments for creating an AutoMlState component.",
 "Aliases": [
 "args"
 ],
 "Required": false,
 "SortOrder": 3.0,
 "IsNullable": false,
 "Default": null
 },
 {
 "Name": "State",
 "Type": {
 "Kind": "C# Object",
 "ItemType": "Microsoft.ML.Runtime.EntryPoints.IMlState"
 },
 "Desc": "The stateful object conducting of the autoML search.",
 "Aliases": [
 "state"
 ],
 "Required": false,
 "SortOrder": 3.0,
 "IsNullable": false,
 "Default": null
 },
 {
 "Name": "BatchSize",
 "Type": "Int",
 "Desc": "Number of candidate pipelines to retrieve each round.",
 "Aliases": [
 "bsize"
 ],
 "Required": true,
 "SortOrder": 4.0,
 "IsNullable": false,
 "Default": 0
 },
 {
 "Name": "CandidateOutputs",
 "Type": {
 "Kind": "Array",
 "ItemType": "DataView"
 },
 "Desc": "Output datasets from previous iteration of sweep.",
 "Required": false,
 "SortOrder": 7.0,
 "IsNullable": false,
 "Default": null
 }
 ],
 "Outputs": [
 {
 "Name": "State",
 "Type": {
 "Kind": "C# Object",
 "ItemType": "Microsoft.ML.Runtime.EntryPoints.IMlState"
 },
 "Desc": "Stateful autoML object, keeps track of where the search in progress."
 },
 {
 "Name": "Results",
 "Type": "DataView",
 "Desc": "Results of the sweep, including pipelines (as graph strings), IDs, and metric values."
 }
 ]
 },
 {
 "Name": "Models.PlattCalibrator",
 "Desc": "Apply a Platt calibrator to an input model",
Expand DownExpand Up@@ -2209,6 +2308,44 @@
 }
 ]
 },
 {
 "Name": "Models.SweepResultExtractor",
 "Desc": "Extracts the sweep result.",
 "FriendlyName": null,
 "ShortName": null,
 "Inputs": [
 {
 "Name": "State",
 "Type": {
 "Kind": "C# Object",
 "ItemType": "Microsoft.ML.Runtime.EntryPoints.IMlState"
 },
 "Desc": "The stateful object conducting of the autoML search.",
 "Aliases": [
 "state"
 ],
 "Required": false,
 "SortOrder": 1.0,
 "IsNullable": false,
 "Default": null
 }
 ],
 "Outputs": [
 {
 "Name": "State",
 "Type": {
 "Kind": "C# Object",
 "ItemType": "Microsoft.ML.Runtime.EntryPoints.IMlState"
 },
 "Desc": "Stateful autoML object, keeps track of where the search in progress."
 },
 {
 "Name": "Results",
 "Type": "DataView",
 "Desc": "Results of the sweep, including pipelines (as graph strings), IDs, and metric values."
 }
 ]
 },
 {
 "Name": "Models.TrainTestBinaryEvaluator",
 "Desc": "Train test for binary classification",
Expand DownExpand Up@@ -17200,6 +17337,203 @@
 }
 ],
 "Components": [
 {
 "Kind": "AutoMlEngine",
 "Components": [
 {
 "Name": "Defaults",
 "Desc": "AutoML engine that returns learners with default settings.",
 "FriendlyName": "Defaults Engine",
 "Settings": []
 },
 {
 "Name": "Rocket",
 "Desc": "AutoML engine that consists of distinct, hierarchical stages of operation.",
 "FriendlyName": "Rocket Engine",
 "Settings": [
 {
 "Name": "TopKLearners",
 "Type": "Int",
 "Desc": "Number of learners to retain for second stage.",
 "Aliases": [
 "topk"
 ],
 "Required": false,
 "SortOrder": 1.0,
 "IsNullable": false,
 "Default": 2
 },
 {
 "Name": "SecondRoundTrialsPerLearner",
 "Type": "Int",
 "Desc": "Number of trials for retained second stage learners.",
 "Aliases": [
 "stage2num"
 ],
 "Required": false,
 "SortOrder": 2.0,
 "IsNullable": false,
 "Default": 5
 },
 {
 "Name": "RandomInitialization",
 "Type": "Bool",
 "Desc": "Use random initialization only.",
 "Aliases": [
 "randinit"
 ],
 "Required": false,
 "SortOrder": 3.0,
 "IsNullable": false,
 "Default": false
 },
 {
 "Name": "NumInitializationPipelines",
 "Type": "Int",
 "Desc": "Number of initilization pipelines, used for random initialization only.",
 "Aliases": [
 "numinitseeds"
 ],
 "Required": false,
 "SortOrder": 4.0,
 "IsNullable": false,
 "Default": 20
 }
 ]
 },
 {
 "Name": "UniformRandom",
 "Desc": "AutoML engine using uniform random sampling.",
 "FriendlyName": "Uniform Random Engine",
 "Settings": []
 }
 ]
 },
 {
 "Kind": "AutoMlStateBase",
 "Components": [
 {
 "Name": "AutoMlState",
 "Desc": "State of an AutoML search and search space.",
 "FriendlyName": "AutoML State",
 "Aliases": [
 "automlst"
 ],
 "Settings": [
 {
 "Name": "Metric",
 "Type": {
 "Kind": "Enum",
 "Values": [
 "Auc",
 "AccuracyMicro",
 "AccuracyMacro",
 "L2",
 "F1",
 "AuPrc",
 "TopKAccuracy",
 "Rms",
 "LossFn",
 "RSquared",
 "LogLoss",
 "LogLossReduction",
 "Ndcg",
 "Dcg",
 "PositivePrecision",
 "PositiveRecall",
 "NegativePrecision",
 "NegativeRecall",
 "DrAtK",
 "DrAtPFpr",
 "DrAtNumPos",
 "NumAnomalies",
 "ThreshAtK",
 "ThreshAtP",
 "ThreshAtNumPos",
 "Nmi",
 "AvgMinScore",
 "Dbi"
 ]
 },
 "Desc": "Supported metric for evaluator.",
 "Aliases": [
 "metric"
 ],
 "Required": true,
 "SortOrder": 150.0,
 "IsNullable": false,
 "Default": "Auc"
 },
 {
 "Name": "Engine",
 "Type": {
 "Kind": "Component",
 "ComponentKind": "AutoMlEngine"
 },
 "Desc": "AutoML engine (pipeline optimizer) that generates next candidates.",
 "Aliases": [
 "engine"
 ],
 "Required": true,
 "SortOrder": 150.0,
 "IsNullable": false
 },
 {
 "Name": "TrainerKind",
 "Type": {
 "Kind": "Enum",
 "Values": [
 "SignatureBinaryClassifierTrainer",
 "SignatureMultiClassClassifierTrainer",
 "SignatureRankerTrainer",
 "SignatureRegressorTrainer",
 "SignatureMultiOutputRegressorTrainer",
 "SignatureAnomalyDetectorTrainer",
 "SignatureClusteringTrainer"
 ]
 },
 "Desc": "Kind of trainer for task, such as binary classification trainer, multiclass trainer, etc.",
 "Aliases": [
 "tk"
 ],
 "Required": true,
 "SortOrder": 150.0,
 "IsNullable": false,
 "Default": "SignatureBinaryClassifierTrainer"
 },
 {
 "Name": "TerminatorArgs",
 "Type": {
 "Kind": "Component",
 "ComponentKind": "SearchTerminator"
 },
 "Desc": "Arguments for creating terminator, which determines when to stop search.",
 "Aliases": [
 "term"
 ],
 "Required": true,
 "SortOrder": 150.0,
 "IsNullable": false
 },
 {
 "Name": "RequestedLearners",
 "Type": {
 "Kind": "Array",
 "ItemType": "String"
 },
 "Desc": "Learner set to sweep over (if available).",
 "Aliases": [
 "learners"
 ],
 "Required": false,
 "SortOrder": 150.0,
 "IsNullable": false,
 "Default": null
 }
 ]
 }
 ]
 },
 {
 "Kind": "CalibratorTrainer",
 "Components": [
Expand DownExpand Up@@ -21441,6 +21775,30 @@
 }
 ]
 },
 {
 "Kind": "SearchTerminator",
 "Components": [
 {
 "Name": "IterationLimited",
 "Desc": "Terminators a sweep based on total number of iterations.",
 "FriendlyName": "Pipeline Sweep Iteration Terminator",
 "Settings": [
 {
 "Name": "FinalHistoryLength",
 "Type": "Int",
 "Desc": "Total number of iterations.",
 "Aliases": [
 "length"
 ],
 "Required": true,
 "SortOrder": 150.0,
 "IsNullable": false,
 "Default": 0
 }
 ]
 }
 ]
 },
 {
 "Kind": "StopWordsRemover",
 "Components": [
Expand Down
Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default");
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Update MacroUtils to map trainer kinds to the correct suffix of trainer entry point names#113

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

yaeldekel
merged 8 commits into

from

May 14, 2018
 Merged


Update MacroUtils to map trainer kinds to the correct suffix of trainer entry point names
#113
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


8 commits


Select commit
Hold shift + click to select a range
02f72c9

Update suffix of trainer entry point names by trainer kind.


yaeldMS May 10, 2018
ff6736d

Address PR comments.


yaeldMS May 10, 2018
1d01831

Add unit test.


yaeldMS May 10, 2018
f1bbeda

Merge remote-tracking branch 'upstream/master' into macroutils


yaeldMS May 11, 2018
db8492d

Update C# API


yaeldMS May 11, 2018
4267412

Move unit test to TestAutoInference and fix EntryPointCatalog test.


yaeldMS May 11, 2018
9bb6c89

Trigger build.


yaeldMS May 11, 2018
be06047

Add reference to the test project to make the sweeper entry point vis…


yaeldMS May 11, 2018

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


2 changes: 2 additions & 0 deletions
ZBaselines/Common/EntryPoints/core_ep-list.tsv

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -15,11 +15,13 @@ Models.NaiveCalibrator	Apply a Naive calibrator to an input model	Microsoft.ML.R
Models.OneVersusAll	One-vs-All macro (OVA)	Microsoft.ML.Runtime.EntryPoints.OneVersusAllMacro	OVA	Microsoft.ML.Runtime.EntryPoints.OneVersusAllMacro+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+MacroOutput`1[Microsoft.ML.Runtime.EntryPoints.OneVersusAllMacro+Output]
Models.OvaModelCombiner	Combines a sequence of PredictorModels into a single model	Microsoft.ML.Runtime.Learners.OvaPredictor	CombineOvaModels	Microsoft.ML.Runtime.EntryPoints.ModelOperations+CombineOvaPredictorModelsInput	Microsoft.ML.Runtime.EntryPoints.ModelOperations+PredictorModelOutput
Models.PAVCalibrator	Apply a PAV calibrator to an input model	Microsoft.ML.Runtime.Internal.Calibration.Calibrate	Pav	Microsoft.ML.Runtime.Internal.Calibration.Calibrate+NoArgumentsInput	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+CalibratorOutput
Models.PipelineSweeper	AutoML pipeline sweeping optimzation macro.	Microsoft.ML.Runtime.EntryPoints.PipelineSweeperMacro	PipelineSweep	Microsoft.ML.Runtime.EntryPoints.PipelineSweeperMacro+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+MacroOutput`1[Microsoft.ML.Runtime.EntryPoints.PipelineSweeperMacro+Output]
Models.PlattCalibrator	Apply a Platt calibrator to an input model	Microsoft.ML.Runtime.Internal.Calibration.Calibrate	Platt	Microsoft.ML.Runtime.Internal.Calibration.Calibrate+NoArgumentsInput	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+CalibratorOutput
Models.QuantileRegressionEvaluator	Evaluates a quantile regression scored dataset.	Microsoft.ML.Runtime.Data.Evaluate	QuantileRegression	Microsoft.ML.Runtime.Data.QuantileRegressionMamlEvaluator+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+CommonEvaluateOutput
Models.RankerEvaluator	Evaluates a ranking scored dataset.	Microsoft.ML.Runtime.Data.Evaluate	Ranking	Microsoft.ML.Runtime.Data.RankerMamlEvaluator+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+CommonEvaluateOutput
Models.RegressionEvaluator	Evaluates a regression scored dataset.	Microsoft.ML.Runtime.Data.Evaluate	Regression	Microsoft.ML.Runtime.Data.RegressionMamlEvaluator+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+CommonEvaluateOutput
Models.Summarizer	Summarize a linear regression predictor.	Microsoft.ML.Runtime.EntryPoints.SummarizePredictor	Summarize	Microsoft.ML.Runtime.EntryPoints.SummarizePredictor+Input	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+SummaryOutput
Models.SweepResultExtractor	Extracts the sweep result.	Microsoft.ML.Runtime.EntryPoints.PipelineSweeperMacro	ExtractSweepResult	Microsoft.ML.Runtime.EntryPoints.PipelineSweeperMacro+ResultInput	Microsoft.ML.Runtime.EntryPoints.PipelineSweeperMacro+Output
Models.TrainTestBinaryEvaluator	Train test for binary classification	Microsoft.ML.Runtime.EntryPoints.TrainTestBinaryMacro	TrainTestBinary	Microsoft.ML.Runtime.EntryPoints.TrainTestBinaryMacro+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+MacroOutput`1[Microsoft.ML.Runtime.EntryPoints.TrainTestBinaryMacro+Output]
Models.TrainTestEvaluator	General train test for any supported evaluator	Microsoft.ML.Runtime.EntryPoints.TrainTestMacro	TrainTest	Microsoft.ML.Runtime.EntryPoints.TrainTestMacro+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+MacroOutput`1[Microsoft.ML.Runtime.EntryPoints.TrainTestMacro+Output]
Trainers.AveragedPerceptronBinaryClassifier	Train a Average perceptron.	Microsoft.ML.Runtime.Learners.AveragedPerceptronTrainer	TrainBinary	Microsoft.ML.Runtime.Learners.AveragedPerceptronTrainer+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+BinaryClassificationOutput
Expand Down

358 changes: 358 additions & 0 deletions
ZBaselines/Common/EntryPoints/core_manifest.json

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -1723,6 +1723,105 @@
 "ITrainerOutput"
 ]
 },
 {
 "Name": "Models.PipelineSweeper",
 "Desc": "AutoML pipeline sweeping optimzation macro.",
 "FriendlyName": null,
 "ShortName": null,
 "Inputs": [
 {
 "Name": "TrainingData",
 "Type": "DataView",
 "Desc": "The data to be used for training.",
 "Aliases": [
 "train"
 ],
 "Required": true,
 "SortOrder": 1.0,
 "IsNullable": false
 },
 {
 "Name": "TestingData",
 "Type": "DataView",
 "Desc": "The data to be used for testing.",
 "Aliases": [
 "test"
 ],
 "Required": true,
 "SortOrder": 2.0,
 "IsNullable": false
 },
 {
 "Name": "StateArguments",
 "Type": {
 "Kind": "Component",
 "ComponentKind": "AutoMlStateBase"
 },
 "Desc": "The arguments for creating an AutoMlState component.",
 "Aliases": [
 "args"
 ],
 "Required": false,
 "SortOrder": 3.0,
 "IsNullable": false,
 "Default": null
 },
 {
 "Name": "State",
 "Type": {
 "Kind": "C# Object",
 "ItemType": "Microsoft.ML.Runtime.EntryPoints.IMlState"
 },
 "Desc": "The stateful object conducting of the autoML search.",
 "Aliases": [
 "state"
 ],
 "Required": false,
 "SortOrder": 3.0,
 "IsNullable": false,
 "Default": null
 },
 {
 "Name": "BatchSize",
 "Type": "Int",
 "Desc": "Number of candidate pipelines to retrieve each round.",
 "Aliases": [
 "bsize"
 ],
 "Required": true,
 "SortOrder": 4.0,
 "IsNullable": false,
 "Default": 0
 },
 {
 "Name": "CandidateOutputs",
 "Type": {
 "Kind": "Array",
 "ItemType": "DataView"
 },
 "Desc": "Output datasets from previous iteration of sweep.",
 "Required": false,
 "SortOrder": 7.0,
 "IsNullable": false,
 "Default": null
 }
 ],
 "Outputs": [
 {
 "Name": "State",
 "Type": {
 "Kind": "C# Object",
 "ItemType": "Microsoft.ML.Runtime.EntryPoints.IMlState"
 },
 "Desc": "Stateful autoML object, keeps track of where the search in progress."
 },
 {
 "Name": "Results",
 "Type": "DataView",
 "Desc": "Results of the sweep, including pipelines (as graph strings), IDs, and metric values."
 }
 ]
 },
 {
 "Name": "Models.PlattCalibrator",
 "Desc": "Apply a Platt calibrator to an input model",
Expand DownExpand Up@@ -2209,6 +2308,44 @@
 }
 ]
 },
 {
 "Name": "Models.SweepResultExtractor",
 "Desc": "Extracts the sweep result.",
 "FriendlyName": null,
 "ShortName": null,
 "Inputs": [
 {
 "Name": "State",
 "Type": {
 "Kind": "C# Object",
 "ItemType": "Microsoft.ML.Runtime.EntryPoints.IMlState"
 },
 "Desc": "The stateful object conducting of the autoML search.",
 "Aliases": [
 "state"
 ],
 "Required": false,
 "SortOrder": 1.0,
 "IsNullable": false,
 "Default": null
 }
 ],
 "Outputs": [
 {
 "Name": "State",
 "Type": {
 "Kind": "C# Object",
 "ItemType": "Microsoft.ML.Runtime.EntryPoints.IMlState"
 },
 "Desc": "Stateful autoML object, keeps track of where the search in progress."
 },
 {
 "Name": "Results",
 "Type": "DataView",
 "Desc": "Results of the sweep, including pipelines (as graph strings), IDs, and metric values."
 }
 ]
 },
 {
 "Name": "Models.TrainTestBinaryEvaluator",
 "Desc": "Train test for binary classification",
Expand DownExpand Up@@ -17200,6 +17337,203 @@
 }
 ],
 "Components": [
 {
 "Kind": "AutoMlEngine",
 "Components": [
 {
 "Name": "Defaults",
 "Desc": "AutoML engine that returns learners with default settings.",
 "FriendlyName": "Defaults Engine",
 "Settings": []
 },
 {
 "Name": "Rocket",
 "Desc": "AutoML engine that consists of distinct, hierarchical stages of operation.",
 "FriendlyName": "Rocket Engine",
 "Settings": [
 {
 "Name": "TopKLearners",
 "Type": "Int",
 "Desc": "Number of learners to retain for second stage.",
 "Aliases": [
 "topk"
 ],
 "Required": false,
 "SortOrder": 1.0,
 "IsNullable": false,
 "Default": 2
 },
 {
 "Name": "SecondRoundTrialsPerLearner",
 "Type": "Int",
 "Desc": "Number of trials for retained second stage learners.",
 "Aliases": [
 "stage2num"
 ],
 "Required": false,
 "SortOrder": 2.0,
 "IsNullable": false,
 "Default": 5
 },
 {
 "Name": "RandomInitialization",
 "Type": "Bool",
 "Desc": "Use random initialization only.",
 "Aliases": [
 "randinit"
 ],
 "Required": false,
 "SortOrder": 3.0,
 "IsNullable": false,
 "Default": false
 },
 {
 "Name": "NumInitializationPipelines",
 "Type": "Int",
 "Desc": "Number of initilization pipelines, used for random initialization only.",
 "Aliases": [
 "numinitseeds"
 ],
 "Required": false,
 "SortOrder": 4.0,
 "IsNullable": false,
 "Default": 20
 }
 ]
 },
 {
 "Name": "UniformRandom",
 "Desc": "AutoML engine using uniform random sampling.",
 "FriendlyName": "Uniform Random Engine",
 "Settings": []
 }
 ]
 },
 {
 "Kind": "AutoMlStateBase",
 "Components": [
 {
 "Name": "AutoMlState",
 "Desc": "State of an AutoML search and search space.",
 "FriendlyName": "AutoML State",
 "Aliases": [
 "automlst"
 ],
 "Settings": [
 {
 "Name": "Metric",
 "Type": {
 "Kind": "Enum",
 "Values": [
 "Auc",
 "AccuracyMicro",
 "AccuracyMacro",
 "L2",
 "F1",
 "AuPrc",
 "TopKAccuracy",
 "Rms",
 "LossFn",
 "RSquared",
 "LogLoss",
 "LogLossReduction",
 "Ndcg",
 "Dcg",
 "PositivePrecision",
 "PositiveRecall",
 "NegativePrecision",
 "NegativeRecall",
 "DrAtK",
 "DrAtPFpr",
 "DrAtNumPos",
 "NumAnomalies",
 "ThreshAtK",
 "ThreshAtP",
 "ThreshAtNumPos",
 "Nmi",
 "AvgMinScore",
 "Dbi"
 ]
 },
 "Desc": "Supported metric for evaluator.",
 "Aliases": [
 "metric"
 ],
 "Required": true,
 "SortOrder": 150.0,
 "IsNullable": false,
 "Default": "Auc"
 },
 {
 "Name": "Engine",
 "Type": {
 "Kind": "Component",
 "ComponentKind": "AutoMlEngine"
 },
 "Desc": "AutoML engine (pipeline optimizer) that generates next candidates.",
 "Aliases": [
 "engine"
 ],
 "Required": true,
 "SortOrder": 150.0,
 "IsNullable": false
 },
 {
 "Name": "TrainerKind",
 "Type": {
 "Kind": "Enum",
 "Values": [
 "SignatureBinaryClassifierTrainer",
 "SignatureMultiClassClassifierTrainer",
 "SignatureRankerTrainer",
 "SignatureRegressorTrainer",
 "SignatureMultiOutputRegressorTrainer",
 "SignatureAnomalyDetectorTrainer",
 "SignatureClusteringTrainer"
 ]
 },
 "Desc": "Kind of trainer for task, such as binary classification trainer, multiclass trainer, etc.",
 "Aliases": [
 "tk"
 ],
 "Required": true,
 "SortOrder": 150.0,
 "IsNullable": false,
 "Default": "SignatureBinaryClassifierTrainer"
 },
 {
 "Name": "TerminatorArgs",
 "Type": {
 "Kind": "Component",
 "ComponentKind": "SearchTerminator"
 },
 "Desc": "Arguments for creating terminator, which determines when to stop search.",
 "Aliases": [
 "term"
 ],
 "Required": true,
 "SortOrder": 150.0,
 "IsNullable": false
 },
 {
 "Name": "RequestedLearners",
 "Type": {
 "Kind": "Array",
 "ItemType": "String"
 },
 "Desc": "Learner set to sweep over (if available).",
 "Aliases": [
 "learners"
 ],
 "Required": false,
 "SortOrder": 150.0,
 "IsNullable": false,
 "Default": null
 }
 ]
 }
 ]
 },
 {
 "Kind": "CalibratorTrainer",
 "Components": [
Expand DownExpand Up@@ -21441,6 +21775,30 @@
 }
 ]
 },
 {
 "Kind": "SearchTerminator",
 "Components": [
 {
 "Name": "IterationLimited",
 "Desc": "Terminators a sweep based on total number of iterations.",
 "FriendlyName": "Pipeline Sweep Iteration Terminator",
 "Settings": [
 {
 "Name": "FinalHistoryLength",
 "Type": "Int",
 "Desc": "Total number of iterations.",
 "Aliases": [
 "length"
 ],
 "Required": true,
 "SortOrder": 150.0,
 "IsNullable": false,
 "Default": 0
 }
 ]
 }
 ]
 },
 {
 "Kind": "StopWordsRemover",
 "Components": [
Expand Down
Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers");
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Update MacroUtils to map trainer kinds to the correct suffix of trainer entry point names#113

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

yaeldekel
merged 8 commits into

from

May 14, 2018
 Merged


Update MacroUtils to map trainer kinds to the correct suffix of trainer entry point names
#113
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


8 commits


Select commit
Hold shift + click to select a range
02f72c9

Update suffix of trainer entry point names by trainer kind.


yaeldMS May 10, 2018
ff6736d

Address PR comments.


yaeldMS May 10, 2018
1d01831

Add unit test.


yaeldMS May 10, 2018
f1bbeda

Merge remote-tracking branch 'upstream/master' into macroutils


yaeldMS May 11, 2018
db8492d

Update C# API


yaeldMS May 11, 2018
4267412

Move unit test to TestAutoInference and fix EntryPointCatalog test.


yaeldMS May 11, 2018
9bb6c89

Trigger build.


yaeldMS May 11, 2018
be06047

Add reference to the test project to make the sweeper entry point vis…


yaeldMS May 11, 2018

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


2 changes: 2 additions & 0 deletions
ZBaselines/Common/EntryPoints/core_ep-list.tsv

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -15,11 +15,13 @@ Models.NaiveCalibrator	Apply a Naive calibrator to an input model	Microsoft.ML.R
Models.OneVersusAll	One-vs-All macro (OVA)	Microsoft.ML.Runtime.EntryPoints.OneVersusAllMacro	OVA	Microsoft.ML.Runtime.EntryPoints.OneVersusAllMacro+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+MacroOutput`1[Microsoft.ML.Runtime.EntryPoints.OneVersusAllMacro+Output]
Models.OvaModelCombiner	Combines a sequence of PredictorModels into a single model	Microsoft.ML.Runtime.Learners.OvaPredictor	CombineOvaModels	Microsoft.ML.Runtime.EntryPoints.ModelOperations+CombineOvaPredictorModelsInput	Microsoft.ML.Runtime.EntryPoints.ModelOperations+PredictorModelOutput
Models.PAVCalibrator	Apply a PAV calibrator to an input model	Microsoft.ML.Runtime.Internal.Calibration.Calibrate	Pav	Microsoft.ML.Runtime.Internal.Calibration.Calibrate+NoArgumentsInput	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+CalibratorOutput
Models.PipelineSweeper	AutoML pipeline sweeping optimzation macro.	Microsoft.ML.Runtime.EntryPoints.PipelineSweeperMacro	PipelineSweep	Microsoft.ML.Runtime.EntryPoints.PipelineSweeperMacro+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+MacroOutput`1[Microsoft.ML.Runtime.EntryPoints.PipelineSweeperMacro+Output]
Models.PlattCalibrator	Apply a Platt calibrator to an input model	Microsoft.ML.Runtime.Internal.Calibration.Calibrate	Platt	Microsoft.ML.Runtime.Internal.Calibration.Calibrate+NoArgumentsInput	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+CalibratorOutput
Models.QuantileRegressionEvaluator	Evaluates a quantile regression scored dataset.	Microsoft.ML.Runtime.Data.Evaluate	QuantileRegression	Microsoft.ML.Runtime.Data.QuantileRegressionMamlEvaluator+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+CommonEvaluateOutput
Models.RankerEvaluator	Evaluates a ranking scored dataset.	Microsoft.ML.Runtime.Data.Evaluate	Ranking	Microsoft.ML.Runtime.Data.RankerMamlEvaluator+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+CommonEvaluateOutput
Models.RegressionEvaluator	Evaluates a regression scored dataset.	Microsoft.ML.Runtime.Data.Evaluate	Regression	Microsoft.ML.Runtime.Data.RegressionMamlEvaluator+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+CommonEvaluateOutput
Models.Summarizer	Summarize a linear regression predictor.	Microsoft.ML.Runtime.EntryPoints.SummarizePredictor	Summarize	Microsoft.ML.Runtime.EntryPoints.SummarizePredictor+Input	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+SummaryOutput
Models.SweepResultExtractor	Extracts the sweep result.	Microsoft.ML.Runtime.EntryPoints.PipelineSweeperMacro	ExtractSweepResult	Microsoft.ML.Runtime.EntryPoints.PipelineSweeperMacro+ResultInput	Microsoft.ML.Runtime.EntryPoints.PipelineSweeperMacro+Output
Models.TrainTestBinaryEvaluator	Train test for binary classification	Microsoft.ML.Runtime.EntryPoints.TrainTestBinaryMacro	TrainTestBinary	Microsoft.ML.Runtime.EntryPoints.TrainTestBinaryMacro+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+MacroOutput`1[Microsoft.ML.Runtime.EntryPoints.TrainTestBinaryMacro+Output]
Models.TrainTestEvaluator	General train test for any supported evaluator	Microsoft.ML.Runtime.EntryPoints.TrainTestMacro	TrainTest	Microsoft.ML.Runtime.EntryPoints.TrainTestMacro+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+MacroOutput`1[Microsoft.ML.Runtime.EntryPoints.TrainTestMacro+Output]
Trainers.AveragedPerceptronBinaryClassifier	Train a Average perceptron.	Microsoft.ML.Runtime.Learners.AveragedPerceptronTrainer	TrainBinary	Microsoft.ML.Runtime.Learners.AveragedPerceptronTrainer+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+BinaryClassificationOutput
Expand Down

358 changes: 358 additions & 0 deletions
ZBaselines/Common/EntryPoints/core_manifest.json

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -1723,6 +1723,105 @@
 "ITrainerOutput"
 ]
 },
 {
 "Name": "Models.PipelineSweeper",
 "Desc": "AutoML pipeline sweeping optimzation macro.",
 "FriendlyName": null,
 "ShortName": null,
 "Inputs": [
 {
 "Name": "TrainingData",
 "Type": "DataView",
 "Desc": "The data to be used for training.",
 "Aliases": [
 "train"
 ],
 "Required": true,
 "SortOrder": 1.0,
 "IsNullable": false
 },
 {
 "Name": "TestingData",
 "Type": "DataView",
 "Desc": "The data to be used for testing.",
 "Aliases": [
 "test"
 ],
 "Required": true,
 "SortOrder": 2.0,
 "IsNullable": false
 },
 {
 "Name": "StateArguments",
 "Type": {
 "Kind": "Component",
 "ComponentKind": "AutoMlStateBase"
 },
 "Desc": "The arguments for creating an AutoMlState component.",
 "Aliases": [
 "args"
 ],
 "Required": false,
 "SortOrder": 3.0,
 "IsNullable": false,
 "Default": null
 },
 {
 "Name": "State",
 "Type": {
 "Kind": "C# Object",
 "ItemType": "Microsoft.ML.Runtime.EntryPoints.IMlState"
 },
 "Desc": "The stateful object conducting of the autoML search.",
 "Aliases": [
 "state"
 ],
 "Required": false,
 "SortOrder": 3.0,
 "IsNullable": false,
 "Default": null
 },
 {
 "Name": "BatchSize",
 "Type": "Int",
 "Desc": "Number of candidate pipelines to retrieve each round.",
 "Aliases": [
 "bsize"
 ],
 "Required": true,
 "SortOrder": 4.0,
 "IsNullable": false,
 "Default": 0
 },
 {
 "Name": "CandidateOutputs",
 "Type": {
 "Kind": "Array",
 "ItemType": "DataView"
 },
 "Desc": "Output datasets from previous iteration of sweep.",
 "Required": false,
 "SortOrder": 7.0,
 "IsNullable": false,
 "Default": null
 }
 ],
 "Outputs": [
 {
 "Name": "State",
 "Type": {
 "Kind": "C# Object",
 "ItemType": "Microsoft.ML.Runtime.EntryPoints.IMlState"
 },
 "Desc": "Stateful autoML object, keeps track of where the search in progress."
 },
 {
 "Name": "Results",
 "Type": "DataView",
 "Desc": "Results of the sweep, including pipelines (as graph strings), IDs, and metric values."
 }
 ]
 },
 {
 "Name": "Models.PlattCalibrator",
 "Desc": "Apply a Platt calibrator to an input model",
Expand DownExpand Up@@ -2209,6 +2308,44 @@
 }
 ]
 },
 {
 "Name": "Models.SweepResultExtractor",
 "Desc": "Extracts the sweep result.",
 "FriendlyName": null,
 "ShortName": null,
 "Inputs": [
 {
 "Name": "State",
 "Type": {
 "Kind": "C# Object",
 "ItemType": "Microsoft.ML.Runtime.EntryPoints.IMlState"
 },
 "Desc": "The stateful object conducting of the autoML search.",
 "Aliases": [
 "state"
 ],
 "Required": false,
 "SortOrder": 1.0,
 "IsNullable": false,
 "Default": null
 }
 ],
 "Outputs": [
 {
 "Name": "State",
 "Type": {
 "Kind": "C# Object",
 "ItemType": "Microsoft.ML.Runtime.EntryPoints.IMlState"
 },
 "Desc": "Stateful autoML object, keeps track of where the search in progress."
 },
 {
 "Name": "Results",
 "Type": "DataView",
 "Desc": "Results of the sweep, including pipelines (as graph strings), IDs, and metric values."
 }
 ]
 },
 {
 "Name": "Models.TrainTestBinaryEvaluator",
 "Desc": "Train test for binary classification",
Expand DownExpand Up@@ -17200,6 +17337,203 @@
 }
 ],
 "Components": [
 {
 "Kind": "AutoMlEngine",
 "Components": [
 {
 "Name": "Defaults",
 "Desc": "AutoML engine that returns learners with default settings.",
 "FriendlyName": "Defaults Engine",
 "Settings": []
 },
 {
 "Name": "Rocket",
 "Desc": "AutoML engine that consists of distinct, hierarchical stages of operation.",
 "FriendlyName": "Rocket Engine",
 "Settings": [
 {
 "Name": "TopKLearners",
 "Type": "Int",
 "Desc": "Number of learners to retain for second stage.",
 "Aliases": [
 "topk"
 ],
 "Required": false,
 "SortOrder": 1.0,
 "IsNullable": false,
 "Default": 2
 },
 {
 "Name": "SecondRoundTrialsPerLearner",
 "Type": "Int",
 "Desc": "Number of trials for retained second stage learners.",
 "Aliases": [
 "stage2num"
 ],
 "Required": false,
 "SortOrder": 2.0,
 "IsNullable": false,
 "Default": 5
 },
 {
 "Name": "RandomInitialization",
 "Type": "Bool",
 "Desc": "Use random initialization only.",
 "Aliases": [
 "randinit"
 ],
 "Required": false,
 "SortOrder": 3.0,
 "IsNullable": false,
 "Default": false
 },
 {
 "Name": "NumInitializationPipelines",
 "Type": "Int",
 "Desc": "Number of initilization pipelines, used for random initialization only.",
 "Aliases": [
 "numinitseeds"
 ],
 "Required": false,
 "SortOrder": 4.0,
 "IsNullable": false,
 "Default": 20
 }
 ]
 },
 {
 "Name": "UniformRandom",
 "Desc": "AutoML engine using uniform random sampling.",
 "FriendlyName": "Uniform Random Engine",
 "Settings": []
 }
 ]
 },
 {
 "Kind": "AutoMlStateBase",
 "Components": [
 {
 "Name": "AutoMlState",
 "Desc": "State of an AutoML search and search space.",
 "FriendlyName": "AutoML State",
 "Aliases": [
 "automlst"
 ],
 "Settings": [
 {
 "Name": "Metric",
 "Type": {
 "Kind": "Enum",
 "Values": [
 "Auc",
 "AccuracyMicro",
 "AccuracyMacro",
 "L2",
 "F1",
 "AuPrc",
 "TopKAccuracy",
 "Rms",
 "LossFn",
 "RSquared",
 "LogLoss",
 "LogLossReduction",
 "Ndcg",
 "Dcg",
 "PositivePrecision",
 "PositiveRecall",
 "NegativePrecision",
 "NegativeRecall",
 "DrAtK",
 "DrAtPFpr",
 "DrAtNumPos",
 "NumAnomalies",
 "ThreshAtK",
 "ThreshAtP",
 "ThreshAtNumPos",
 "Nmi",
 "AvgMinScore",
 "Dbi"
 ]
 },
 "Desc": "Supported metric for evaluator.",
 "Aliases": [
 "metric"
 ],
 "Required": true,
 "SortOrder": 150.0,
 "IsNullable": false,
 "Default": "Auc"
 },
 {
 "Name": "Engine",
 "Type": {
 "Kind": "Component",
 "ComponentKind": "AutoMlEngine"
 },
 "Desc": "AutoML engine (pipeline optimizer) that generates next candidates.",
 "Aliases": [
 "engine"
 ],
 "Required": true,
 "SortOrder": 150.0,
 "IsNullable": false
 },
 {
 "Name": "TrainerKind",
 "Type": {
 "Kind": "Enum",
 "Values": [
 "SignatureBinaryClassifierTrainer",
 "SignatureMultiClassClassifierTrainer",
 "SignatureRankerTrainer",
 "SignatureRegressorTrainer",
 "SignatureMultiOutputRegressorTrainer",
 "SignatureAnomalyDetectorTrainer",
 "SignatureClusteringTrainer"
 ]
 },
 "Desc": "Kind of trainer for task, such as binary classification trainer, multiclass trainer, etc.",
 "Aliases": [
 "tk"
 ],
 "Required": true,
 "SortOrder": 150.0,
 "IsNullable": false,
 "Default": "SignatureBinaryClassifierTrainer"
 },
 {
 "Name": "TerminatorArgs",
 "Type": {
 "Kind": "Component",
 "ComponentKind": "SearchTerminator"
 },
 "Desc": "Arguments for creating terminator, which determines when to stop search.",
 "Aliases": [
 "term"
 ],
 "Required": true,
 "SortOrder": 150.0,
 "IsNullable": false
 },
 {
 "Name": "RequestedLearners",
 "Type": {
 "Kind": "Array",
 "ItemType": "String"
 },
 "Desc": "Learner set to sweep over (if available).",
 "Aliases": [
 "learners"
 ],
 "Required": false,
 "SortOrder": 150.0,
 "IsNullable": false,
 "Default": null
 }
 ]
 }
 ]
 },
 {
 "Kind": "CalibratorTrainer",
 "Components": [
Expand DownExpand Up@@ -21441,6 +21775,30 @@
 }
 ]
 },
 {
 "Kind": "SearchTerminator",
 "Components": [
 {
 "Name": "IterationLimited",
 "Desc": "Terminators a sweep based on total number of iterations.",
 "FriendlyName": "Pipeline Sweep Iteration Terminator",
 "Settings": [
 {
 "Name": "FinalHistoryLength",
 "Type": "Int",
 "Desc": "Total number of iterations.",
 "Aliases": [
 "length"
 ],
 "Required": true,
 "SortOrder": 150.0,
 "IsNullable": false,
 "Default": 0
 }
 ]
 }
 ]
 },
 {
 "Kind": "StopWordsRemover",
 "Components": [
Expand Down
Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode");
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();

Skip to content
Update MacroUtils to map trainer kinds to the correct suffix of trainer entry point names#113

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

yaeldekel
merged 8 commits into

from

May 14, 2018
 Merged


Update MacroUtils to map trainer kinds to the correct suffix of trainer entry point names
#113
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


8 commits


Select commit
Hold shift + click to select a range
02f72c9

Update suffix of trainer entry point names by trainer kind.


yaeldMS May 10, 2018
ff6736d

Address PR comments.


yaeldMS May 10, 2018
1d01831

Add unit test.


yaeldMS May 10, 2018
f1bbeda

Merge remote-tracking branch 'upstream/master' into macroutils


yaeldMS May 11, 2018
db8492d

Update C# API


yaeldMS May 11, 2018
4267412

Move unit test to TestAutoInference and fix EntryPointCatalog test.


yaeldMS May 11, 2018
9bb6c89

Trigger build.


yaeldMS May 11, 2018
be06047

Add reference to the test project to make the sweeper entry point vis…


yaeldMS May 11, 2018

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


2 changes: 2 additions & 0 deletions
ZBaselines/Common/EntryPoints/core_ep-list.tsv

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -15,11 +15,13 @@ Models.NaiveCalibrator	Apply a Naive calibrator to an input model	Microsoft.ML.R
Models.OneVersusAll	One-vs-All macro (OVA)	Microsoft.ML.Runtime.EntryPoints.OneVersusAllMacro	OVA	Microsoft.ML.Runtime.EntryPoints.OneVersusAllMacro+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+MacroOutput`1[Microsoft.ML.Runtime.EntryPoints.OneVersusAllMacro+Output]
Models.OvaModelCombiner	Combines a sequence of PredictorModels into a single model	Microsoft.ML.Runtime.Learners.OvaPredictor	CombineOvaModels	Microsoft.ML.Runtime.EntryPoints.ModelOperations+CombineOvaPredictorModelsInput	Microsoft.ML.Runtime.EntryPoints.ModelOperations+PredictorModelOutput
Models.PAVCalibrator	Apply a PAV calibrator to an input model	Microsoft.ML.Runtime.Internal.Calibration.Calibrate	Pav	Microsoft.ML.Runtime.Internal.Calibration.Calibrate+NoArgumentsInput	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+CalibratorOutput
Models.PipelineSweeper	AutoML pipeline sweeping optimzation macro.	Microsoft.ML.Runtime.EntryPoints.PipelineSweeperMacro	PipelineSweep	Microsoft.ML.Runtime.EntryPoints.PipelineSweeperMacro+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+MacroOutput`1[Microsoft.ML.Runtime.EntryPoints.PipelineSweeperMacro+Output]
Models.PlattCalibrator	Apply a Platt calibrator to an input model	Microsoft.ML.Runtime.Internal.Calibration.Calibrate	Platt	Microsoft.ML.Runtime.Internal.Calibration.Calibrate+NoArgumentsInput	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+CalibratorOutput
Models.QuantileRegressionEvaluator	Evaluates a quantile regression scored dataset.	Microsoft.ML.Runtime.Data.Evaluate	QuantileRegression	Microsoft.ML.Runtime.Data.QuantileRegressionMamlEvaluator+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+CommonEvaluateOutput
Models.RankerEvaluator	Evaluates a ranking scored dataset.	Microsoft.ML.Runtime.Data.Evaluate	Ranking	Microsoft.ML.Runtime.Data.RankerMamlEvaluator+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+CommonEvaluateOutput
Models.RegressionEvaluator	Evaluates a regression scored dataset.	Microsoft.ML.Runtime.Data.Evaluate	Regression	Microsoft.ML.Runtime.Data.RegressionMamlEvaluator+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+CommonEvaluateOutput
Models.Summarizer	Summarize a linear regression predictor.	Microsoft.ML.Runtime.EntryPoints.SummarizePredictor	Summarize	Microsoft.ML.Runtime.EntryPoints.SummarizePredictor+Input	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+SummaryOutput
Models.SweepResultExtractor	Extracts the sweep result.	Microsoft.ML.Runtime.EntryPoints.PipelineSweeperMacro	ExtractSweepResult	Microsoft.ML.Runtime.EntryPoints.PipelineSweeperMacro+ResultInput	Microsoft.ML.Runtime.EntryPoints.PipelineSweeperMacro+Output
Models.TrainTestBinaryEvaluator	Train test for binary classification	Microsoft.ML.Runtime.EntryPoints.TrainTestBinaryMacro	TrainTestBinary	Microsoft.ML.Runtime.EntryPoints.TrainTestBinaryMacro+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+MacroOutput`1[Microsoft.ML.Runtime.EntryPoints.TrainTestBinaryMacro+Output]
Models.TrainTestEvaluator	General train test for any supported evaluator	Microsoft.ML.Runtime.EntryPoints.TrainTestMacro	TrainTest	Microsoft.ML.Runtime.EntryPoints.TrainTestMacro+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+MacroOutput`1[Microsoft.ML.Runtime.EntryPoints.TrainTestMacro+Output]
Trainers.AveragedPerceptronBinaryClassifier	Train a Average perceptron.	Microsoft.ML.Runtime.Learners.AveragedPerceptronTrainer	TrainBinary	Microsoft.ML.Runtime.Learners.AveragedPerceptronTrainer+Arguments	Microsoft.ML.Runtime.EntryPoints.CommonOutputs+BinaryClassificationOutput
Expand Down

358 changes: 358 additions & 0 deletions
ZBaselines/Common/EntryPoints/core_manifest.json

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -1723,6 +1723,105 @@
 "ITrainerOutput"
 ]
 },
 {
 "Name": "Models.PipelineSweeper",
 "Desc": "AutoML pipeline sweeping optimzation macro.",
 "FriendlyName": null,
 "ShortName": null,
 "Inputs": [
 {
 "Name": "TrainingData",
 "Type": "DataView",
 "Desc": "The data to be used for training.",
 "Aliases": [
 "train"
 ],
 "Required": true,
 "SortOrder": 1.0,
 "IsNullable": false
 },
 {
 "Name": "TestingData",
 "Type": "DataView",
 "Desc": "The data to be used for testing.",
 "Aliases": [
 "test"
 ],
 "Required": true,
 "SortOrder": 2.0,
 "IsNullable": false
 },
 {
 "Name": "StateArguments",
 "Type": {
 "Kind": "Component",
 "ComponentKind": "AutoMlStateBase"
 },
 "Desc": "The arguments for creating an AutoMlState component.",
 "Aliases": [
 "args"
 ],
 "Required": false,
 "SortOrder": 3.0,
 "IsNullable": false,
 "Default": null
 },
 {
 "Name": "State",
 "Type": {
 "Kind": "C# Object",
 "ItemType": "Microsoft.ML.Runtime.EntryPoints.IMlState"
 },
 "Desc": "The stateful object conducting of the autoML search.",
 "Aliases": [
 "state"
 ],
 "Required": false,
 "SortOrder": 3.0,
 "IsNullable": false,
 "Default": null
 },
 {
 "Name": "BatchSize",
 "Type": "Int",
 "Desc": "Number of candidate pipelines to retrieve each round.",
 "Aliases": [
 "bsize"
 ],
 "Required": true,
 "SortOrder": 4.0,
 "IsNullable": false,
 "Default": 0
 },
 {
 "Name": "CandidateOutputs",
 "Type": {
 "Kind": "Array",
 "ItemType": "DataView"
 },
 "Desc": "Output datasets from previous iteration of sweep.",
 "Required": false,
 "SortOrder": 7.0,
 "IsNullable": false,
 "Default": null
 }
 ],
 "Outputs": [
 {
 "Name": "State",
 "Type": {
 "Kind": "C# Object",
 "ItemType": "Microsoft.ML.Runtime.EntryPoints.IMlState"
 },
 "Desc": "Stateful autoML object, keeps track of where the search in progress."
 },
 {
 "Name": "Results",
 "Type": "DataView",
 "Desc": "Results of the sweep, including pipelines (as graph strings), IDs, and metric values."
 }
 ]
 },
 {
 "Name": "Models.PlattCalibrator",
 "Desc": "Apply a Platt calibrator to an input model",
Expand DownExpand Up@@ -2209,6 +2308,44 @@
 }
 ]
 },
 {
 "Name": "Models.SweepResultExtractor",
 "Desc": "Extracts the sweep result.",
 "FriendlyName": null,
 "ShortName": null,
 "Inputs": [
 {
 "Name": "State",
 "Type": {
 "Kind": "C# Object",
 "ItemType": "Microsoft.ML.Runtime.EntryPoints.IMlState"
 },
 "Desc": "The stateful object conducting of the autoML search.",
 "Aliases": [
 "state"
 ],
 "Required": false,
 "SortOrder": 1.0,
 "IsNullable": false,
 "Default": null
 }
 ],
 "Outputs": [
 {
 "Name": "State",
 "Type": {
 "Kind": "C# Object",
 "ItemType": "Microsoft.ML.Runtime.EntryPoints.IMlState"
 },
 "Desc": "Stateful autoML object, keeps track of where the search in progress."
 },
 {
 "Name": "Results",
 "Type": "DataView",
 "Desc": "Results of the sweep, including pipelines (as graph strings), IDs, and metric values."
 }
 ]
 },
 {
 "Name": "Models.TrainTestBinaryEvaluator",
 "Desc": "Train test for binary classification",
Expand DownExpand Up@@ -17200,6 +17337,203 @@
 }
 ],
 "Components": [
 {
 "Kind": "AutoMlEngine",
 "Components": [
 {
 "Name": "Defaults",
 "Desc": "AutoML engine that returns learners with default settings.",
 "FriendlyName": "Defaults Engine",
 "Settings": []
 },
 {
 "Name": "Rocket",
 "Desc": "AutoML engine that consists of distinct, hierarchical stages of operation.",
 "FriendlyName": "Rocket Engine",
 "Settings": [
 {
 "Name": "TopKLearners",
 "Type": "Int",
 "Desc": "Number of learners to retain for second stage.",
 "Aliases": [
 "topk"
 ],
 "Required": false,
 "SortOrder": 1.0,
 "IsNullable": false,
 "Default": 2
 },
 {
 "Name": "SecondRoundTrialsPerLearner",
 "Type": "Int",
 "Desc": "Number of trials for retained second stage learners.",
 "Aliases": [
 "stage2num"
 ],
 "Required": false,
 "SortOrder": 2.0,
 "IsNullable": false,
 "Default": 5
 },
 {
 "Name": "RandomInitialization",
 "Type": "Bool",
 "Desc": "Use random initialization only.",
 "Aliases": [
 "randinit"
 ],
 "Required": false,
 "SortOrder": 3.0,
 "IsNullable": false,
 "Default": false
 },
 {
 "Name": "NumInitializationPipelines",
 "Type": "Int",
 "Desc": "Number of initilization pipelines, used for random initialization only.",
 "Aliases": [
 "numinitseeds"
 ],
 "Required": false,
 "SortOrder": 4.0,
 "IsNullable": false,
 "Default": 20
 }
 ]
 },
 {
 "Name": "UniformRandom",
 "Desc": "AutoML engine using uniform random sampling.",
 "FriendlyName": "Uniform Random Engine",
 "Settings": []
 }
 ]
 },
 {
 "Kind": "AutoMlStateBase",
 "Components": [
 {
 "Name": "AutoMlState",
 "Desc": "State of an AutoML search and search space.",
 "FriendlyName": "AutoML State",
 "Aliases": [
 "automlst"
 ],
 "Settings": [
 {
 "Name": "Metric",
 "Type": {
 "Kind": "Enum",
 "Values": [
 "Auc",
 "AccuracyMicro",
 "AccuracyMacro",
 "L2",
 "F1",
 "AuPrc",
 "TopKAccuracy",
 "Rms",
 "LossFn",
 "RSquared",
 "LogLoss",
 "LogLossReduction",
 "Ndcg",
 "Dcg",
 "PositivePrecision",
 "PositiveRecall",
 "NegativePrecision",
 "NegativeRecall",
 "DrAtK",
 "DrAtPFpr",
 "DrAtNumPos",
 "NumAnomalies",
 "ThreshAtK",
 "ThreshAtP",
 "ThreshAtNumPos",
 "Nmi",
 "AvgMinScore",
 "Dbi"
 ]
 },
 "Desc": "Supported metric for evaluator.",
 "Aliases": [
 "metric"
 ],
 "Required": true,
 "SortOrder": 150.0,
 "IsNullable": false,
 "Default": "Auc"
 },
 {
 "Name": "Engine",
 "Type": {
 "Kind": "Component",
 "ComponentKind": "AutoMlEngine"
 },
 "Desc": "AutoML engine (pipeline optimizer) that generates next candidates.",
 "Aliases": [
 "engine"
 ],
 "Required": true,
 "SortOrder": 150.0,
 "IsNullable": false
 },
 {
 "Name": "TrainerKind",
 "Type": {
 "Kind": "Enum",
 "Values": [
 "SignatureBinaryClassifierTrainer",
 "SignatureMultiClassClassifierTrainer",
 "SignatureRankerTrainer",
 "SignatureRegressorTrainer",
 "SignatureMultiOutputRegressorTrainer",
 "SignatureAnomalyDetectorTrainer",
 "SignatureClusteringTrainer"
 ]
 },
 "Desc": "Kind of trainer for task, such as binary classification trainer, multiclass trainer, etc.",
 "Aliases": [
 "tk"
 ],
 "Required": true,
 "SortOrder": 150.0,
 "IsNullable": false,
 "Default": "SignatureBinaryClassifierTrainer"
 },
 {
 "Name": "TerminatorArgs",
 "Type": {
 "Kind": "Component",
 "ComponentKind": "SearchTerminator"
 },
 "Desc": "Arguments for creating terminator, which determines when to stop search.",
 "Aliases": [
 "term"
 ],
 "Required": true,
 "SortOrder": 150.0,
 "IsNullable": false
 },
 {
 "Name": "RequestedLearners",
 "Type": {
 "Kind": "Array",
 "ItemType": "String"
 },
 "Desc": "Learner set to sweep over (if available).",
 "Aliases": [
 "learners"
 ],
 "Required": false,
 "SortOrder": 150.0,
 "IsNullable": false,
 "Default": null
 }
 ]
 }
 ]
 },
 {
 "Kind": "CalibratorTrainer",
 "Components": [
Expand DownExpand Up@@ -21441,6 +21775,30 @@
 }
 ]
 },
 {
 "Kind": "SearchTerminator",
 "Components": [
 {
 "Name": "IterationLimited",
 "Desc": "Terminators a sweep based on total number of iterations.",
 "FriendlyName": "Pipeline Sweep Iteration Terminator",
 "Settings": [
 {
 "Name": "FinalHistoryLength",
 "Type": "Int",
 "Desc": "Total number of iterations.",
 "Aliases": [
 "length"
 ],
 "Required": true,
 "SortOrder": 150.0,
 "IsNullable": false,
 "Default": 0
 }
 ]
 }
 ]
 },
 {
 "Kind": "StopWordsRemover",
 "Components": [
Expand Down
Loading