WIP: Make sure Done() is called in cleanup if it hasn't been called#4817

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
harishsk wants to merge 3 commits into from
Closed

WIP: Make sure Done() is called in cleanup if it hasn't been called #4817

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
cf46e5c
Make sure Done() is called in cleanup if it hasn't been called
harishsk Feb 8, 2020
836a44e
Fixed EntryPointCalibrate and TestEnsembleCombiner
harishsk Feb 8, 2020
6993980
Updated baselines to account for the change in default parameters for…
harishsk Feb 9, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion src/Microsoft.ML.StandardTrainers/Standard/LogisticRegression/LogisticRegression.cs
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -455,7 +455,7 @@ private protected override CalibratedModelParametersBase<LinearBinaryModelParame
CurrentWeights.CopyTo(ref weights, 1, CurrentWeights.Length - 1);
return new ParameterMixingCalibratedModelParameters<LinearBinaryModelParameters, PlattCalibrator>(Host,
new LinearBinaryModelParameters(Host, in weights, bias, _stats),
new PlattCalibrator(Host, -1, 0));
new PlattCalibrator(Host, 1, 0));

@yaeldekelyaeldekelFeb 10, 2020

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems that the bug is actually in the PlattCalibrator itself:

While in the comment it says this:

 /// The Platt calibrator calculates the probability following:
/// P(x) = 1 / (1 + exp(-<see cref="PlattCalibrator.Slope"/> * x + <see cref="PlattCalibrator.Offset"/>)

The actual calculation does this:

 public float PredictProbability(float output)
{
if (float.IsNaN(output))
return output;
return PredictProbability(output, Slope, Offset);
}
internal static float PredictProbability(float output, Double a, Double b)
{
return (float)(1 / (1 + Math.Exp(a * output + b)));
}

The value here should be changed to 1 only if we change the computation in PlattCalibrator. (looking at the baselines, the computation without the change looks better: label 1 examples give higher probabilities, and label 0 examples give lower probabilities.)

}

[TlcModule.EntryPoint(Name = "Trainers.LogisticRegressionBinaryClassifier",
Expand Down
12 changes: 6 additions & 6 deletions ...BaselineOutput/SingleDebug/LogisticRegression/LogisticRegression-CV-breast-cancer-out.txt
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -24,9 +24,9 @@ TRUTH ||======================
||======================
Precision || 0.9489 | 0.9816 |
OVERALL 0/1 ACCURACY: 0.968927
LOG LOSS/instance: 0.143504
LOG LOSS/instance: 8.202349
Test-set entropy (prior Log-Loss/instance): 0.956998
LOG-LOSS REDUCTION (RIG): 0.850048
LOG-LOSS REDUCTION (RIG): -7.570916
AUC: 0.994132
Warning: The predictor produced non-finite prediction values on 8 instances during testing. Possible causes: abnormal data or the predictor is numerically unstable.
TEST POSITIVE RATIO: 0.3191 (105.0/(105.0+224.0))
Expand All@@ -39,9 +39,9 @@ TRUTH ||======================
||======================
Precision || 0.9697 | 0.9609 |
OVERALL 0/1 ACCURACY: 0.963526
LOG LOSS/instance: 0.111794
LOG LOSS/instance: 6.914975
Test-set entropy (prior Log-Loss/instance): 0.903454
LOG-LOSS REDUCTION (RIG): 0.876260
LOG-LOSS REDUCTION (RIG): -6.653935
AUC: 0.997236

OVERALL RESULTS
Expand All@@ -52,8 +52,8 @@ Positive precision: 0.959301 (0.0104)
Positive recall: 0.942217 (0.0279)
Negative precision: 0.971218 (0.0103)
Negative recall: 0.977394 (0.0092)
Log-loss: 0.127649 (0.0159)
Log-loss reduction: 0.863154 (0.0131)
Log-loss: 7.558662 (0.6437)
Log-loss reduction: -7.112425 (0.4585)
F1 Score: 0.950293 (0.0091)
AUPRC: 0.991584 (0.0025)

Expand Down
2 changes: 1 addition & 1 deletion .../BaselineOutput/SingleDebug/LogisticRegression/LogisticRegression-CV-breast-cancer-rp.txt
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
LogisticRegression
AUC Accuracy Positive precision Positive recall Negative precision Negative recall Log-loss Log-loss reduction F1 Score AUPRC /l2 /ot /nt Learner Name Train Dataset Test Dataset Results File Run Time Physical Memory Virtual Memory Command Line Settings
0.995684 0.966226 0.959301 0.942217 0.971218 0.977394 0.127649 0.863154 0.950293 0.991584 0.1 0.001 1 LogisticRegression %Data% %Output% 99 0 0 maml.exe CV tr=LogisticRegression{l1=1.0 l2=0.1 ot=1e-3 nt=1} threads=- norm=No dout=%Output% data=%Data% seed=1 /l2:0.1;/ot:0.001;/nt:1
0.995684 0.966226 0.959301 0.942217 0.971218 0.977394 7.558662 -7.112425 0.950293 0.991584 0.1 0.001 1 LogisticRegression %Data% %Output% 99 0 0 maml.exe CV tr=LogisticRegression{l1=1.0 l2=0.1 ot=1e-3 nt=1} threads=- norm=No dout=%Output% data=%Data% seed=1 /l2:0.1;/ot:0.001;/nt:1

1,366 changes: 683 additions & 683 deletions test/BaselineOutput/SingleDebug/LogisticRegression/LogisticRegression-CV-breast-cancer.txt
View file
Open in desktop

Large diffs are not rendered by default.

40 changes: 20 additions & 20 deletions .../SingleDebug/LogisticRegression/LogisticRegression-CV-breast-cancer.withThreshold-out.txt
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -19,42 +19,42 @@ Confusion table
||======================
PREDICTED || positive | negative | Recall
TRUTH ||======================
positive || 118 | 16 | 0.8806
negative || 3 | 217 | 0.9864
positive || 0 | 134 | 0.0000
negative || 202 | 18 | 0.0818
||======================
Precision || 0.9752 | 0.9313 |
OVERALL 0/1 ACCURACY: 0.946328
LOG LOSS/instance: 0.143504
Precision || 0.0000 | 0.1184 |
OVERALL 0/1 ACCURACY: 0.050847
LOG LOSS/instance: 8.202349
Test-set entropy (prior Log-Loss/instance): 0.956998
LOG-LOSS REDUCTION (RIG): 0.850048
LOG-LOSS REDUCTION (RIG): -7.570916
AUC: 0.994132
Warning: The predictor produced non-finite prediction values on 8 instances during testing. Possible causes: abnormal data or the predictor is numerically unstable.
TEST POSITIVE RATIO: 0.3191 (105.0/(105.0+224.0))
Confusion table
||======================
PREDICTED || positive | negative | Recall
TRUTH ||======================
positive || 81 | 24 | 0.7714
negative || 0 | 224 | 1.0000
positive || 0 | 105 | 0.0000
negative || 210 | 14 | 0.0625
||======================
Precision || 1.0000 | 0.9032 |
OVERALL 0/1 ACCURACY: 0.927052
LOG LOSS/instance: 0.111794
Precision || 0.0000 | 0.1176 |
OVERALL 0/1 ACCURACY: 0.042553
LOG LOSS/instance: 6.914975
Test-set entropy (prior Log-Loss/instance): 0.903454
LOG-LOSS REDUCTION (RIG): 0.876260
LOG-LOSS REDUCTION (RIG): -6.653935
AUC: 0.997236

OVERALL RESULTS
---------------------------------------
AUC: 0.995684 (0.0016)
Accuracy: 0.936690 (0.0096)
Positive precision: 0.987603 (0.0124)
Positive recall: 0.826013 (0.0546)
Negative precision: 0.917278 (0.0141)
Negative recall: 0.993182 (0.0068)
Log-loss: 0.127649 (0.0159)
Log-loss reduction: 0.863154 (0.0131)
F1 Score: 0.898229 (0.0273)
Accuracy: 0.046700 (0.0041)
Positive precision: 0.000000 (0.0000)
Positive recall: 0.000000 (0.0000)
Negative precision: 0.118034 (0.0004)
Negative recall: 0.072159 (0.0097)
Log-loss: 7.558662 (0.6437)
Log-loss reduction: -7.112425 (0.4585)
F1 Score: 0.000000 (0.0000)
AUPRC: 0.991584 (0.0025)

---------------------------------------
Expand Down
2 changes: 1 addition & 1 deletion ...t/SingleDebug/LogisticRegression/LogisticRegression-CV-breast-cancer.withThreshold-rp.txt
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
LogisticRegression
AUC Accuracy Positive precision Positive recall Negative precision Negative recall Log-loss Log-loss reduction F1 Score AUPRC /l2 /ot /nt Learner Name Train Dataset Test Dataset Results File Run Time Physical Memory Virtual Memory Command Line Settings
0.995684 0.93669 0.987603 0.826013 0.917278 0.993182 0.127649 0.863154 0.898229 0.991584 0.1 0.001 1 LogisticRegression %Data% %Output% 99 0 0 maml.exe CV tr=LogisticRegression{l1=1.0 l2=0.1 ot=1e-3 nt=1} eval=BinaryClassifier{threshold=0.95 useRawScore=-} threads=- norm=No dout=%Output% data=%Data% seed=1 /l2:0.1;/ot:0.001;/nt:1
0.995684 0.0467 0 0 0.118034 0.072159 7.558662 -7.112425 0 0.991584 0.1 0.001 1 LogisticRegression %Data% %Output% 99 0 0 maml.exe CV tr=LogisticRegression{l1=1.0 l2=0.1 ot=1e-3 nt=1} eval=BinaryClassifier{threshold=0.95 useRawScore=-} threads=- norm=No dout=%Output% data=%Data% seed=1 /l2:0.1;/ot:0.001;/nt:1

Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all 
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '

Skip to content
WIP: Make sure Done() is called in cleanup if it hasn't been called#4817

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Closed

harishsk
wants to merge
3
commits into

from

 Closed


WIP: Make sure Done() is called in cleanup if it hasn't been called
#4817
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


3 commits


Select commit
Hold shift + click to select a range
cf46e5c

Make sure Done() is called in cleanup if it hasn't been called


harishsk Feb 8, 2020
836a44e

Fixed EntryPointCalibrate and TestEnsembleCombiner


harishsk Feb 8, 2020
6993980

Updated baselines to account for the change in default parameters for…


harishsk Feb 9, 2020

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


2 changes: 1 addition & 1 deletion
src/Microsoft.ML.StandardTrainers/Standard/LogisticRegression/LogisticRegression.cs

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -455,7 +455,7 @@ private protected override CalibratedModelParametersBase<LinearBinaryModelParame
 CurrentWeights.CopyTo(ref weights, 1, CurrentWeights.Length - 1);
 return new ParameterMixingCalibratedModelParameters<LinearBinaryModelParameters, PlattCalibrator>(Host,
 new LinearBinaryModelParameters(Host, in weights, bias, _stats),
 new PlattCalibrator(Host, -1, 0));
 new PlattCalibrator(Host, 1, 0));
@yaeldekelyaeldekelFeb 10, 2020

Copy link


Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems that the bug is actually in the PlattCalibrator itself:
While in the comment it says this:
 /// The Platt calibrator calculates the probability following:
/// P(x) = 1 / (1 + exp(-<see cref="PlattCalibrator.Slope"/> * x + <see cref="PlattCalibrator.Offset"/>)

The actual calculation does this:
 public float PredictProbability(float output)
{
if (float.IsNaN(output))
return output;
return PredictProbability(output, Slope, Offset);
}
internal static float PredictProbability(float output, Double a, Double b)
{
return (float)(1 / (1 + Math.Exp(a * output + b)));
}

The value here should be changed to 1 only if we change the computation in PlattCalibrator. (looking at the baselines, the computation without the change looks better: label 1 examples give higher probabilities, and label 0 examples give lower probabilities.)
 }

 [TlcModule.EntryPoint(Name = "Trainers.LogisticRegressionBinaryClassifier",
Expand Down

12 changes: 6 additions & 6 deletions
...BaselineOutput/SingleDebug/LogisticRegression/LogisticRegression-CV-breast-cancer-out.txt

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -24,9 +24,9 @@ TRUTH ||======================
 ||======================
Precision || 0.9489 | 0.9816 |
OVERALL 0/1 ACCURACY: 0.968927
LOG LOSS/instance: 0.143504
LOG LOSS/instance: 8.202349
Test-set entropy (prior Log-Loss/instance): 0.956998
LOG-LOSS REDUCTION (RIG): 0.850048
LOG-LOSS REDUCTION (RIG): -7.570916
AUC: 0.994132
Warning: The predictor produced non-finite prediction values on 8 instances during testing. Possible causes: abnormal data or the predictor is numerically unstable.
TEST POSITIVE RATIO:	0.3191 (105.0/(105.0+224.0))
Expand All@@ -39,9 +39,9 @@ TRUTH ||======================
 ||======================
Precision || 0.9697 | 0.9609 |
OVERALL 0/1 ACCURACY: 0.963526
LOG LOSS/instance: 0.111794
LOG LOSS/instance: 6.914975
Test-set entropy (prior Log-Loss/instance): 0.903454
LOG-LOSS REDUCTION (RIG): 0.876260
LOG-LOSS REDUCTION (RIG): -6.653935
AUC: 0.997236

OVERALL RESULTS
Expand All@@ -52,8 +52,8 @@ Positive precision: 0.959301 (0.0104)
Positive recall: 0.942217 (0.0279)
Negative precision: 0.971218 (0.0103)
Negative recall: 0.977394 (0.0092)
Log-loss: 0.127649 (0.0159)
Log-loss reduction: 0.863154 (0.0131)
Log-loss: 7.558662 (0.6437)
Log-loss reduction: -7.112425 (0.4585)
F1 Score: 0.950293 (0.0091)
AUPRC: 0.991584 (0.0025)

Expand Down

2 changes: 1 addition & 1 deletion
.../BaselineOutput/SingleDebug/LogisticRegression/LogisticRegression-CV-breast-cancer-rp.txt

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
LogisticRegression
AUC	Accuracy	Positive precision	Positive recall	Negative precision	Negative recall	Log-loss	Log-loss reduction	F1 Score	AUPRC	/l2	/ot	/nt	Learner Name	Train Dataset	Test Dataset	Results File	Run Time	Physical Memory	Virtual Memory	Command Line	Settings	
0.995684	0.966226	0.959301	0.942217	0.971218	0.977394	0.127649	0.863154	0.950293	0.991584	0.1	0.001	1	LogisticRegression	%Data% %Output%	99	0	0	maml.exe CV tr=LogisticRegression{l1=1.0 l2=0.1 ot=1e-3 nt=1} threads=- norm=No dout=%Output% data=%Data% seed=1	/l2:0.1;/ot:0.001;/nt:1	
0.995684	0.966226	0.959301	0.942217	0.971218	0.977394	7.558662	-7.112425	0.950293	0.991584	0.1	0.001	1	LogisticRegression	%Data% %Output%	99	0	0	maml.exe CV tr=LogisticRegression{l1=1.0 l2=0.1 ot=1e-3 nt=1} threads=- norm=No dout=%Output% data=%Data% seed=1	/l2:0.1;/ot:0.001;/nt:1	


1,366 changes: 683 additions & 683 deletions
test/BaselineOutput/SingleDebug/LogisticRegression/LogisticRegression-CV-breast-cancer.txt

View file


Open in desktop


Large diffs are not rendered by default.


40 changes: 20 additions & 20 deletions
.../SingleDebug/LogisticRegression/LogisticRegression-CV-breast-cancer.withThreshold-out.txt

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -19,42 +19,42 @@ Confusion table
 ||======================
PREDICTED || positive | negative | Recall
TRUTH ||======================
 positive || 118 | 16 | 0.8806
 negative ||  3 | 217 | 0.9864
 positive ||  0 | 134 | 0.0000
 negative || 202 |  18 | 0.0818
 ||======================
Precision || 0.9752 | 0.9313 |
OVERALL 0/1 ACCURACY: 0.946328
LOG LOSS/instance: 0.143504
Precision || 0.0000 | 0.1184 |
OVERALL 0/1 ACCURACY: 0.050847
LOG LOSS/instance: 8.202349
Test-set entropy (prior Log-Loss/instance): 0.956998
LOG-LOSS REDUCTION (RIG): 0.850048
LOG-LOSS REDUCTION (RIG): -7.570916
AUC: 0.994132
Warning: The predictor produced non-finite prediction values on 8 instances during testing. Possible causes: abnormal data or the predictor is numerically unstable.
TEST POSITIVE RATIO:	0.3191 (105.0/(105.0+224.0))
Confusion table
 ||======================
PREDICTED || positive | negative | Recall
TRUTH ||======================
 positive || 81 |  24 | 0.7714
 negative ||  0 | 224 | 1.0000
 positive ||  0 | 105 | 0.0000
 negative || 210 |  14 | 0.0625
 ||======================
Precision || 1.0000 | 0.9032 |
OVERALL 0/1 ACCURACY: 0.927052
LOG LOSS/instance: 0.111794
Precision || 0.0000 | 0.1176 |
OVERALL 0/1 ACCURACY: 0.042553
LOG LOSS/instance: 6.914975
Test-set entropy (prior Log-Loss/instance): 0.903454
LOG-LOSS REDUCTION (RIG): 0.876260
LOG-LOSS REDUCTION (RIG): -6.653935
AUC: 0.997236

OVERALL RESULTS
---------------------------------------
AUC: 0.995684 (0.0016)
Accuracy: 0.936690 (0.0096)
Positive precision: 0.987603 (0.0124)
Positive recall: 0.826013 (0.0546)
Negative precision: 0.917278 (0.0141)
Negative recall: 0.993182 (0.0068)
Log-loss: 0.127649 (0.0159)
Log-loss reduction: 0.863154 (0.0131)
F1 Score: 0.898229 (0.0273)
Accuracy: 0.046700 (0.0041)
Positive precision: 0.000000 (0.0000)
Positive recall: 0.000000 (0.0000)
Negative precision: 0.118034 (0.0004)
Negative recall: 0.072159 (0.0097)
Log-loss: 7.558662 (0.6437)
Log-loss reduction: -7.112425 (0.4585)
F1 Score: 0.000000 (0.0000)
AUPRC: 0.991584 (0.0025)

---------------------------------------
Expand Down

2 changes: 1 addition & 1 deletion
...t/SingleDebug/LogisticRegression/LogisticRegression-CV-breast-cancer.withThreshold-rp.txt

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
LogisticRegression
AUC	Accuracy	Positive precision	Positive recall	Negative precision	Negative recall	Log-loss	Log-loss reduction	F1 Score	AUPRC	/l2	/ot	/nt	Learner Name	Train Dataset	Test Dataset	Results File	Run Time	Physical Memory	Virtual Memory	Command Line	Settings	
0.995684	0.93669	0.987603	0.826013	0.917278	0.993182	0.127649	0.863154	0.898229	0.991584	0.1	0.001	1	LogisticRegression	%Data% %Output%	99	0	0	maml.exe CV tr=LogisticRegression{l1=1.0 l2=0.1 ot=1e-3 nt=1} eval=BinaryClassifier{threshold=0.95 useRawScore=-} threads=- norm=No dout=%Output% data=%Data% seed=1	/l2:0.1;/ot:0.001;/nt:1	
0.995684	0.0467	0	0	0.118034	0.072159	7.558662	-7.112425	0	0.991584	0.1	0.001	1	LogisticRegression	%Data% %Output%	99	0	0	maml.exe CV tr=LogisticRegression{l1=1.0 l2=0.1 ot=1e-3 nt=1} eval=BinaryClassifier{threshold=0.95 useRawScore=-} threads=- norm=No dout=%Output% data=%Data% seed=1	/l2:0.1;/ot:0.001;/nt:1	

Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix");
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
WIP: Make sure Done() is called in cleanup if it hasn't been called#4817

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Closed

harishsk
wants to merge
3
commits into

from

 Closed


WIP: Make sure Done() is called in cleanup if it hasn't been called
#4817
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


3 commits


Select commit
Hold shift + click to select a range
cf46e5c

Make sure Done() is called in cleanup if it hasn't been called


harishsk Feb 8, 2020
836a44e

Fixed EntryPointCalibrate and TestEnsembleCombiner


harishsk Feb 8, 2020
6993980

Updated baselines to account for the change in default parameters for…


harishsk Feb 9, 2020

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


2 changes: 1 addition & 1 deletion
src/Microsoft.ML.StandardTrainers/Standard/LogisticRegression/LogisticRegression.cs

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -455,7 +455,7 @@ private protected override CalibratedModelParametersBase<LinearBinaryModelParame
 CurrentWeights.CopyTo(ref weights, 1, CurrentWeights.Length - 1);
 return new ParameterMixingCalibratedModelParameters<LinearBinaryModelParameters, PlattCalibrator>(Host,
 new LinearBinaryModelParameters(Host, in weights, bias, _stats),
 new PlattCalibrator(Host, -1, 0));
 new PlattCalibrator(Host, 1, 0));
@yaeldekelyaeldekelFeb 10, 2020

Copy link


Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems that the bug is actually in the PlattCalibrator itself:
While in the comment it says this:
 /// The Platt calibrator calculates the probability following:
/// P(x) = 1 / (1 + exp(-<see cref="PlattCalibrator.Slope"/> * x + <see cref="PlattCalibrator.Offset"/>)

The actual calculation does this:
 public float PredictProbability(float output)
{
if (float.IsNaN(output))
return output;
return PredictProbability(output, Slope, Offset);
}
internal static float PredictProbability(float output, Double a, Double b)
{
return (float)(1 / (1 + Math.Exp(a * output + b)));
}

The value here should be changed to 1 only if we change the computation in PlattCalibrator. (looking at the baselines, the computation without the change looks better: label 1 examples give higher probabilities, and label 0 examples give lower probabilities.)
 }

 [TlcModule.EntryPoint(Name = "Trainers.LogisticRegressionBinaryClassifier",
Expand Down

12 changes: 6 additions & 6 deletions
...BaselineOutput/SingleDebug/LogisticRegression/LogisticRegression-CV-breast-cancer-out.txt

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -24,9 +24,9 @@ TRUTH ||======================
 ||======================
Precision || 0.9489 | 0.9816 |
OVERALL 0/1 ACCURACY: 0.968927
LOG LOSS/instance: 0.143504
LOG LOSS/instance: 8.202349
Test-set entropy (prior Log-Loss/instance): 0.956998
LOG-LOSS REDUCTION (RIG): 0.850048
LOG-LOSS REDUCTION (RIG): -7.570916
AUC: 0.994132
Warning: The predictor produced non-finite prediction values on 8 instances during testing. Possible causes: abnormal data or the predictor is numerically unstable.
TEST POSITIVE RATIO:	0.3191 (105.0/(105.0+224.0))
Expand All@@ -39,9 +39,9 @@ TRUTH ||======================
 ||======================
Precision || 0.9697 | 0.9609 |
OVERALL 0/1 ACCURACY: 0.963526
LOG LOSS/instance: 0.111794
LOG LOSS/instance: 6.914975
Test-set entropy (prior Log-Loss/instance): 0.903454
LOG-LOSS REDUCTION (RIG): 0.876260
LOG-LOSS REDUCTION (RIG): -6.653935
AUC: 0.997236

OVERALL RESULTS
Expand All@@ -52,8 +52,8 @@ Positive precision: 0.959301 (0.0104)
Positive recall: 0.942217 (0.0279)
Negative precision: 0.971218 (0.0103)
Negative recall: 0.977394 (0.0092)
Log-loss: 0.127649 (0.0159)
Log-loss reduction: 0.863154 (0.0131)
Log-loss: 7.558662 (0.6437)
Log-loss reduction: -7.112425 (0.4585)
F1 Score: 0.950293 (0.0091)
AUPRC: 0.991584 (0.0025)

Expand Down

2 changes: 1 addition & 1 deletion
.../BaselineOutput/SingleDebug/LogisticRegression/LogisticRegression-CV-breast-cancer-rp.txt

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
LogisticRegression
AUC	Accuracy	Positive precision	Positive recall	Negative precision	Negative recall	Log-loss	Log-loss reduction	F1 Score	AUPRC	/l2	/ot	/nt	Learner Name	Train Dataset	Test Dataset	Results File	Run Time	Physical Memory	Virtual Memory	Command Line	Settings	
0.995684	0.966226	0.959301	0.942217	0.971218	0.977394	0.127649	0.863154	0.950293	0.991584	0.1	0.001	1	LogisticRegression	%Data% %Output%	99	0	0	maml.exe CV tr=LogisticRegression{l1=1.0 l2=0.1 ot=1e-3 nt=1} threads=- norm=No dout=%Output% data=%Data% seed=1	/l2:0.1;/ot:0.001;/nt:1	
0.995684	0.966226	0.959301	0.942217	0.971218	0.977394	7.558662	-7.112425	0.950293	0.991584	0.1	0.001	1	LogisticRegression	%Data% %Output%	99	0	0	maml.exe CV tr=LogisticRegression{l1=1.0 l2=0.1 ot=1e-3 nt=1} threads=- norm=No dout=%Output% data=%Data% seed=1	/l2:0.1;/ot:0.001;/nt:1	


1,366 changes: 683 additions & 683 deletions
test/BaselineOutput/SingleDebug/LogisticRegression/LogisticRegression-CV-breast-cancer.txt

View file


Open in desktop


Large diffs are not rendered by default.


40 changes: 20 additions & 20 deletions
.../SingleDebug/LogisticRegression/LogisticRegression-CV-breast-cancer.withThreshold-out.txt

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -19,42 +19,42 @@ Confusion table
 ||======================
PREDICTED || positive | negative | Recall
TRUTH ||======================
 positive || 118 | 16 | 0.8806
 negative ||  3 | 217 | 0.9864
 positive ||  0 | 134 | 0.0000
 negative || 202 |  18 | 0.0818
 ||======================
Precision || 0.9752 | 0.9313 |
OVERALL 0/1 ACCURACY: 0.946328
LOG LOSS/instance: 0.143504
Precision || 0.0000 | 0.1184 |
OVERALL 0/1 ACCURACY: 0.050847
LOG LOSS/instance: 8.202349
Test-set entropy (prior Log-Loss/instance): 0.956998
LOG-LOSS REDUCTION (RIG): 0.850048
LOG-LOSS REDUCTION (RIG): -7.570916
AUC: 0.994132
Warning: The predictor produced non-finite prediction values on 8 instances during testing. Possible causes: abnormal data or the predictor is numerically unstable.
TEST POSITIVE RATIO:	0.3191 (105.0/(105.0+224.0))
Confusion table
 ||======================
PREDICTED || positive | negative | Recall
TRUTH ||======================
 positive || 81 |  24 | 0.7714
 negative ||  0 | 224 | 1.0000
 positive ||  0 | 105 | 0.0000
 negative || 210 |  14 | 0.0625
 ||======================
Precision || 1.0000 | 0.9032 |
OVERALL 0/1 ACCURACY: 0.927052
LOG LOSS/instance: 0.111794
Precision || 0.0000 | 0.1176 |
OVERALL 0/1 ACCURACY: 0.042553
LOG LOSS/instance: 6.914975
Test-set entropy (prior Log-Loss/instance): 0.903454
LOG-LOSS REDUCTION (RIG): 0.876260
LOG-LOSS REDUCTION (RIG): -6.653935
AUC: 0.997236

OVERALL RESULTS
---------------------------------------
AUC: 0.995684 (0.0016)
Accuracy: 0.936690 (0.0096)
Positive precision: 0.987603 (0.0124)
Positive recall: 0.826013 (0.0546)
Negative precision: 0.917278 (0.0141)
Negative recall: 0.993182 (0.0068)
Log-loss: 0.127649 (0.0159)
Log-loss reduction: 0.863154 (0.0131)
F1 Score: 0.898229 (0.0273)
Accuracy: 0.046700 (0.0041)
Positive precision: 0.000000 (0.0000)
Positive recall: 0.000000 (0.0000)
Negative precision: 0.118034 (0.0004)
Negative recall: 0.072159 (0.0097)
Log-loss: 7.558662 (0.6437)
Log-loss reduction: -7.112425 (0.4585)
F1 Score: 0.000000 (0.0000)
AUPRC: 0.991584 (0.0025)

---------------------------------------
Expand Down

2 changes: 1 addition & 1 deletion
...t/SingleDebug/LogisticRegression/LogisticRegression-CV-breast-cancer.withThreshold-rp.txt

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
LogisticRegression
AUC	Accuracy	Positive precision	Positive recall	Negative precision	Negative recall	Log-loss	Log-loss reduction	F1 Score	AUPRC	/l2	/ot	/nt	Learner Name	Train Dataset	Test Dataset	Results File	Run Time	Physical Memory	Virtual Memory	Command Line	Settings	
0.995684	0.93669	0.987603	0.826013	0.917278	0.993182	0.127649	0.863154	0.898229	0.991584	0.1	0.001	1	LogisticRegression	%Data% %Output%	99	0	0	maml.exe CV tr=LogisticRegression{l1=1.0 l2=0.1 ot=1e-3 nt=1} eval=BinaryClassifier{threshold=0.95 useRawScore=-} threads=- norm=No dout=%Output% data=%Data% seed=1	/l2:0.1;/ot:0.001;/nt:1	
0.995684	0.0467	0	0	0.118034	0.072159	7.558662	-7.112425	0	0.991584	0.1	0.001	1	LogisticRegression	%Data% %Output%	99	0	0	maml.exe CV tr=LogisticRegression{l1=1.0 l2=0.1 ot=1e-3 nt=1} eval=BinaryClassifier{threshold=0.95 useRawScore=-} threads=- norm=No dout=%Output% data=%Data% seed=1	/l2:0.1;/ot:0.001;/nt:1	

Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms");
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
WIP: Make sure Done() is called in cleanup if it hasn't been called#4817

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Closed

harishsk
wants to merge
3
commits into

from

 Closed


WIP: Make sure Done() is called in cleanup if it hasn't been called
#4817
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


3 commits


Select commit
Hold shift + click to select a range
cf46e5c

Make sure Done() is called in cleanup if it hasn't been called


harishsk Feb 8, 2020
836a44e

Fixed EntryPointCalibrate and TestEnsembleCombiner


harishsk Feb 8, 2020
6993980

Updated baselines to account for the change in default parameters for…


harishsk Feb 9, 2020

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


2 changes: 1 addition & 1 deletion
src/Microsoft.ML.StandardTrainers/Standard/LogisticRegression/LogisticRegression.cs

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -455,7 +455,7 @@ private protected override CalibratedModelParametersBase<LinearBinaryModelParame
 CurrentWeights.CopyTo(ref weights, 1, CurrentWeights.Length - 1);
 return new ParameterMixingCalibratedModelParameters<LinearBinaryModelParameters, PlattCalibrator>(Host,
 new LinearBinaryModelParameters(Host, in weights, bias, _stats),
 new PlattCalibrator(Host, -1, 0));
 new PlattCalibrator(Host, 1, 0));
@yaeldekelyaeldekelFeb 10, 2020

Copy link


Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems that the bug is actually in the PlattCalibrator itself:
While in the comment it says this:
 /// The Platt calibrator calculates the probability following:
/// P(x) = 1 / (1 + exp(-<see cref="PlattCalibrator.Slope"/> * x + <see cref="PlattCalibrator.Offset"/>)

The actual calculation does this:
 public float PredictProbability(float output)
{
if (float.IsNaN(output))
return output;
return PredictProbability(output, Slope, Offset);
}
internal static float PredictProbability(float output, Double a, Double b)
{
return (float)(1 / (1 + Math.Exp(a * output + b)));
}

The value here should be changed to 1 only if we change the computation in PlattCalibrator. (looking at the baselines, the computation without the change looks better: label 1 examples give higher probabilities, and label 0 examples give lower probabilities.)
 }

 [TlcModule.EntryPoint(Name = "Trainers.LogisticRegressionBinaryClassifier",
Expand Down

12 changes: 6 additions & 6 deletions
...BaselineOutput/SingleDebug/LogisticRegression/LogisticRegression-CV-breast-cancer-out.txt

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -24,9 +24,9 @@ TRUTH ||======================
 ||======================
Precision || 0.9489 | 0.9816 |
OVERALL 0/1 ACCURACY: 0.968927
LOG LOSS/instance: 0.143504
LOG LOSS/instance: 8.202349
Test-set entropy (prior Log-Loss/instance): 0.956998
LOG-LOSS REDUCTION (RIG): 0.850048
LOG-LOSS REDUCTION (RIG): -7.570916
AUC: 0.994132
Warning: The predictor produced non-finite prediction values on 8 instances during testing. Possible causes: abnormal data or the predictor is numerically unstable.
TEST POSITIVE RATIO:	0.3191 (105.0/(105.0+224.0))
Expand All@@ -39,9 +39,9 @@ TRUTH ||======================
 ||======================
Precision || 0.9697 | 0.9609 |
OVERALL 0/1 ACCURACY: 0.963526
LOG LOSS/instance: 0.111794
LOG LOSS/instance: 6.914975
Test-set entropy (prior Log-Loss/instance): 0.903454
LOG-LOSS REDUCTION (RIG): 0.876260
LOG-LOSS REDUCTION (RIG): -6.653935
AUC: 0.997236

OVERALL RESULTS
Expand All@@ -52,8 +52,8 @@ Positive precision: 0.959301 (0.0104)
Positive recall: 0.942217 (0.0279)
Negative precision: 0.971218 (0.0103)
Negative recall: 0.977394 (0.0092)
Log-loss: 0.127649 (0.0159)
Log-loss reduction: 0.863154 (0.0131)
Log-loss: 7.558662 (0.6437)
Log-loss reduction: -7.112425 (0.4585)
F1 Score: 0.950293 (0.0091)
AUPRC: 0.991584 (0.0025)

Expand Down

2 changes: 1 addition & 1 deletion
.../BaselineOutput/SingleDebug/LogisticRegression/LogisticRegression-CV-breast-cancer-rp.txt

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
LogisticRegression
AUC	Accuracy	Positive precision	Positive recall	Negative precision	Negative recall	Log-loss	Log-loss reduction	F1 Score	AUPRC	/l2	/ot	/nt	Learner Name	Train Dataset	Test Dataset	Results File	Run Time	Physical Memory	Virtual Memory	Command Line	Settings	
0.995684	0.966226	0.959301	0.942217	0.971218	0.977394	0.127649	0.863154	0.950293	0.991584	0.1	0.001	1	LogisticRegression	%Data% %Output%	99	0	0	maml.exe CV tr=LogisticRegression{l1=1.0 l2=0.1 ot=1e-3 nt=1} threads=- norm=No dout=%Output% data=%Data% seed=1	/l2:0.1;/ot:0.001;/nt:1	
0.995684	0.966226	0.959301	0.942217	0.971218	0.977394	7.558662	-7.112425	0.950293	0.991584	0.1	0.001	1	LogisticRegression	%Data% %Output%	99	0	0	maml.exe CV tr=LogisticRegression{l1=1.0 l2=0.1 ot=1e-3 nt=1} threads=- norm=No dout=%Output% data=%Data% seed=1	/l2:0.1;/ot:0.001;/nt:1	


1,366 changes: 683 additions & 683 deletions
test/BaselineOutput/SingleDebug/LogisticRegression/LogisticRegression-CV-breast-cancer.txt

View file


Open in desktop


Large diffs are not rendered by default.


40 changes: 20 additions & 20 deletions
.../SingleDebug/LogisticRegression/LogisticRegression-CV-breast-cancer.withThreshold-out.txt

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -19,42 +19,42 @@ Confusion table
 ||======================
PREDICTED || positive | negative | Recall
TRUTH ||======================
 positive || 118 | 16 | 0.8806
 negative ||  3 | 217 | 0.9864
 positive ||  0 | 134 | 0.0000
 negative || 202 |  18 | 0.0818
 ||======================
Precision || 0.9752 | 0.9313 |
OVERALL 0/1 ACCURACY: 0.946328
LOG LOSS/instance: 0.143504
Precision || 0.0000 | 0.1184 |
OVERALL 0/1 ACCURACY: 0.050847
LOG LOSS/instance: 8.202349
Test-set entropy (prior Log-Loss/instance): 0.956998
LOG-LOSS REDUCTION (RIG): 0.850048
LOG-LOSS REDUCTION (RIG): -7.570916
AUC: 0.994132
Warning: The predictor produced non-finite prediction values on 8 instances during testing. Possible causes: abnormal data or the predictor is numerically unstable.
TEST POSITIVE RATIO:	0.3191 (105.0/(105.0+224.0))
Confusion table
 ||======================
PREDICTED || positive | negative | Recall
TRUTH ||======================
 positive || 81 |  24 | 0.7714
 negative ||  0 | 224 | 1.0000
 positive ||  0 | 105 | 0.0000
 negative || 210 |  14 | 0.0625
 ||======================
Precision || 1.0000 | 0.9032 |
OVERALL 0/1 ACCURACY: 0.927052
LOG LOSS/instance: 0.111794
Precision || 0.0000 | 0.1176 |
OVERALL 0/1 ACCURACY: 0.042553
LOG LOSS/instance: 6.914975
Test-set entropy (prior Log-Loss/instance): 0.903454
LOG-LOSS REDUCTION (RIG): 0.876260
LOG-LOSS REDUCTION (RIG): -6.653935
AUC: 0.997236

OVERALL RESULTS
---------------------------------------
AUC: 0.995684 (0.0016)
Accuracy: 0.936690 (0.0096)
Positive precision: 0.987603 (0.0124)
Positive recall: 0.826013 (0.0546)
Negative precision: 0.917278 (0.0141)
Negative recall: 0.993182 (0.0068)
Log-loss: 0.127649 (0.0159)
Log-loss reduction: 0.863154 (0.0131)
F1 Score: 0.898229 (0.0273)
Accuracy: 0.046700 (0.0041)
Positive precision: 0.000000 (0.0000)
Positive recall: 0.000000 (0.0000)
Negative precision: 0.118034 (0.0004)
Negative recall: 0.072159 (0.0097)
Log-loss: 7.558662 (0.6437)
Log-loss reduction: -7.112425 (0.4585)
F1 Score: 0.000000 (0.0000)
AUPRC: 0.991584 (0.0025)

---------------------------------------
Expand Down

2 changes: 1 addition & 1 deletion
...t/SingleDebug/LogisticRegression/LogisticRegression-CV-breast-cancer.withThreshold-rp.txt

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
LogisticRegression
AUC	Accuracy	Positive precision	Positive recall	Negative precision	Negative recall	Log-loss	Log-loss reduction	F1 Score	AUPRC	/l2	/ot	/nt	Learner Name	Train Dataset	Test Dataset	Results File	Run Time	Physical Memory	Virtual Memory	Command Line	Settings	
0.995684	0.93669	0.987603	0.826013	0.917278	0.993182	0.127649	0.863154	0.898229	0.991584	0.1	0.001	1	LogisticRegression	%Data% %Output%	99	0	0	maml.exe CV tr=LogisticRegression{l1=1.0 l2=0.1 ot=1e-3 nt=1} eval=BinaryClassifier{threshold=0.95 useRawScore=-} threads=- norm=No dout=%Output% data=%Data% seed=1	/l2:0.1;/ot:0.001;/nt:1	
0.995684	0.0467	0	0	0.118034	0.072159	7.558662	-7.112425	0	0.991584	0.1	0.001	1	LogisticRegression	%Data% %Output%	99	0	0	maml.exe CV tr=LogisticRegression{l1=1.0 l2=0.1 ot=1e-3 nt=1} eval=BinaryClassifier{threshold=0.95 useRawScore=-} threads=- norm=No dout=%Output% data=%Data% seed=1	/l2:0.1;/ot:0.001;/nt:1	

Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links");
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '

Skip to content
WIP: Make sure Done() is called in cleanup if it hasn't been called#4817

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Closed

harishsk
wants to merge
3
commits into

from

 Closed


WIP: Make sure Done() is called in cleanup if it hasn't been called
#4817
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


3 commits


Select commit
Hold shift + click to select a range
cf46e5c

Make sure Done() is called in cleanup if it hasn't been called


harishsk Feb 8, 2020
836a44e

Fixed EntryPointCalibrate and TestEnsembleCombiner


harishsk Feb 8, 2020
6993980

Updated baselines to account for the change in default parameters for…


harishsk Feb 9, 2020

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


2 changes: 1 addition & 1 deletion
src/Microsoft.ML.StandardTrainers/Standard/LogisticRegression/LogisticRegression.cs

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -455,7 +455,7 @@ private protected override CalibratedModelParametersBase<LinearBinaryModelParame
 CurrentWeights.CopyTo(ref weights, 1, CurrentWeights.Length - 1);
 return new ParameterMixingCalibratedModelParameters<LinearBinaryModelParameters, PlattCalibrator>(Host,
 new LinearBinaryModelParameters(Host, in weights, bias, _stats),
 new PlattCalibrator(Host, -1, 0));
 new PlattCalibrator(Host, 1, 0));
@yaeldekelyaeldekelFeb 10, 2020

Copy link


Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems that the bug is actually in the PlattCalibrator itself:
While in the comment it says this:
 /// The Platt calibrator calculates the probability following:
/// P(x) = 1 / (1 + exp(-<see cref="PlattCalibrator.Slope"/> * x + <see cref="PlattCalibrator.Offset"/>)

The actual calculation does this:
 public float PredictProbability(float output)
{
if (float.IsNaN(output))
return output;
return PredictProbability(output, Slope, Offset);
}
internal static float PredictProbability(float output, Double a, Double b)
{
return (float)(1 / (1 + Math.Exp(a * output + b)));
}

The value here should be changed to 1 only if we change the computation in PlattCalibrator. (looking at the baselines, the computation without the change looks better: label 1 examples give higher probabilities, and label 0 examples give lower probabilities.)
 }

 [TlcModule.EntryPoint(Name = "Trainers.LogisticRegressionBinaryClassifier",
Expand Down

12 changes: 6 additions & 6 deletions
...BaselineOutput/SingleDebug/LogisticRegression/LogisticRegression-CV-breast-cancer-out.txt

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -24,9 +24,9 @@ TRUTH ||======================
 ||======================
Precision || 0.9489 | 0.9816 |
OVERALL 0/1 ACCURACY: 0.968927
LOG LOSS/instance: 0.143504
LOG LOSS/instance: 8.202349
Test-set entropy (prior Log-Loss/instance): 0.956998
LOG-LOSS REDUCTION (RIG): 0.850048
LOG-LOSS REDUCTION (RIG): -7.570916
AUC: 0.994132
Warning: The predictor produced non-finite prediction values on 8 instances during testing. Possible causes: abnormal data or the predictor is numerically unstable.
TEST POSITIVE RATIO:	0.3191 (105.0/(105.0+224.0))
Expand All@@ -39,9 +39,9 @@ TRUTH ||======================
 ||======================
Precision || 0.9697 | 0.9609 |
OVERALL 0/1 ACCURACY: 0.963526
LOG LOSS/instance: 0.111794
LOG LOSS/instance: 6.914975
Test-set entropy (prior Log-Loss/instance): 0.903454
LOG-LOSS REDUCTION (RIG): 0.876260
LOG-LOSS REDUCTION (RIG): -6.653935
AUC: 0.997236

OVERALL RESULTS
Expand All@@ -52,8 +52,8 @@ Positive precision: 0.959301 (0.0104)
Positive recall: 0.942217 (0.0279)
Negative precision: 0.971218 (0.0103)
Negative recall: 0.977394 (0.0092)
Log-loss: 0.127649 (0.0159)
Log-loss reduction: 0.863154 (0.0131)
Log-loss: 7.558662 (0.6437)
Log-loss reduction: -7.112425 (0.4585)
F1 Score: 0.950293 (0.0091)
AUPRC: 0.991584 (0.0025)

Expand Down

2 changes: 1 addition & 1 deletion
.../BaselineOutput/SingleDebug/LogisticRegression/LogisticRegression-CV-breast-cancer-rp.txt

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
LogisticRegression
AUC	Accuracy	Positive precision	Positive recall	Negative precision	Negative recall	Log-loss	Log-loss reduction	F1 Score	AUPRC	/l2	/ot	/nt	Learner Name	Train Dataset	Test Dataset	Results File	Run Time	Physical Memory	Virtual Memory	Command Line	Settings	
0.995684	0.966226	0.959301	0.942217	0.971218	0.977394	0.127649	0.863154	0.950293	0.991584	0.1	0.001	1	LogisticRegression	%Data% %Output%	99	0	0	maml.exe CV tr=LogisticRegression{l1=1.0 l2=0.1 ot=1e-3 nt=1} threads=- norm=No dout=%Output% data=%Data% seed=1	/l2:0.1;/ot:0.001;/nt:1	
0.995684	0.966226	0.959301	0.942217	0.971218	0.977394	7.558662	-7.112425	0.950293	0.991584	0.1	0.001	1	LogisticRegression	%Data% %Output%	99	0	0	maml.exe CV tr=LogisticRegression{l1=1.0 l2=0.1 ot=1e-3 nt=1} threads=- norm=No dout=%Output% data=%Data% seed=1	/l2:0.1;/ot:0.001;/nt:1	


1,366 changes: 683 additions & 683 deletions
test/BaselineOutput/SingleDebug/LogisticRegression/LogisticRegression-CV-breast-cancer.txt

View file


Open in desktop


Large diffs are not rendered by default.


40 changes: 20 additions & 20 deletions
.../SingleDebug/LogisticRegression/LogisticRegression-CV-breast-cancer.withThreshold-out.txt

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -19,42 +19,42 @@ Confusion table
 ||======================
PREDICTED || positive | negative | Recall
TRUTH ||======================
 positive || 118 | 16 | 0.8806
 negative ||  3 | 217 | 0.9864
 positive ||  0 | 134 | 0.0000
 negative || 202 |  18 | 0.0818
 ||======================
Precision || 0.9752 | 0.9313 |
OVERALL 0/1 ACCURACY: 0.946328
LOG LOSS/instance: 0.143504
Precision || 0.0000 | 0.1184 |
OVERALL 0/1 ACCURACY: 0.050847
LOG LOSS/instance: 8.202349
Test-set entropy (prior Log-Loss/instance): 0.956998
LOG-LOSS REDUCTION (RIG): 0.850048
LOG-LOSS REDUCTION (RIG): -7.570916
AUC: 0.994132
Warning: The predictor produced non-finite prediction values on 8 instances during testing. Possible causes: abnormal data or the predictor is numerically unstable.
TEST POSITIVE RATIO:	0.3191 (105.0/(105.0+224.0))
Confusion table
 ||======================
PREDICTED || positive | negative | Recall
TRUTH ||======================
 positive || 81 |  24 | 0.7714
 negative ||  0 | 224 | 1.0000
 positive ||  0 | 105 | 0.0000
 negative || 210 |  14 | 0.0625
 ||======================
Precision || 1.0000 | 0.9032 |
OVERALL 0/1 ACCURACY: 0.927052
LOG LOSS/instance: 0.111794
Precision || 0.0000 | 0.1176 |
OVERALL 0/1 ACCURACY: 0.042553
LOG LOSS/instance: 6.914975
Test-set entropy (prior Log-Loss/instance): 0.903454
LOG-LOSS REDUCTION (RIG): 0.876260
LOG-LOSS REDUCTION (RIG): -6.653935
AUC: 0.997236

OVERALL RESULTS
---------------------------------------
AUC: 0.995684 (0.0016)
Accuracy: 0.936690 (0.0096)
Positive precision: 0.987603 (0.0124)
Positive recall: 0.826013 (0.0546)
Negative precision: 0.917278 (0.0141)
Negative recall: 0.993182 (0.0068)
Log-loss: 0.127649 (0.0159)
Log-loss reduction: 0.863154 (0.0131)
F1 Score: 0.898229 (0.0273)
Accuracy: 0.046700 (0.0041)
Positive precision: 0.000000 (0.0000)
Positive recall: 0.000000 (0.0000)
Negative precision: 0.118034 (0.0004)
Negative recall: 0.072159 (0.0097)
Log-loss: 7.558662 (0.6437)
Log-loss reduction: -7.112425 (0.4585)
F1 Score: 0.000000 (0.0000)
AUPRC: 0.991584 (0.0025)

---------------------------------------
Expand Down

2 changes: 1 addition & 1 deletion
...t/SingleDebug/LogisticRegression/LogisticRegression-CV-breast-cancer.withThreshold-rp.txt

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
LogisticRegression
AUC	Accuracy	Positive precision	Positive recall	Negative precision	Negative recall	Log-loss	Log-loss reduction	F1 Score	AUPRC	/l2	/ot	/nt	Learner Name	Train Dataset	Test Dataset	Results File	Run Time	Physical Memory	Virtual Memory	Command Line	Settings	
0.995684	0.93669	0.987603	0.826013	0.917278	0.993182	0.127649	0.863154	0.898229	0.991584	0.1	0.001	1	LogisticRegression	%Data% %Output%	99	0	0	maml.exe CV tr=LogisticRegression{l1=1.0 l2=0.1 ot=1e-3 nt=1} eval=BinaryClassifier{threshold=0.95 useRawScore=-} threads=- norm=No dout=%Output% data=%Data% seed=1	/l2:0.1;/ot:0.001;/nt:1	
0.995684	0.0467	0	0	0.118034	0.072159	7.558662	-7.112425	0	0.991584	0.1	0.001	1	LogisticRegression	%Data% %Output%	99	0	0	maml.exe CV tr=LogisticRegression{l1=1.0 l2=0.1 ot=1e-3 nt=1} eval=BinaryClassifier{threshold=0.95 useRawScore=-} threads=- norm=No dout=%Output% data=%Data% seed=1	/l2:0.1;/ot:0.001;/nt:1	

Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default");
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
WIP: Make sure Done() is called in cleanup if it hasn't been called#4817

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Closed

harishsk
wants to merge
3
commits into

from

 Closed


WIP: Make sure Done() is called in cleanup if it hasn't been called
#4817
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


3 commits


Select commit
Hold shift + click to select a range
cf46e5c

Make sure Done() is called in cleanup if it hasn't been called


harishsk Feb 8, 2020
836a44e

Fixed EntryPointCalibrate and TestEnsembleCombiner


harishsk Feb 8, 2020
6993980

Updated baselines to account for the change in default parameters for…


harishsk Feb 9, 2020

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


2 changes: 1 addition & 1 deletion
src/Microsoft.ML.StandardTrainers/Standard/LogisticRegression/LogisticRegression.cs

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -455,7 +455,7 @@ private protected override CalibratedModelParametersBase<LinearBinaryModelParame
 CurrentWeights.CopyTo(ref weights, 1, CurrentWeights.Length - 1);
 return new ParameterMixingCalibratedModelParameters<LinearBinaryModelParameters, PlattCalibrator>(Host,
 new LinearBinaryModelParameters(Host, in weights, bias, _stats),
 new PlattCalibrator(Host, -1, 0));
 new PlattCalibrator(Host, 1, 0));
@yaeldekelyaeldekelFeb 10, 2020

Copy link


Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems that the bug is actually in the PlattCalibrator itself:
While in the comment it says this:
 /// The Platt calibrator calculates the probability following:
/// P(x) = 1 / (1 + exp(-<see cref="PlattCalibrator.Slope"/> * x + <see cref="PlattCalibrator.Offset"/>)

The actual calculation does this:
 public float PredictProbability(float output)
{
if (float.IsNaN(output))
return output;
return PredictProbability(output, Slope, Offset);
}
internal static float PredictProbability(float output, Double a, Double b)
{
return (float)(1 / (1 + Math.Exp(a * output + b)));
}

The value here should be changed to 1 only if we change the computation in PlattCalibrator. (looking at the baselines, the computation without the change looks better: label 1 examples give higher probabilities, and label 0 examples give lower probabilities.)
 }

 [TlcModule.EntryPoint(Name = "Trainers.LogisticRegressionBinaryClassifier",
Expand Down

12 changes: 6 additions & 6 deletions
...BaselineOutput/SingleDebug/LogisticRegression/LogisticRegression-CV-breast-cancer-out.txt

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -24,9 +24,9 @@ TRUTH ||======================
 ||======================
Precision || 0.9489 | 0.9816 |
OVERALL 0/1 ACCURACY: 0.968927
LOG LOSS/instance: 0.143504
LOG LOSS/instance: 8.202349
Test-set entropy (prior Log-Loss/instance): 0.956998
LOG-LOSS REDUCTION (RIG): 0.850048
LOG-LOSS REDUCTION (RIG): -7.570916
AUC: 0.994132
Warning: The predictor produced non-finite prediction values on 8 instances during testing. Possible causes: abnormal data or the predictor is numerically unstable.
TEST POSITIVE RATIO:	0.3191 (105.0/(105.0+224.0))
Expand All@@ -39,9 +39,9 @@ TRUTH ||======================
 ||======================
Precision || 0.9697 | 0.9609 |
OVERALL 0/1 ACCURACY: 0.963526
LOG LOSS/instance: 0.111794
LOG LOSS/instance: 6.914975
Test-set entropy (prior Log-Loss/instance): 0.903454
LOG-LOSS REDUCTION (RIG): 0.876260
LOG-LOSS REDUCTION (RIG): -6.653935
AUC: 0.997236

OVERALL RESULTS
Expand All@@ -52,8 +52,8 @@ Positive precision: 0.959301 (0.0104)
Positive recall: 0.942217 (0.0279)
Negative precision: 0.971218 (0.0103)
Negative recall: 0.977394 (0.0092)
Log-loss: 0.127649 (0.0159)
Log-loss reduction: 0.863154 (0.0131)
Log-loss: 7.558662 (0.6437)
Log-loss reduction: -7.112425 (0.4585)
F1 Score: 0.950293 (0.0091)
AUPRC: 0.991584 (0.0025)

Expand Down

2 changes: 1 addition & 1 deletion
.../BaselineOutput/SingleDebug/LogisticRegression/LogisticRegression-CV-breast-cancer-rp.txt

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
LogisticRegression
AUC	Accuracy	Positive precision	Positive recall	Negative precision	Negative recall	Log-loss	Log-loss reduction	F1 Score	AUPRC	/l2	/ot	/nt	Learner Name	Train Dataset	Test Dataset	Results File	Run Time	Physical Memory	Virtual Memory	Command Line	Settings	
0.995684	0.966226	0.959301	0.942217	0.971218	0.977394	0.127649	0.863154	0.950293	0.991584	0.1	0.001	1	LogisticRegression	%Data% %Output%	99	0	0	maml.exe CV tr=LogisticRegression{l1=1.0 l2=0.1 ot=1e-3 nt=1} threads=- norm=No dout=%Output% data=%Data% seed=1	/l2:0.1;/ot:0.001;/nt:1	
0.995684	0.966226	0.959301	0.942217	0.971218	0.977394	7.558662	-7.112425	0.950293	0.991584	0.1	0.001	1	LogisticRegression	%Data% %Output%	99	0	0	maml.exe CV tr=LogisticRegression{l1=1.0 l2=0.1 ot=1e-3 nt=1} threads=- norm=No dout=%Output% data=%Data% seed=1	/l2:0.1;/ot:0.001;/nt:1	


1,366 changes: 683 additions & 683 deletions
test/BaselineOutput/SingleDebug/LogisticRegression/LogisticRegression-CV-breast-cancer.txt

View file


Open in desktop


Large diffs are not rendered by default.


40 changes: 20 additions & 20 deletions
.../SingleDebug/LogisticRegression/LogisticRegression-CV-breast-cancer.withThreshold-out.txt

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -19,42 +19,42 @@ Confusion table
 ||======================
PREDICTED || positive | negative | Recall
TRUTH ||======================
 positive || 118 | 16 | 0.8806
 negative ||  3 | 217 | 0.9864
 positive ||  0 | 134 | 0.0000
 negative || 202 |  18 | 0.0818
 ||======================
Precision || 0.9752 | 0.9313 |
OVERALL 0/1 ACCURACY: 0.946328
LOG LOSS/instance: 0.143504
Precision || 0.0000 | 0.1184 |
OVERALL 0/1 ACCURACY: 0.050847
LOG LOSS/instance: 8.202349
Test-set entropy (prior Log-Loss/instance): 0.956998
LOG-LOSS REDUCTION (RIG): 0.850048
LOG-LOSS REDUCTION (RIG): -7.570916
AUC: 0.994132
Warning: The predictor produced non-finite prediction values on 8 instances during testing. Possible causes: abnormal data or the predictor is numerically unstable.
TEST POSITIVE RATIO:	0.3191 (105.0/(105.0+224.0))
Confusion table
 ||======================
PREDICTED || positive | negative | Recall
TRUTH ||======================
 positive || 81 |  24 | 0.7714
 negative ||  0 | 224 | 1.0000
 positive ||  0 | 105 | 0.0000
 negative || 210 |  14 | 0.0625
 ||======================
Precision || 1.0000 | 0.9032 |
OVERALL 0/1 ACCURACY: 0.927052
LOG LOSS/instance: 0.111794
Precision || 0.0000 | 0.1176 |
OVERALL 0/1 ACCURACY: 0.042553
LOG LOSS/instance: 6.914975
Test-set entropy (prior Log-Loss/instance): 0.903454
LOG-LOSS REDUCTION (RIG): 0.876260
LOG-LOSS REDUCTION (RIG): -6.653935
AUC: 0.997236

OVERALL RESULTS
---------------------------------------
AUC: 0.995684 (0.0016)
Accuracy: 0.936690 (0.0096)
Positive precision: 0.987603 (0.0124)
Positive recall: 0.826013 (0.0546)
Negative precision: 0.917278 (0.0141)
Negative recall: 0.993182 (0.0068)
Log-loss: 0.127649 (0.0159)
Log-loss reduction: 0.863154 (0.0131)
F1 Score: 0.898229 (0.0273)
Accuracy: 0.046700 (0.0041)
Positive precision: 0.000000 (0.0000)
Positive recall: 0.000000 (0.0000)
Negative precision: 0.118034 (0.0004)
Negative recall: 0.072159 (0.0097)
Log-loss: 7.558662 (0.6437)
Log-loss reduction: -7.112425 (0.4585)
F1 Score: 0.000000 (0.0000)
AUPRC: 0.991584 (0.0025)

---------------------------------------
Expand Down

2 changes: 1 addition & 1 deletion
...t/SingleDebug/LogisticRegression/LogisticRegression-CV-breast-cancer.withThreshold-rp.txt

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
LogisticRegression
AUC	Accuracy	Positive precision	Positive recall	Negative precision	Negative recall	Log-loss	Log-loss reduction	F1 Score	AUPRC	/l2	/ot	/nt	Learner Name	Train Dataset	Test Dataset	Results File	Run Time	Physical Memory	Virtual Memory	Command Line	Settings	
0.995684	0.93669	0.987603	0.826013	0.917278	0.993182	0.127649	0.863154	0.898229	0.991584	0.1	0.001	1	LogisticRegression	%Data% %Output%	99	0	0	maml.exe CV tr=LogisticRegression{l1=1.0 l2=0.1 ot=1e-3 nt=1} eval=BinaryClassifier{threshold=0.95 useRawScore=-} threads=- norm=No dout=%Output% data=%Data% seed=1	/l2:0.1;/ot:0.001;/nt:1	
0.995684	0.0467	0	0	0.118034	0.072159	7.558662	-7.112425	0	0.991584	0.1	0.001	1	LogisticRegression	%Data% %Output%	99	0	0	maml.exe CV tr=LogisticRegression{l1=1.0 l2=0.1 ot=1e-3 nt=1} eval=BinaryClassifier{threshold=0.95 useRawScore=-} threads=- norm=No dout=%Output% data=%Data% seed=1	/l2:0.1;/ot:0.001;/nt:1	

Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers");
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
WIP: Make sure Done() is called in cleanup if it hasn't been called#4817

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Closed

harishsk
wants to merge
3
commits into

from

 Closed


WIP: Make sure Done() is called in cleanup if it hasn't been called
#4817
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


3 commits


Select commit
Hold shift + click to select a range
cf46e5c

Make sure Done() is called in cleanup if it hasn't been called


harishsk Feb 8, 2020
836a44e

Fixed EntryPointCalibrate and TestEnsembleCombiner


harishsk Feb 8, 2020
6993980

Updated baselines to account for the change in default parameters for…


harishsk Feb 9, 2020

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


2 changes: 1 addition & 1 deletion
src/Microsoft.ML.StandardTrainers/Standard/LogisticRegression/LogisticRegression.cs

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -455,7 +455,7 @@ private protected override CalibratedModelParametersBase<LinearBinaryModelParame
 CurrentWeights.CopyTo(ref weights, 1, CurrentWeights.Length - 1);
 return new ParameterMixingCalibratedModelParameters<LinearBinaryModelParameters, PlattCalibrator>(Host,
 new LinearBinaryModelParameters(Host, in weights, bias, _stats),
 new PlattCalibrator(Host, -1, 0));
 new PlattCalibrator(Host, 1, 0));
@yaeldekelyaeldekelFeb 10, 2020

Copy link


Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems that the bug is actually in the PlattCalibrator itself:
While in the comment it says this:
 /// The Platt calibrator calculates the probability following:
/// P(x) = 1 / (1 + exp(-<see cref="PlattCalibrator.Slope"/> * x + <see cref="PlattCalibrator.Offset"/>)

The actual calculation does this:
 public float PredictProbability(float output)
{
if (float.IsNaN(output))
return output;
return PredictProbability(output, Slope, Offset);
}
internal static float PredictProbability(float output, Double a, Double b)
{
return (float)(1 / (1 + Math.Exp(a * output + b)));
}

The value here should be changed to 1 only if we change the computation in PlattCalibrator. (looking at the baselines, the computation without the change looks better: label 1 examples give higher probabilities, and label 0 examples give lower probabilities.)
 }

 [TlcModule.EntryPoint(Name = "Trainers.LogisticRegressionBinaryClassifier",
Expand Down

12 changes: 6 additions & 6 deletions
...BaselineOutput/SingleDebug/LogisticRegression/LogisticRegression-CV-breast-cancer-out.txt

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -24,9 +24,9 @@ TRUTH ||======================
 ||======================
Precision || 0.9489 | 0.9816 |
OVERALL 0/1 ACCURACY: 0.968927
LOG LOSS/instance: 0.143504
LOG LOSS/instance: 8.202349
Test-set entropy (prior Log-Loss/instance): 0.956998
LOG-LOSS REDUCTION (RIG): 0.850048
LOG-LOSS REDUCTION (RIG): -7.570916
AUC: 0.994132
Warning: The predictor produced non-finite prediction values on 8 instances during testing. Possible causes: abnormal data or the predictor is numerically unstable.
TEST POSITIVE RATIO:	0.3191 (105.0/(105.0+224.0))
Expand All@@ -39,9 +39,9 @@ TRUTH ||======================
 ||======================
Precision || 0.9697 | 0.9609 |
OVERALL 0/1 ACCURACY: 0.963526
LOG LOSS/instance: 0.111794
LOG LOSS/instance: 6.914975
Test-set entropy (prior Log-Loss/instance): 0.903454
LOG-LOSS REDUCTION (RIG): 0.876260
LOG-LOSS REDUCTION (RIG): -6.653935
AUC: 0.997236

OVERALL RESULTS
Expand All@@ -52,8 +52,8 @@ Positive precision: 0.959301 (0.0104)
Positive recall: 0.942217 (0.0279)
Negative precision: 0.971218 (0.0103)
Negative recall: 0.977394 (0.0092)
Log-loss: 0.127649 (0.0159)
Log-loss reduction: 0.863154 (0.0131)
Log-loss: 7.558662 (0.6437)
Log-loss reduction: -7.112425 (0.4585)
F1 Score: 0.950293 (0.0091)
AUPRC: 0.991584 (0.0025)

Expand Down

2 changes: 1 addition & 1 deletion
.../BaselineOutput/SingleDebug/LogisticRegression/LogisticRegression-CV-breast-cancer-rp.txt

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
LogisticRegression
AUC	Accuracy	Positive precision	Positive recall	Negative precision	Negative recall	Log-loss	Log-loss reduction	F1 Score	AUPRC	/l2	/ot	/nt	Learner Name	Train Dataset	Test Dataset	Results File	Run Time	Physical Memory	Virtual Memory	Command Line	Settings	
0.995684	0.966226	0.959301	0.942217	0.971218	0.977394	0.127649	0.863154	0.950293	0.991584	0.1	0.001	1	LogisticRegression	%Data% %Output%	99	0	0	maml.exe CV tr=LogisticRegression{l1=1.0 l2=0.1 ot=1e-3 nt=1} threads=- norm=No dout=%Output% data=%Data% seed=1	/l2:0.1;/ot:0.001;/nt:1	
0.995684	0.966226	0.959301	0.942217	0.971218	0.977394	7.558662	-7.112425	0.950293	0.991584	0.1	0.001	1	LogisticRegression	%Data% %Output%	99	0	0	maml.exe CV tr=LogisticRegression{l1=1.0 l2=0.1 ot=1e-3 nt=1} threads=- norm=No dout=%Output% data=%Data% seed=1	/l2:0.1;/ot:0.001;/nt:1	


1,366 changes: 683 additions & 683 deletions
test/BaselineOutput/SingleDebug/LogisticRegression/LogisticRegression-CV-breast-cancer.txt

View file


Open in desktop


Large diffs are not rendered by default.


40 changes: 20 additions & 20 deletions
.../SingleDebug/LogisticRegression/LogisticRegression-CV-breast-cancer.withThreshold-out.txt

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -19,42 +19,42 @@ Confusion table
 ||======================
PREDICTED || positive | negative | Recall
TRUTH ||======================
 positive || 118 | 16 | 0.8806
 negative ||  3 | 217 | 0.9864
 positive ||  0 | 134 | 0.0000
 negative || 202 |  18 | 0.0818
 ||======================
Precision || 0.9752 | 0.9313 |
OVERALL 0/1 ACCURACY: 0.946328
LOG LOSS/instance: 0.143504
Precision || 0.0000 | 0.1184 |
OVERALL 0/1 ACCURACY: 0.050847
LOG LOSS/instance: 8.202349
Test-set entropy (prior Log-Loss/instance): 0.956998
LOG-LOSS REDUCTION (RIG): 0.850048
LOG-LOSS REDUCTION (RIG): -7.570916
AUC: 0.994132
Warning: The predictor produced non-finite prediction values on 8 instances during testing. Possible causes: abnormal data or the predictor is numerically unstable.
TEST POSITIVE RATIO:	0.3191 (105.0/(105.0+224.0))
Confusion table
 ||======================
PREDICTED || positive | negative | Recall
TRUTH ||======================
 positive || 81 |  24 | 0.7714
 negative ||  0 | 224 | 1.0000
 positive ||  0 | 105 | 0.0000
 negative || 210 |  14 | 0.0625
 ||======================
Precision || 1.0000 | 0.9032 |
OVERALL 0/1 ACCURACY: 0.927052
LOG LOSS/instance: 0.111794
Precision || 0.0000 | 0.1176 |
OVERALL 0/1 ACCURACY: 0.042553
LOG LOSS/instance: 6.914975
Test-set entropy (prior Log-Loss/instance): 0.903454
LOG-LOSS REDUCTION (RIG): 0.876260
LOG-LOSS REDUCTION (RIG): -6.653935
AUC: 0.997236

OVERALL RESULTS
---------------------------------------
AUC: 0.995684 (0.0016)
Accuracy: 0.936690 (0.0096)
Positive precision: 0.987603 (0.0124)
Positive recall: 0.826013 (0.0546)
Negative precision: 0.917278 (0.0141)
Negative recall: 0.993182 (0.0068)
Log-loss: 0.127649 (0.0159)
Log-loss reduction: 0.863154 (0.0131)
F1 Score: 0.898229 (0.0273)
Accuracy: 0.046700 (0.0041)
Positive precision: 0.000000 (0.0000)
Positive recall: 0.000000 (0.0000)
Negative precision: 0.118034 (0.0004)
Negative recall: 0.072159 (0.0097)
Log-loss: 7.558662 (0.6437)
Log-loss reduction: -7.112425 (0.4585)
F1 Score: 0.000000 (0.0000)
AUPRC: 0.991584 (0.0025)

---------------------------------------
Expand Down

2 changes: 1 addition & 1 deletion
...t/SingleDebug/LogisticRegression/LogisticRegression-CV-breast-cancer.withThreshold-rp.txt

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
LogisticRegression
AUC	Accuracy	Positive precision	Positive recall	Negative precision	Negative recall	Log-loss	Log-loss reduction	F1 Score	AUPRC	/l2	/ot	/nt	Learner Name	Train Dataset	Test Dataset	Results File	Run Time	Physical Memory	Virtual Memory	Command Line	Settings	
0.995684	0.93669	0.987603	0.826013	0.917278	0.993182	0.127649	0.863154	0.898229	0.991584	0.1	0.001	1	LogisticRegression	%Data% %Output%	99	0	0	maml.exe CV tr=LogisticRegression{l1=1.0 l2=0.1 ot=1e-3 nt=1} eval=BinaryClassifier{threshold=0.95 useRawScore=-} threads=- norm=No dout=%Output% data=%Data% seed=1	/l2:0.1;/ot:0.001;/nt:1	
0.995684	0.0467	0	0	0.118034	0.072159	7.558662	-7.112425	0	0.991584	0.1	0.001	1	LogisticRegression	%Data% %Output%	99	0	0	maml.exe CV tr=LogisticRegression{l1=1.0 l2=0.1 ot=1e-3 nt=1} eval=BinaryClassifier{threshold=0.95 useRawScore=-} threads=- norm=No dout=%Output% data=%Data% seed=1	/l2:0.1;/ot:0.001;/nt:1	

Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode");
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();

Skip to content
WIP: Make sure Done() is called in cleanup if it hasn't been called#4817

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Closed

harishsk
wants to merge
3
commits into

from

 Closed


WIP: Make sure Done() is called in cleanup if it hasn't been called
#4817
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


3 commits


Select commit
Hold shift + click to select a range
cf46e5c

Make sure Done() is called in cleanup if it hasn't been called


harishsk Feb 8, 2020
836a44e

Fixed EntryPointCalibrate and TestEnsembleCombiner


harishsk Feb 8, 2020
6993980

Updated baselines to account for the change in default parameters for…


harishsk Feb 9, 2020

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


2 changes: 1 addition & 1 deletion
src/Microsoft.ML.StandardTrainers/Standard/LogisticRegression/LogisticRegression.cs

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -455,7 +455,7 @@ private protected override CalibratedModelParametersBase<LinearBinaryModelParame
 CurrentWeights.CopyTo(ref weights, 1, CurrentWeights.Length - 1);
 return new ParameterMixingCalibratedModelParameters<LinearBinaryModelParameters, PlattCalibrator>(Host,
 new LinearBinaryModelParameters(Host, in weights, bias, _stats),
 new PlattCalibrator(Host, -1, 0));
 new PlattCalibrator(Host, 1, 0));
@yaeldekelyaeldekelFeb 10, 2020

Copy link


Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems that the bug is actually in the PlattCalibrator itself:
While in the comment it says this:
 /// The Platt calibrator calculates the probability following:
/// P(x) = 1 / (1 + exp(-<see cref="PlattCalibrator.Slope"/> * x + <see cref="PlattCalibrator.Offset"/>)

The actual calculation does this:
 public float PredictProbability(float output)
{
if (float.IsNaN(output))
return output;
return PredictProbability(output, Slope, Offset);
}
internal static float PredictProbability(float output, Double a, Double b)
{
return (float)(1 / (1 + Math.Exp(a * output + b)));
}

The value here should be changed to 1 only if we change the computation in PlattCalibrator. (looking at the baselines, the computation without the change looks better: label 1 examples give higher probabilities, and label 0 examples give lower probabilities.)
 }

 [TlcModule.EntryPoint(Name = "Trainers.LogisticRegressionBinaryClassifier",
Expand Down

12 changes: 6 additions & 6 deletions
...BaselineOutput/SingleDebug/LogisticRegression/LogisticRegression-CV-breast-cancer-out.txt

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -24,9 +24,9 @@ TRUTH ||======================
 ||======================
Precision || 0.9489 | 0.9816 |
OVERALL 0/1 ACCURACY: 0.968927
LOG LOSS/instance: 0.143504
LOG LOSS/instance: 8.202349
Test-set entropy (prior Log-Loss/instance): 0.956998
LOG-LOSS REDUCTION (RIG): 0.850048
LOG-LOSS REDUCTION (RIG): -7.570916
AUC: 0.994132
Warning: The predictor produced non-finite prediction values on 8 instances during testing. Possible causes: abnormal data or the predictor is numerically unstable.
TEST POSITIVE RATIO:	0.3191 (105.0/(105.0+224.0))
Expand All@@ -39,9 +39,9 @@ TRUTH ||======================
 ||======================
Precision || 0.9697 | 0.9609 |
OVERALL 0/1 ACCURACY: 0.963526
LOG LOSS/instance: 0.111794
LOG LOSS/instance: 6.914975
Test-set entropy (prior Log-Loss/instance): 0.903454
LOG-LOSS REDUCTION (RIG): 0.876260
LOG-LOSS REDUCTION (RIG): -6.653935
AUC: 0.997236

OVERALL RESULTS
Expand All@@ -52,8 +52,8 @@ Positive precision: 0.959301 (0.0104)
Positive recall: 0.942217 (0.0279)
Negative precision: 0.971218 (0.0103)
Negative recall: 0.977394 (0.0092)
Log-loss: 0.127649 (0.0159)
Log-loss reduction: 0.863154 (0.0131)
Log-loss: 7.558662 (0.6437)
Log-loss reduction: -7.112425 (0.4585)
F1 Score: 0.950293 (0.0091)
AUPRC: 0.991584 (0.0025)

Expand Down

2 changes: 1 addition & 1 deletion
.../BaselineOutput/SingleDebug/LogisticRegression/LogisticRegression-CV-breast-cancer-rp.txt

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
LogisticRegression
AUC	Accuracy	Positive precision	Positive recall	Negative precision	Negative recall	Log-loss	Log-loss reduction	F1 Score	AUPRC	/l2	/ot	/nt	Learner Name	Train Dataset	Test Dataset	Results File	Run Time	Physical Memory	Virtual Memory	Command Line	Settings	
0.995684	0.966226	0.959301	0.942217	0.971218	0.977394	0.127649	0.863154	0.950293	0.991584	0.1	0.001	1	LogisticRegression	%Data% %Output%	99	0	0	maml.exe CV tr=LogisticRegression{l1=1.0 l2=0.1 ot=1e-3 nt=1} threads=- norm=No dout=%Output% data=%Data% seed=1	/l2:0.1;/ot:0.001;/nt:1	
0.995684	0.966226	0.959301	0.942217	0.971218	0.977394	7.558662	-7.112425	0.950293	0.991584	0.1	0.001	1	LogisticRegression	%Data% %Output%	99	0	0	maml.exe CV tr=LogisticRegression{l1=1.0 l2=0.1 ot=1e-3 nt=1} threads=- norm=No dout=%Output% data=%Data% seed=1	/l2:0.1;/ot:0.001;/nt:1	


1,366 changes: 683 additions & 683 deletions
test/BaselineOutput/SingleDebug/LogisticRegression/LogisticRegression-CV-breast-cancer.txt

View file


Open in desktop


Large diffs are not rendered by default.


40 changes: 20 additions & 20 deletions
.../SingleDebug/LogisticRegression/LogisticRegression-CV-breast-cancer.withThreshold-out.txt

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -19,42 +19,42 @@ Confusion table
 ||======================
PREDICTED || positive | negative | Recall
TRUTH ||======================
 positive || 118 | 16 | 0.8806
 negative ||  3 | 217 | 0.9864
 positive ||  0 | 134 | 0.0000
 negative || 202 |  18 | 0.0818
 ||======================
Precision || 0.9752 | 0.9313 |
OVERALL 0/1 ACCURACY: 0.946328
LOG LOSS/instance: 0.143504
Precision || 0.0000 | 0.1184 |
OVERALL 0/1 ACCURACY: 0.050847
LOG LOSS/instance: 8.202349
Test-set entropy (prior Log-Loss/instance): 0.956998
LOG-LOSS REDUCTION (RIG): 0.850048
LOG-LOSS REDUCTION (RIG): -7.570916
AUC: 0.994132
Warning: The predictor produced non-finite prediction values on 8 instances during testing. Possible causes: abnormal data or the predictor is numerically unstable.
TEST POSITIVE RATIO:	0.3191 (105.0/(105.0+224.0))
Confusion table
 ||======================
PREDICTED || positive | negative | Recall
TRUTH ||======================
 positive || 81 |  24 | 0.7714
 negative ||  0 | 224 | 1.0000
 positive ||  0 | 105 | 0.0000
 negative || 210 |  14 | 0.0625
 ||======================
Precision || 1.0000 | 0.9032 |
OVERALL 0/1 ACCURACY: 0.927052
LOG LOSS/instance: 0.111794
Precision || 0.0000 | 0.1176 |
OVERALL 0/1 ACCURACY: 0.042553
LOG LOSS/instance: 6.914975
Test-set entropy (prior Log-Loss/instance): 0.903454
LOG-LOSS REDUCTION (RIG): 0.876260
LOG-LOSS REDUCTION (RIG): -6.653935
AUC: 0.997236

OVERALL RESULTS
---------------------------------------
AUC: 0.995684 (0.0016)
Accuracy: 0.936690 (0.0096)
Positive precision: 0.987603 (0.0124)
Positive recall: 0.826013 (0.0546)
Negative precision: 0.917278 (0.0141)
Negative recall: 0.993182 (0.0068)
Log-loss: 0.127649 (0.0159)
Log-loss reduction: 0.863154 (0.0131)
F1 Score: 0.898229 (0.0273)
Accuracy: 0.046700 (0.0041)
Positive precision: 0.000000 (0.0000)
Positive recall: 0.000000 (0.0000)
Negative precision: 0.118034 (0.0004)
Negative recall: 0.072159 (0.0097)
Log-loss: 7.558662 (0.6437)
Log-loss reduction: -7.112425 (0.4585)
F1 Score: 0.000000 (0.0000)
AUPRC: 0.991584 (0.0025)

---------------------------------------
Expand Down

2 changes: 1 addition & 1 deletion
...t/SingleDebug/LogisticRegression/LogisticRegression-CV-breast-cancer.withThreshold-rp.txt

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
LogisticRegression
AUC	Accuracy	Positive precision	Positive recall	Negative precision	Negative recall	Log-loss	Log-loss reduction	F1 Score	AUPRC	/l2	/ot	/nt	Learner Name	Train Dataset	Test Dataset	Results File	Run Time	Physical Memory	Virtual Memory	Command Line	Settings	
0.995684	0.93669	0.987603	0.826013	0.917278	0.993182	0.127649	0.863154	0.898229	0.991584	0.1	0.001	1	LogisticRegression	%Data% %Output%	99	0	0	maml.exe CV tr=LogisticRegression{l1=1.0 l2=0.1 ot=1e-3 nt=1} eval=BinaryClassifier{threshold=0.95 useRawScore=-} threads=- norm=No dout=%Output% data=%Data% seed=1	/l2:0.1;/ot:0.001;/nt:1	
0.995684	0.0467	0	0	0.118034	0.072159	7.558662	-7.112425	0	0.991584	0.1	0.001	1	LogisticRegression	%Data% %Output%	99	0	0	maml.exe CV tr=LogisticRegression{l1=1.0 l2=0.1 ot=1e-3 nt=1} eval=BinaryClassifier{threshold=0.95 useRawScore=-} threads=- norm=No dout=%Output% data=%Data% seed=1	/l2:0.1;/ot:0.001;/nt:1	

Loading