Using invariance culture when converting to string - #4635

Merged
LittleLittleCloud merged 2 commits into
dotnet:masterfrom
LittleLittleCloud:u/xiaoyun/removeCultureInfo
Jan 21, 2020
Merged

Using invariance culture when converting to string#4635
LittleLittleCloud merged 2 commits into
dotnet:masterfrom
LittleLittleCloud:u/xiaoyun/removeCultureInfo

Conversation

@LittleLittleCloud

@LittleLittleCloudLittleLittleCloud commented Jan 7, 2020

Copy link
Copy Markdown
Member

@LittleLittleCloud
LittleLittleCloud requested a review from a team as a code ownerJanuary 7, 2020 22:38
@LittleLittleCloudLittleLittleCloud added the AutoML.NET Automating various steps of the machine learning process label Jan 7, 2020
justinormont
justinormont reviewed Jan 7, 2020
View reviewed changes
Comment threadsrc/Microsoft.ML.AutoML/Utils/SweepableParamAttributes.cs Outdated
public override void SetUsingValueText(string valueText)
{
RawValue = float.Parse(valueText, CultureInfo.InvariantCulture);
RawValue = float.Parse(valueText);

@justinormontjustinormontJan 7, 2020

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would instead push the culture invariant value into the sweepable parameter:

machinelearning/src/Microsoft.ML.Sweeper/Parameters.cs

Line 140 in b861b5d

_valueText=_value.ToString("R");

@codecov

codecovBot commented Jan 9, 2020•
edited
Loading

Copy link
Copy Markdown

Codecov Report

Merging #4635 into master will increase coverage by 0.39%.
The diff coverage is 100%.

@@ Coverage Diff @@## master #4635 +/- ##
==========================================
+ Coverage 75.63% 76.02% +0.39% 
==========================================
Files 938 951 +13 Lines 168669 173945 +5276 Branches 18217 18885 +668 ==========================================
+ Hits 127574 132247 +4673 - Misses 36065 36541 +476 - Partials 5030 5157 +127
FlagCoverage Δ
#Debug76.02% <100%> (+0.39%)⬆️
#production71.48% <100%> (+0.24%)⬆️
#test90.78% <ø> (+0.33%)⬆️
Impacted FilesCoverage Δ
src/Microsoft.ML.AutoML/Sweepers/Parameters.cs84.32% <100%> (ø)⬆️
src/Microsoft.ML.Data/Utilities/SlotDropper.cs96.5% <0%> (-2.75%)⬇️
...rc/Microsoft.ML.Featurizers/DateTimeTransformer.cs89.42% <0%> (-1.16%)⬇️
...s/Api/CookbookSamples/CookbookSamplesDynamicApi.cs95.28% <0%> (-0.2%)⬇️
...ML.Transforms/Text/StopWordsRemovingTransformer.cs86.1% <0%> (-0.16%)⬇️
...Microsoft.ML.Tests/Transformers/NormalizerTests.cs100% <0%> (ø)⬆️
...soft.ML.Tests/PermutationFeatureImportanceTests.cs100% <0%> (ø)⬆️
...ft.ML.Tests/Transformers/TimeSeriesImputerTests.cs100% <0%> (ø)
src/Microsoft.ML.Featurizers/TimeSeriesImputer.cs87.08% <0%> (ø)
....ML.Tests/Transformers/ToStringTransformerTests.cs100% <0%> (ø)
... and 35 more

@justinormont

justinormont commented Jan 21, 2020

Copy link
Copy Markdown
Contributor

Closing/reopening to restart CI tests.

CI is failing w/ an odd Git checkout issue:

2020-01-21T17:01:18.2670411Z ##[command]git checkout --progress --force 36a6ca3
2020-01-21T17:01:18.2671965Z fatal: reference is not a tree: 36a6ca3
2020-01-21T17:01:18.2722749Z ##[error]Git checkout failed with exit code: 128

https://dev.azure.com/dnceng/public/_build/results?buildId=477842&view=logs&j=2302d3fe-129a-52d2-3973-322aaec0c902&t=12a19c0e-c998-51b7-645f-93b51cf4ff66&l=291

It sounds like it its trying to read from an non-existent commit hash. Perhaps restarting the CI will get the CI to read the current commit ID.

@sharwell

sharwell commented Jan 21, 2020

Copy link
Copy Markdown
Contributor

Requesting a review from @tannergooding

@tannergooding

tannergooding commented Jan 21, 2020

Copy link
Copy Markdown
Member

This will break +/-Infinity as Invariant Culture is Infinity while other cultures may use things like Inf or ∞ (the latter is the default for en-us).

It is also likely to break users who expect or depend on the invariant culture working (a culture which uses , rather than . but gets 0.1 as the input may fail to parse or give an unexpected result).

I would recommend just updating the ToString call to also print using InvariantCulture as @justinormont suggested here: #4635 (comment)

@tannergooding

tannergooding commented Jan 21, 2020

Copy link
Copy Markdown
Member

Ah, nevermind. I see that the title no longer matches the change being provided. The actual change is updating ToString which should work as expected. 👍

@LittleLittleCloudLittleLittleCloud changed the title remove culture infoUsing invariance culture in toStringJan 21, 2020
@LittleLittleCloudLittleLittleCloud changed the title Using invariance culture in toStringUsing invariance culture when converting to stringJan 21, 2020
@justinormont

justinormont commented Jan 21, 2020

Copy link
Copy Markdown
Contributor

For the required MachineLearning-CI, one unit test is failing (randomly). The current unstable one is a TensorFlow test (TensorFlowImageClassificationDefault):

2020-01-21T18:41:12.4604444Z Starting test: Microsoft.ML.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassificationDefault
2020-01-21T18:41:13.0889399Z X Microsoft.ML.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassificationWithPolynomialLRScheduling [17s 474ms]
2020-01-21T18:41:13.3391570Z Error Message:
2020-01-21T18:41:13.3878708Z System.Runtime.InteropServices.SEHException : External component has thrown an exception.
2020-01-21T18:41:13.3881327Z Stack Trace:
2020-01-21T18:41:13.4477236Z at Tensorflow.c_api.TF_SessionRun(IntPtr session, TF_Buffer* run_options, TF_Output[] inputs, IntPtr[] input_values, Int32 ninputs, TF_Output[] outputs, IntPtr[] output_values, Int32 noutputs, IntPtr[] target_opers, Int32 ntargets, IntPtr run_metadata, IntPtr status)
2020-01-21T18:41:13.4477710Z at Microsoft.ML.TensorFlow.TensorFlowUtils.Runner.Run() in D:\a\1\s\src\Microsoft.ML.TensorFlow\TensorflowUtils.cs:line 491
2020-01-21T18:41:13.4478074Z at Microsoft.ML.Vision.ImageClassificationTrainer.TrainAndEvaluateClassificationLayerCore(Int32 epoch, Single learningRate, Int32 featureFileStartOffset, ImageClassificationMetrics metrics, Int64[] labelTensorShape, Int64[] featureTensorShape, Int32 batchSize, Stream trainSetLabelReader, Stream trainSetFeatureReader, Byte[] labelBufferBytes, Byte[] featuresBufferBytes, Int32 labelBufferSizeInBytes, Int32 featureBufferSizeInBytes, Int32 featureFileRecordSize, LearningRateScheduler learningRateScheduler, DnnTrainState trainState, Runner runner, IntPtr featureBufferPtr, IntPtr labelBufferPtr, Action`2 metricsAggregator) in D:\a\1\s\src\Microsoft.ML.Vision\ImageClassificationTrainer.cs:line 1115
2020-01-21T18:41:13.4478827Z at Microsoft.ML.Vision.ImageClassificationTrainer.TrainAndEvaluateClassificationLayer(String trainBottleneckFilePath, String validationSetBottleneckFilePath) in D:\a\1\s\src\Microsoft.ML.Vision\ImageClassificationTrainer.cs:line 999
2020-01-21T18:41:13.4479684Z at Microsoft.ML.Vision.ImageClassificationTrainer.TrainModelCore(TrainContext trainContext) in D:\a\1\s\src\Microsoft.ML.Vision\ImageClassificationTrainer.cs:line 710
2020-01-21T18:41:13.4480104Z at Microsoft.ML.Trainers.TrainerEstimatorBase`2.TrainTransformer(IDataView trainSet, IDataView validationSet, IPredictor initPredictor) in D:\a\1\s\src\Microsoft.ML.Data\Training\TrainerEstimatorBase.cs:line 157
2020-01-21T18:41:13.4480343Z at Microsoft.ML.Trainers.TrainerEstimatorBase`2.Fit(IDataView input) in D:\a\1\s\src\Microsoft.ML.Data\Training\TrainerEstimatorBase.cs:line 77
2020-01-21T18:41:13.4480554Z at Microsoft.ML.Data.EstimatorChain`1.Fit(IDataView input) in D:\a\1\s\src\Microsoft.ML.Data\DataLoadSave\EstimatorChain.cs:line 67
2020-01-21T18:41:13.4480901Z at Microsoft.ML.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassificationWithLRScheduling(LearningRateScheduler learningRateScheduler, Int32 epoch) in D:\a\1\s\test\Microsoft.ML.Tests\ScenariosWithDirectInstantiation\TensorflowTests.cs:line 1544
2020-01-21T18:41:13.4481204Z at Microsoft.ML.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassificationWithPolynomialLRScheduling() in D:\a\1\s\test\Microsoft.ML.Tests\ScenariosWithDirectInstantiation\TensorflowTests.cs:line 1474

https://dev.azure.com/dnceng/public/_build/results?buildId=491199&view=logs&j=dd8eddb6-ecc6-5f65-73e6-df90e5693b94

It's only failing under Windows_x64_NetCoreApp21 Debug_Build, so it's unlikely to be deterministic.

I'll rerun the failing CI leg and hopefully the randomness falls in our favor. The previous Git issue was deterministic and restarting the CI seems to cause it to pick up the right commit ID.

@LittleLittleCloud
LittleLittleCloud merged commit fa75f4f into dotnet:masterJan 21, 2020
@LittleLittleCloudLittleLittleCloud mentioned this pull request Feb 6, 2020
Merged
@justinormontjustinormont mentioned this pull request Mar 3, 2020
Closed
This was referenced May 25, 2020
Closed
Merged
@justinormontjustinormont mentioned this pull request Jul 23, 2020
Closed
@ghostghost locked as resolved and limited conversation to collaborators Mar 19, 2022
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

@tannergoodingtannergoodingAwaiting requested review from tannergooding
1 more reviewer
@justinormontjustinormontjustinormont approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

AutoML.NETAutomating various steps of the machine learning process

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@LittleLittleCloud@justinormont@sharwell@tannergooding
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all 
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '

Skip to content
Using invariance culture when converting to string - #4635
Merged
LittleLittleCloud merged 2 commits into
dotnet:masterfrom 
LittleLittleCloud:u/xiaoyun/removeCultureInfo
Jan 21, 2020
Merged
Using invariance culture when converting to string#4635
LittleLittleCloud merged 2 commits into
dotnet:masterfrom 
LittleLittleCloud:u/xiaoyun/removeCultureInfo
Conversation
@LittleLittleCloud
@LittleLittleCloudLittleLittleCloud
commented
Jan 7, 2020

Copy link


Copy Markdown

Member
@LittleLittleCloud
LittleLittleCloud
requested a review
from a team
as a code ownerJanuary 7, 2020 22:38
@LittleLittleCloudLittleLittleCloud
added
the 
AutoML.NET
Automating various steps of the machine learning process
label
Jan 7, 2020
justinormont
justinormont
reviewed
Jan 7, 2020
View reviewed changes
Comment threadsrc/Microsoft.ML.AutoML/Utils/SweepableParamAttributes.cs
Outdated

 public override void SetUsingValueText(string valueText)
 {
 RawValue = float.Parse(valueText, CultureInfo.InvariantCulture);
 RawValue = float.Parse(valueText);
@justinormontjustinormontJan 7, 2020

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would instead push the culture invariant value into the sweepable parameter:
machinelearning/src/Microsoft.ML.Sweeper/Parameters.cs

Line 140
in
b861b5d
_valueText=_value.ToString("R");
@codecov
codecovBot
commented
Jan 9, 2020•

edited

Loading

Copy link


Copy Markdown

Codecov Report
Merging #4635 into master will increase coverage by 0.39%.

The diff coverage is 100%.
@@ Coverage Diff @@## master #4635 +/- ##
==========================================
+ Coverage 75.63% 76.02% +0.39% 
==========================================
Files 938 951 +13 Lines 168669 173945 +5276 Branches 18217 18885 +668 ==========================================
+ Hits 127574 132247 +4673 - Misses 36065 36541 +476 - Partials 5030 5157 +127
FlagCoverage Δ
#Debug76.02% <100%> (+0.39%)⬆️
#production71.48% <100%> (+0.24%)⬆️
#test90.78% <ø> (+0.33%)⬆️
Impacted FilesCoverage Δ
src/Microsoft.ML.AutoML/Sweepers/Parameters.cs84.32% <100%> (ø)⬆️
src/Microsoft.ML.Data/Utilities/SlotDropper.cs96.5% <0%> (-2.75%)⬇️
...rc/Microsoft.ML.Featurizers/DateTimeTransformer.cs89.42% <0%> (-1.16%)⬇️
...s/Api/CookbookSamples/CookbookSamplesDynamicApi.cs95.28% <0%> (-0.2%)⬇️
...ML.Transforms/Text/StopWordsRemovingTransformer.cs86.1% <0%> (-0.16%)⬇️
...Microsoft.ML.Tests/Transformers/NormalizerTests.cs100% <0%> (ø)⬆️
...soft.ML.Tests/PermutationFeatureImportanceTests.cs100% <0%> (ø)⬆️
...ft.ML.Tests/Transformers/TimeSeriesImputerTests.cs100% <0%> (ø)
src/Microsoft.ML.Featurizers/TimeSeriesImputer.cs87.08% <0%> (ø)
....ML.Tests/Transformers/ToStringTransformerTests.cs100% <0%> (ø)
... and 35 more
@justinormont
justinormont
commented
Jan 21, 2020

Copy link


Copy Markdown

Contributor
Closing/reopening to restart CI tests.
CI is failing w/ an odd Git checkout issue:
2020-01-21T17:01:18.2670411Z ##[command]git checkout --progress --force 36a6ca3

2020-01-21T17:01:18.2671965Z fatal: reference is not a tree: 36a6ca3

2020-01-21T17:01:18.2722749Z ##[error]Git checkout failed with exit code: 128
https://dev.azure.com/dnceng/public/_build/results?buildId=477842&view=logs&j=2302d3fe-129a-52d2-3973-322aaec0c902&t=12a19c0e-c998-51b7-645f-93b51cf4ff66&l=291
It sounds like it its trying to read from an non-existent commit hash. Perhaps restarting the CI will get the CI to read the current commit ID.
@sharwell
sharwell
commented
Jan 21, 2020

Copy link


Copy Markdown

Contributor
Requesting a review from @tannergooding
@tannergooding
tannergooding
commented
Jan 21, 2020

Copy link


Copy Markdown

Member
This will break +/-Infinity as Invariant Culture is Infinity while other cultures may use things like Inf or ∞ (the latter is the default for en-us).
It is also likely to break users who expect or depend on the invariant culture working (a culture which uses , rather than . but gets 0.1 as the input may fail to parse or give an unexpected result).
I would recommend just updating the ToString call to also print using InvariantCulture as @justinormont suggested here: #4635 (comment)
@tannergooding
tannergooding
commented
Jan 21, 2020

Copy link


Copy Markdown

Member
Ah, nevermind. I see that the title no longer matches the change being provided. The actual change is updating ToString which should work as expected. 👍
@LittleLittleCloudLittleLittleCloud
changed the title
remove culture infoUsing invariance culture in toStringJan 21, 2020
@LittleLittleCloudLittleLittleCloud
changed the title
Using invariance culture in toStringUsing invariance culture when converting to stringJan 21, 2020
@justinormont
justinormont
commented
Jan 21, 2020

Copy link


Copy Markdown

Contributor
For the required MachineLearning-CI, one unit test is failing (randomly). The current unstable one is a TensorFlow test (TensorFlowImageClassificationDefault):
2020-01-21T18:41:12.4604444Z Starting test: Microsoft.ML.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassificationDefault
2020-01-21T18:41:13.0889399Z X Microsoft.ML.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassificationWithPolynomialLRScheduling [17s 474ms]
2020-01-21T18:41:13.3391570Z Error Message:
2020-01-21T18:41:13.3878708Z System.Runtime.InteropServices.SEHException : External component has thrown an exception.
2020-01-21T18:41:13.3881327Z Stack Trace:
2020-01-21T18:41:13.4477236Z at Tensorflow.c_api.TF_SessionRun(IntPtr session, TF_Buffer* run_options, TF_Output[] inputs, IntPtr[] input_values, Int32 ninputs, TF_Output[] outputs, IntPtr[] output_values, Int32 noutputs, IntPtr[] target_opers, Int32 ntargets, IntPtr run_metadata, IntPtr status)
2020-01-21T18:41:13.4477710Z at Microsoft.ML.TensorFlow.TensorFlowUtils.Runner.Run() in D:\a\1\s\src\Microsoft.ML.TensorFlow\TensorflowUtils.cs:line 491
2020-01-21T18:41:13.4478074Z at Microsoft.ML.Vision.ImageClassificationTrainer.TrainAndEvaluateClassificationLayerCore(Int32 epoch, Single learningRate, Int32 featureFileStartOffset, ImageClassificationMetrics metrics, Int64[] labelTensorShape, Int64[] featureTensorShape, Int32 batchSize, Stream trainSetLabelReader, Stream trainSetFeatureReader, Byte[] labelBufferBytes, Byte[] featuresBufferBytes, Int32 labelBufferSizeInBytes, Int32 featureBufferSizeInBytes, Int32 featureFileRecordSize, LearningRateScheduler learningRateScheduler, DnnTrainState trainState, Runner runner, IntPtr featureBufferPtr, IntPtr labelBufferPtr, Action`2 metricsAggregator) in D:\a\1\s\src\Microsoft.ML.Vision\ImageClassificationTrainer.cs:line 1115
2020-01-21T18:41:13.4478827Z at Microsoft.ML.Vision.ImageClassificationTrainer.TrainAndEvaluateClassificationLayer(String trainBottleneckFilePath, String validationSetBottleneckFilePath) in D:\a\1\s\src\Microsoft.ML.Vision\ImageClassificationTrainer.cs:line 999
2020-01-21T18:41:13.4479684Z at Microsoft.ML.Vision.ImageClassificationTrainer.TrainModelCore(TrainContext trainContext) in D:\a\1\s\src\Microsoft.ML.Vision\ImageClassificationTrainer.cs:line 710
2020-01-21T18:41:13.4480104Z at Microsoft.ML.Trainers.TrainerEstimatorBase`2.TrainTransformer(IDataView trainSet, IDataView validationSet, IPredictor initPredictor) in D:\a\1\s\src\Microsoft.ML.Data\Training\TrainerEstimatorBase.cs:line 157
2020-01-21T18:41:13.4480343Z at Microsoft.ML.Trainers.TrainerEstimatorBase`2.Fit(IDataView input) in D:\a\1\s\src\Microsoft.ML.Data\Training\TrainerEstimatorBase.cs:line 77
2020-01-21T18:41:13.4480554Z at Microsoft.ML.Data.EstimatorChain`1.Fit(IDataView input) in D:\a\1\s\src\Microsoft.ML.Data\DataLoadSave\EstimatorChain.cs:line 67
2020-01-21T18:41:13.4480901Z at Microsoft.ML.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassificationWithLRScheduling(LearningRateScheduler learningRateScheduler, Int32 epoch) in D:\a\1\s\test\Microsoft.ML.Tests\ScenariosWithDirectInstantiation\TensorflowTests.cs:line 1544
2020-01-21T18:41:13.4481204Z at Microsoft.ML.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassificationWithPolynomialLRScheduling() in D:\a\1\s\test\Microsoft.ML.Tests\ScenariosWithDirectInstantiation\TensorflowTests.cs:line 1474

https://dev.azure.com/dnceng/public/_build/results?buildId=491199&view=logs&j=dd8eddb6-ecc6-5f65-73e6-df90e5693b94
It's only failing under Windows_x64_NetCoreApp21 Debug_Build, so it's unlikely to be deterministic.
I'll rerun the failing CI leg and hopefully the randomness falls in our favor. The previous Git issue was deterministic and restarting the CI seems to cause it to pick up the right commit ID.
@LittleLittleCloud
LittleLittleCloud
merged commit fa75f4f
into
dotnet:masterJan 21, 2020
@LittleLittleCloudLittleLittleCloud
mentioned this pull request
Feb 6, 2020
 Merged

@justinormontjustinormont
mentioned this pull request
Mar 3, 2020

Closed


This was referenced May 25, 2020

Closed

 Merged

@justinormontjustinormont
mentioned this pull request
Jul 23, 2020

Closed

@ghostghost
locked as resolved and limited conversation to collaborators
Mar 19, 2022
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@tannergoodingtannergoodingAwaiting requested review from tannergooding

1 more reviewer

@justinormontjustinormontjustinormont approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees


No one assigned


Labels

AutoML.NETAutomating various steps of the machine learning process

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

4 participants

@LittleLittleCloud@justinormont@sharwell@tannergooding

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix");
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Using invariance culture when converting to string - #4635
Merged
LittleLittleCloud merged 2 commits into
dotnet:masterfrom 
LittleLittleCloud:u/xiaoyun/removeCultureInfo
Jan 21, 2020
Merged
Using invariance culture when converting to string#4635
LittleLittleCloud merged 2 commits into
dotnet:masterfrom 
LittleLittleCloud:u/xiaoyun/removeCultureInfo
Conversation
@LittleLittleCloud
@LittleLittleCloudLittleLittleCloud
commented
Jan 7, 2020

Copy link


Copy Markdown

Member
@LittleLittleCloud
LittleLittleCloud
requested a review
from a team
as a code ownerJanuary 7, 2020 22:38
@LittleLittleCloudLittleLittleCloud
added
the 
AutoML.NET
Automating various steps of the machine learning process
label
Jan 7, 2020
justinormont
justinormont
reviewed
Jan 7, 2020
View reviewed changes
Comment threadsrc/Microsoft.ML.AutoML/Utils/SweepableParamAttributes.cs
Outdated

 public override void SetUsingValueText(string valueText)
 {
 RawValue = float.Parse(valueText, CultureInfo.InvariantCulture);
 RawValue = float.Parse(valueText);
@justinormontjustinormontJan 7, 2020

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would instead push the culture invariant value into the sweepable parameter:
machinelearning/src/Microsoft.ML.Sweeper/Parameters.cs

Line 140
in
b861b5d
_valueText=_value.ToString("R");
@codecov
codecovBot
commented
Jan 9, 2020•

edited

Loading

Copy link


Copy Markdown

Codecov Report
Merging #4635 into master will increase coverage by 0.39%.

The diff coverage is 100%.
@@ Coverage Diff @@## master #4635 +/- ##
==========================================
+ Coverage 75.63% 76.02% +0.39% 
==========================================
Files 938 951 +13 Lines 168669 173945 +5276 Branches 18217 18885 +668 ==========================================
+ Hits 127574 132247 +4673 - Misses 36065 36541 +476 - Partials 5030 5157 +127
FlagCoverage Δ
#Debug76.02% <100%> (+0.39%)⬆️
#production71.48% <100%> (+0.24%)⬆️
#test90.78% <ø> (+0.33%)⬆️
Impacted FilesCoverage Δ
src/Microsoft.ML.AutoML/Sweepers/Parameters.cs84.32% <100%> (ø)⬆️
src/Microsoft.ML.Data/Utilities/SlotDropper.cs96.5% <0%> (-2.75%)⬇️
...rc/Microsoft.ML.Featurizers/DateTimeTransformer.cs89.42% <0%> (-1.16%)⬇️
...s/Api/CookbookSamples/CookbookSamplesDynamicApi.cs95.28% <0%> (-0.2%)⬇️
...ML.Transforms/Text/StopWordsRemovingTransformer.cs86.1% <0%> (-0.16%)⬇️
...Microsoft.ML.Tests/Transformers/NormalizerTests.cs100% <0%> (ø)⬆️
...soft.ML.Tests/PermutationFeatureImportanceTests.cs100% <0%> (ø)⬆️
...ft.ML.Tests/Transformers/TimeSeriesImputerTests.cs100% <0%> (ø)
src/Microsoft.ML.Featurizers/TimeSeriesImputer.cs87.08% <0%> (ø)
....ML.Tests/Transformers/ToStringTransformerTests.cs100% <0%> (ø)
... and 35 more
@justinormont
justinormont
commented
Jan 21, 2020

Copy link


Copy Markdown

Contributor
Closing/reopening to restart CI tests.
CI is failing w/ an odd Git checkout issue:
2020-01-21T17:01:18.2670411Z ##[command]git checkout --progress --force 36a6ca3

2020-01-21T17:01:18.2671965Z fatal: reference is not a tree: 36a6ca3

2020-01-21T17:01:18.2722749Z ##[error]Git checkout failed with exit code: 128
https://dev.azure.com/dnceng/public/_build/results?buildId=477842&view=logs&j=2302d3fe-129a-52d2-3973-322aaec0c902&t=12a19c0e-c998-51b7-645f-93b51cf4ff66&l=291
It sounds like it its trying to read from an non-existent commit hash. Perhaps restarting the CI will get the CI to read the current commit ID.
@sharwell
sharwell
commented
Jan 21, 2020

Copy link


Copy Markdown

Contributor
Requesting a review from @tannergooding
@tannergooding
tannergooding
commented
Jan 21, 2020

Copy link


Copy Markdown

Member
This will break +/-Infinity as Invariant Culture is Infinity while other cultures may use things like Inf or ∞ (the latter is the default for en-us).
It is also likely to break users who expect or depend on the invariant culture working (a culture which uses , rather than . but gets 0.1 as the input may fail to parse or give an unexpected result).
I would recommend just updating the ToString call to also print using InvariantCulture as @justinormont suggested here: #4635 (comment)
@tannergooding
tannergooding
commented
Jan 21, 2020

Copy link


Copy Markdown

Member
Ah, nevermind. I see that the title no longer matches the change being provided. The actual change is updating ToString which should work as expected. 👍
@LittleLittleCloudLittleLittleCloud
changed the title
remove culture infoUsing invariance culture in toStringJan 21, 2020
@LittleLittleCloudLittleLittleCloud
changed the title
Using invariance culture in toStringUsing invariance culture when converting to stringJan 21, 2020
@justinormont
justinormont
commented
Jan 21, 2020

Copy link


Copy Markdown

Contributor
For the required MachineLearning-CI, one unit test is failing (randomly). The current unstable one is a TensorFlow test (TensorFlowImageClassificationDefault):
2020-01-21T18:41:12.4604444Z Starting test: Microsoft.ML.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassificationDefault
2020-01-21T18:41:13.0889399Z X Microsoft.ML.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassificationWithPolynomialLRScheduling [17s 474ms]
2020-01-21T18:41:13.3391570Z Error Message:
2020-01-21T18:41:13.3878708Z System.Runtime.InteropServices.SEHException : External component has thrown an exception.
2020-01-21T18:41:13.3881327Z Stack Trace:
2020-01-21T18:41:13.4477236Z at Tensorflow.c_api.TF_SessionRun(IntPtr session, TF_Buffer* run_options, TF_Output[] inputs, IntPtr[] input_values, Int32 ninputs, TF_Output[] outputs, IntPtr[] output_values, Int32 noutputs, IntPtr[] target_opers, Int32 ntargets, IntPtr run_metadata, IntPtr status)
2020-01-21T18:41:13.4477710Z at Microsoft.ML.TensorFlow.TensorFlowUtils.Runner.Run() in D:\a\1\s\src\Microsoft.ML.TensorFlow\TensorflowUtils.cs:line 491
2020-01-21T18:41:13.4478074Z at Microsoft.ML.Vision.ImageClassificationTrainer.TrainAndEvaluateClassificationLayerCore(Int32 epoch, Single learningRate, Int32 featureFileStartOffset, ImageClassificationMetrics metrics, Int64[] labelTensorShape, Int64[] featureTensorShape, Int32 batchSize, Stream trainSetLabelReader, Stream trainSetFeatureReader, Byte[] labelBufferBytes, Byte[] featuresBufferBytes, Int32 labelBufferSizeInBytes, Int32 featureBufferSizeInBytes, Int32 featureFileRecordSize, LearningRateScheduler learningRateScheduler, DnnTrainState trainState, Runner runner, IntPtr featureBufferPtr, IntPtr labelBufferPtr, Action`2 metricsAggregator) in D:\a\1\s\src\Microsoft.ML.Vision\ImageClassificationTrainer.cs:line 1115
2020-01-21T18:41:13.4478827Z at Microsoft.ML.Vision.ImageClassificationTrainer.TrainAndEvaluateClassificationLayer(String trainBottleneckFilePath, String validationSetBottleneckFilePath) in D:\a\1\s\src\Microsoft.ML.Vision\ImageClassificationTrainer.cs:line 999
2020-01-21T18:41:13.4479684Z at Microsoft.ML.Vision.ImageClassificationTrainer.TrainModelCore(TrainContext trainContext) in D:\a\1\s\src\Microsoft.ML.Vision\ImageClassificationTrainer.cs:line 710
2020-01-21T18:41:13.4480104Z at Microsoft.ML.Trainers.TrainerEstimatorBase`2.TrainTransformer(IDataView trainSet, IDataView validationSet, IPredictor initPredictor) in D:\a\1\s\src\Microsoft.ML.Data\Training\TrainerEstimatorBase.cs:line 157
2020-01-21T18:41:13.4480343Z at Microsoft.ML.Trainers.TrainerEstimatorBase`2.Fit(IDataView input) in D:\a\1\s\src\Microsoft.ML.Data\Training\TrainerEstimatorBase.cs:line 77
2020-01-21T18:41:13.4480554Z at Microsoft.ML.Data.EstimatorChain`1.Fit(IDataView input) in D:\a\1\s\src\Microsoft.ML.Data\DataLoadSave\EstimatorChain.cs:line 67
2020-01-21T18:41:13.4480901Z at Microsoft.ML.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassificationWithLRScheduling(LearningRateScheduler learningRateScheduler, Int32 epoch) in D:\a\1\s\test\Microsoft.ML.Tests\ScenariosWithDirectInstantiation\TensorflowTests.cs:line 1544
2020-01-21T18:41:13.4481204Z at Microsoft.ML.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassificationWithPolynomialLRScheduling() in D:\a\1\s\test\Microsoft.ML.Tests\ScenariosWithDirectInstantiation\TensorflowTests.cs:line 1474

https://dev.azure.com/dnceng/public/_build/results?buildId=491199&view=logs&j=dd8eddb6-ecc6-5f65-73e6-df90e5693b94
It's only failing under Windows_x64_NetCoreApp21 Debug_Build, so it's unlikely to be deterministic.
I'll rerun the failing CI leg and hopefully the randomness falls in our favor. The previous Git issue was deterministic and restarting the CI seems to cause it to pick up the right commit ID.
@LittleLittleCloud
LittleLittleCloud
merged commit fa75f4f
into
dotnet:masterJan 21, 2020
@LittleLittleCloudLittleLittleCloud
mentioned this pull request
Feb 6, 2020
 Merged

@justinormontjustinormont
mentioned this pull request
Mar 3, 2020

Closed


This was referenced May 25, 2020

Closed

 Merged

@justinormontjustinormont
mentioned this pull request
Jul 23, 2020

Closed

@ghostghost
locked as resolved and limited conversation to collaborators
Mar 19, 2022
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@tannergoodingtannergoodingAwaiting requested review from tannergooding

1 more reviewer

@justinormontjustinormontjustinormont approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees


No one assigned


Labels

AutoML.NETAutomating various steps of the machine learning process

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

4 participants

@LittleLittleCloud@justinormont@sharwell@tannergooding

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms");
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Using invariance culture when converting to string - #4635
Merged
LittleLittleCloud merged 2 commits into
dotnet:masterfrom 
LittleLittleCloud:u/xiaoyun/removeCultureInfo
Jan 21, 2020
Merged
Using invariance culture when converting to string#4635
LittleLittleCloud merged 2 commits into
dotnet:masterfrom 
LittleLittleCloud:u/xiaoyun/removeCultureInfo
Conversation
@LittleLittleCloud
@LittleLittleCloudLittleLittleCloud
commented
Jan 7, 2020

Copy link


Copy Markdown

Member
@LittleLittleCloud
LittleLittleCloud
requested a review
from a team
as a code ownerJanuary 7, 2020 22:38
@LittleLittleCloudLittleLittleCloud
added
the 
AutoML.NET
Automating various steps of the machine learning process
label
Jan 7, 2020
justinormont
justinormont
reviewed
Jan 7, 2020
View reviewed changes
Comment threadsrc/Microsoft.ML.AutoML/Utils/SweepableParamAttributes.cs
Outdated

 public override void SetUsingValueText(string valueText)
 {
 RawValue = float.Parse(valueText, CultureInfo.InvariantCulture);
 RawValue = float.Parse(valueText);
@justinormontjustinormontJan 7, 2020

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would instead push the culture invariant value into the sweepable parameter:
machinelearning/src/Microsoft.ML.Sweeper/Parameters.cs

Line 140
in
b861b5d
_valueText=_value.ToString("R");
@codecov
codecovBot
commented
Jan 9, 2020•

edited

Loading

Copy link


Copy Markdown

Codecov Report
Merging #4635 into master will increase coverage by 0.39%.

The diff coverage is 100%.
@@ Coverage Diff @@## master #4635 +/- ##
==========================================
+ Coverage 75.63% 76.02% +0.39% 
==========================================
Files 938 951 +13 Lines 168669 173945 +5276 Branches 18217 18885 +668 ==========================================
+ Hits 127574 132247 +4673 - Misses 36065 36541 +476 - Partials 5030 5157 +127
FlagCoverage Δ
#Debug76.02% <100%> (+0.39%)⬆️
#production71.48% <100%> (+0.24%)⬆️
#test90.78% <ø> (+0.33%)⬆️
Impacted FilesCoverage Δ
src/Microsoft.ML.AutoML/Sweepers/Parameters.cs84.32% <100%> (ø)⬆️
src/Microsoft.ML.Data/Utilities/SlotDropper.cs96.5% <0%> (-2.75%)⬇️
...rc/Microsoft.ML.Featurizers/DateTimeTransformer.cs89.42% <0%> (-1.16%)⬇️
...s/Api/CookbookSamples/CookbookSamplesDynamicApi.cs95.28% <0%> (-0.2%)⬇️
...ML.Transforms/Text/StopWordsRemovingTransformer.cs86.1% <0%> (-0.16%)⬇️
...Microsoft.ML.Tests/Transformers/NormalizerTests.cs100% <0%> (ø)⬆️
...soft.ML.Tests/PermutationFeatureImportanceTests.cs100% <0%> (ø)⬆️
...ft.ML.Tests/Transformers/TimeSeriesImputerTests.cs100% <0%> (ø)
src/Microsoft.ML.Featurizers/TimeSeriesImputer.cs87.08% <0%> (ø)
....ML.Tests/Transformers/ToStringTransformerTests.cs100% <0%> (ø)
... and 35 more
@justinormont
justinormont
commented
Jan 21, 2020

Copy link


Copy Markdown

Contributor
Closing/reopening to restart CI tests.
CI is failing w/ an odd Git checkout issue:
2020-01-21T17:01:18.2670411Z ##[command]git checkout --progress --force 36a6ca3

2020-01-21T17:01:18.2671965Z fatal: reference is not a tree: 36a6ca3

2020-01-21T17:01:18.2722749Z ##[error]Git checkout failed with exit code: 128
https://dev.azure.com/dnceng/public/_build/results?buildId=477842&view=logs&j=2302d3fe-129a-52d2-3973-322aaec0c902&t=12a19c0e-c998-51b7-645f-93b51cf4ff66&l=291
It sounds like it its trying to read from an non-existent commit hash. Perhaps restarting the CI will get the CI to read the current commit ID.
@sharwell
sharwell
commented
Jan 21, 2020

Copy link


Copy Markdown

Contributor
Requesting a review from @tannergooding
@tannergooding
tannergooding
commented
Jan 21, 2020

Copy link


Copy Markdown

Member
This will break +/-Infinity as Invariant Culture is Infinity while other cultures may use things like Inf or ∞ (the latter is the default for en-us).
It is also likely to break users who expect or depend on the invariant culture working (a culture which uses , rather than . but gets 0.1 as the input may fail to parse or give an unexpected result).
I would recommend just updating the ToString call to also print using InvariantCulture as @justinormont suggested here: #4635 (comment)
@tannergooding
tannergooding
commented
Jan 21, 2020

Copy link


Copy Markdown

Member
Ah, nevermind. I see that the title no longer matches the change being provided. The actual change is updating ToString which should work as expected. 👍
@LittleLittleCloudLittleLittleCloud
changed the title
remove culture infoUsing invariance culture in toStringJan 21, 2020
@LittleLittleCloudLittleLittleCloud
changed the title
Using invariance culture in toStringUsing invariance culture when converting to stringJan 21, 2020
@justinormont
justinormont
commented
Jan 21, 2020

Copy link


Copy Markdown

Contributor
For the required MachineLearning-CI, one unit test is failing (randomly). The current unstable one is a TensorFlow test (TensorFlowImageClassificationDefault):
2020-01-21T18:41:12.4604444Z Starting test: Microsoft.ML.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassificationDefault
2020-01-21T18:41:13.0889399Z X Microsoft.ML.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassificationWithPolynomialLRScheduling [17s 474ms]
2020-01-21T18:41:13.3391570Z Error Message:
2020-01-21T18:41:13.3878708Z System.Runtime.InteropServices.SEHException : External component has thrown an exception.
2020-01-21T18:41:13.3881327Z Stack Trace:
2020-01-21T18:41:13.4477236Z at Tensorflow.c_api.TF_SessionRun(IntPtr session, TF_Buffer* run_options, TF_Output[] inputs, IntPtr[] input_values, Int32 ninputs, TF_Output[] outputs, IntPtr[] output_values, Int32 noutputs, IntPtr[] target_opers, Int32 ntargets, IntPtr run_metadata, IntPtr status)
2020-01-21T18:41:13.4477710Z at Microsoft.ML.TensorFlow.TensorFlowUtils.Runner.Run() in D:\a\1\s\src\Microsoft.ML.TensorFlow\TensorflowUtils.cs:line 491
2020-01-21T18:41:13.4478074Z at Microsoft.ML.Vision.ImageClassificationTrainer.TrainAndEvaluateClassificationLayerCore(Int32 epoch, Single learningRate, Int32 featureFileStartOffset, ImageClassificationMetrics metrics, Int64[] labelTensorShape, Int64[] featureTensorShape, Int32 batchSize, Stream trainSetLabelReader, Stream trainSetFeatureReader, Byte[] labelBufferBytes, Byte[] featuresBufferBytes, Int32 labelBufferSizeInBytes, Int32 featureBufferSizeInBytes, Int32 featureFileRecordSize, LearningRateScheduler learningRateScheduler, DnnTrainState trainState, Runner runner, IntPtr featureBufferPtr, IntPtr labelBufferPtr, Action`2 metricsAggregator) in D:\a\1\s\src\Microsoft.ML.Vision\ImageClassificationTrainer.cs:line 1115
2020-01-21T18:41:13.4478827Z at Microsoft.ML.Vision.ImageClassificationTrainer.TrainAndEvaluateClassificationLayer(String trainBottleneckFilePath, String validationSetBottleneckFilePath) in D:\a\1\s\src\Microsoft.ML.Vision\ImageClassificationTrainer.cs:line 999
2020-01-21T18:41:13.4479684Z at Microsoft.ML.Vision.ImageClassificationTrainer.TrainModelCore(TrainContext trainContext) in D:\a\1\s\src\Microsoft.ML.Vision\ImageClassificationTrainer.cs:line 710
2020-01-21T18:41:13.4480104Z at Microsoft.ML.Trainers.TrainerEstimatorBase`2.TrainTransformer(IDataView trainSet, IDataView validationSet, IPredictor initPredictor) in D:\a\1\s\src\Microsoft.ML.Data\Training\TrainerEstimatorBase.cs:line 157
2020-01-21T18:41:13.4480343Z at Microsoft.ML.Trainers.TrainerEstimatorBase`2.Fit(IDataView input) in D:\a\1\s\src\Microsoft.ML.Data\Training\TrainerEstimatorBase.cs:line 77
2020-01-21T18:41:13.4480554Z at Microsoft.ML.Data.EstimatorChain`1.Fit(IDataView input) in D:\a\1\s\src\Microsoft.ML.Data\DataLoadSave\EstimatorChain.cs:line 67
2020-01-21T18:41:13.4480901Z at Microsoft.ML.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassificationWithLRScheduling(LearningRateScheduler learningRateScheduler, Int32 epoch) in D:\a\1\s\test\Microsoft.ML.Tests\ScenariosWithDirectInstantiation\TensorflowTests.cs:line 1544
2020-01-21T18:41:13.4481204Z at Microsoft.ML.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassificationWithPolynomialLRScheduling() in D:\a\1\s\test\Microsoft.ML.Tests\ScenariosWithDirectInstantiation\TensorflowTests.cs:line 1474

https://dev.azure.com/dnceng/public/_build/results?buildId=491199&view=logs&j=dd8eddb6-ecc6-5f65-73e6-df90e5693b94
It's only failing under Windows_x64_NetCoreApp21 Debug_Build, so it's unlikely to be deterministic.
I'll rerun the failing CI leg and hopefully the randomness falls in our favor. The previous Git issue was deterministic and restarting the CI seems to cause it to pick up the right commit ID.
@LittleLittleCloud
LittleLittleCloud
merged commit fa75f4f
into
dotnet:masterJan 21, 2020
@LittleLittleCloudLittleLittleCloud
mentioned this pull request
Feb 6, 2020
 Merged

@justinormontjustinormont
mentioned this pull request
Mar 3, 2020

Closed


This was referenced May 25, 2020

Closed

 Merged

@justinormontjustinormont
mentioned this pull request
Jul 23, 2020

Closed

@ghostghost
locked as resolved and limited conversation to collaborators
Mar 19, 2022
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@tannergoodingtannergoodingAwaiting requested review from tannergooding

1 more reviewer

@justinormontjustinormontjustinormont approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees


No one assigned


Labels

AutoML.NETAutomating various steps of the machine learning process

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

4 participants

@LittleLittleCloud@justinormont@sharwell@tannergooding

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links");
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '

Skip to content
Using invariance culture when converting to string - #4635
Merged
LittleLittleCloud merged 2 commits into
dotnet:masterfrom 
LittleLittleCloud:u/xiaoyun/removeCultureInfo
Jan 21, 2020
Merged
Using invariance culture when converting to string#4635
LittleLittleCloud merged 2 commits into
dotnet:masterfrom 
LittleLittleCloud:u/xiaoyun/removeCultureInfo
Conversation
@LittleLittleCloud
@LittleLittleCloudLittleLittleCloud
commented
Jan 7, 2020

Copy link


Copy Markdown

Member
@LittleLittleCloud
LittleLittleCloud
requested a review
from a team
as a code ownerJanuary 7, 2020 22:38
@LittleLittleCloudLittleLittleCloud
added
the 
AutoML.NET
Automating various steps of the machine learning process
label
Jan 7, 2020
justinormont
justinormont
reviewed
Jan 7, 2020
View reviewed changes
Comment threadsrc/Microsoft.ML.AutoML/Utils/SweepableParamAttributes.cs
Outdated

 public override void SetUsingValueText(string valueText)
 {
 RawValue = float.Parse(valueText, CultureInfo.InvariantCulture);
 RawValue = float.Parse(valueText);
@justinormontjustinormontJan 7, 2020

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would instead push the culture invariant value into the sweepable parameter:
machinelearning/src/Microsoft.ML.Sweeper/Parameters.cs

Line 140
in
b861b5d
_valueText=_value.ToString("R");
@codecov
codecovBot
commented
Jan 9, 2020•

edited

Loading

Copy link


Copy Markdown

Codecov Report
Merging #4635 into master will increase coverage by 0.39%.

The diff coverage is 100%.
@@ Coverage Diff @@## master #4635 +/- ##
==========================================
+ Coverage 75.63% 76.02% +0.39% 
==========================================
Files 938 951 +13 Lines 168669 173945 +5276 Branches 18217 18885 +668 ==========================================
+ Hits 127574 132247 +4673 - Misses 36065 36541 +476 - Partials 5030 5157 +127
FlagCoverage Δ
#Debug76.02% <100%> (+0.39%)⬆️
#production71.48% <100%> (+0.24%)⬆️
#test90.78% <ø> (+0.33%)⬆️
Impacted FilesCoverage Δ
src/Microsoft.ML.AutoML/Sweepers/Parameters.cs84.32% <100%> (ø)⬆️
src/Microsoft.ML.Data/Utilities/SlotDropper.cs96.5% <0%> (-2.75%)⬇️
...rc/Microsoft.ML.Featurizers/DateTimeTransformer.cs89.42% <0%> (-1.16%)⬇️
...s/Api/CookbookSamples/CookbookSamplesDynamicApi.cs95.28% <0%> (-0.2%)⬇️
...ML.Transforms/Text/StopWordsRemovingTransformer.cs86.1% <0%> (-0.16%)⬇️
...Microsoft.ML.Tests/Transformers/NormalizerTests.cs100% <0%> (ø)⬆️
...soft.ML.Tests/PermutationFeatureImportanceTests.cs100% <0%> (ø)⬆️
...ft.ML.Tests/Transformers/TimeSeriesImputerTests.cs100% <0%> (ø)
src/Microsoft.ML.Featurizers/TimeSeriesImputer.cs87.08% <0%> (ø)
....ML.Tests/Transformers/ToStringTransformerTests.cs100% <0%> (ø)
... and 35 more
@justinormont
justinormont
commented
Jan 21, 2020

Copy link


Copy Markdown

Contributor
Closing/reopening to restart CI tests.
CI is failing w/ an odd Git checkout issue:
2020-01-21T17:01:18.2670411Z ##[command]git checkout --progress --force 36a6ca3

2020-01-21T17:01:18.2671965Z fatal: reference is not a tree: 36a6ca3

2020-01-21T17:01:18.2722749Z ##[error]Git checkout failed with exit code: 128
https://dev.azure.com/dnceng/public/_build/results?buildId=477842&view=logs&j=2302d3fe-129a-52d2-3973-322aaec0c902&t=12a19c0e-c998-51b7-645f-93b51cf4ff66&l=291
It sounds like it its trying to read from an non-existent commit hash. Perhaps restarting the CI will get the CI to read the current commit ID.
@sharwell
sharwell
commented
Jan 21, 2020

Copy link


Copy Markdown

Contributor
Requesting a review from @tannergooding
@tannergooding
tannergooding
commented
Jan 21, 2020

Copy link


Copy Markdown

Member
This will break +/-Infinity as Invariant Culture is Infinity while other cultures may use things like Inf or ∞ (the latter is the default for en-us).
It is also likely to break users who expect or depend on the invariant culture working (a culture which uses , rather than . but gets 0.1 as the input may fail to parse or give an unexpected result).
I would recommend just updating the ToString call to also print using InvariantCulture as @justinormont suggested here: #4635 (comment)
@tannergooding
tannergooding
commented
Jan 21, 2020

Copy link


Copy Markdown

Member
Ah, nevermind. I see that the title no longer matches the change being provided. The actual change is updating ToString which should work as expected. 👍
@LittleLittleCloudLittleLittleCloud
changed the title
remove culture infoUsing invariance culture in toStringJan 21, 2020
@LittleLittleCloudLittleLittleCloud
changed the title
Using invariance culture in toStringUsing invariance culture when converting to stringJan 21, 2020
@justinormont
justinormont
commented
Jan 21, 2020

Copy link


Copy Markdown

Contributor
For the required MachineLearning-CI, one unit test is failing (randomly). The current unstable one is a TensorFlow test (TensorFlowImageClassificationDefault):
2020-01-21T18:41:12.4604444Z Starting test: Microsoft.ML.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassificationDefault
2020-01-21T18:41:13.0889399Z X Microsoft.ML.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassificationWithPolynomialLRScheduling [17s 474ms]
2020-01-21T18:41:13.3391570Z Error Message:
2020-01-21T18:41:13.3878708Z System.Runtime.InteropServices.SEHException : External component has thrown an exception.
2020-01-21T18:41:13.3881327Z Stack Trace:
2020-01-21T18:41:13.4477236Z at Tensorflow.c_api.TF_SessionRun(IntPtr session, TF_Buffer* run_options, TF_Output[] inputs, IntPtr[] input_values, Int32 ninputs, TF_Output[] outputs, IntPtr[] output_values, Int32 noutputs, IntPtr[] target_opers, Int32 ntargets, IntPtr run_metadata, IntPtr status)
2020-01-21T18:41:13.4477710Z at Microsoft.ML.TensorFlow.TensorFlowUtils.Runner.Run() in D:\a\1\s\src\Microsoft.ML.TensorFlow\TensorflowUtils.cs:line 491
2020-01-21T18:41:13.4478074Z at Microsoft.ML.Vision.ImageClassificationTrainer.TrainAndEvaluateClassificationLayerCore(Int32 epoch, Single learningRate, Int32 featureFileStartOffset, ImageClassificationMetrics metrics, Int64[] labelTensorShape, Int64[] featureTensorShape, Int32 batchSize, Stream trainSetLabelReader, Stream trainSetFeatureReader, Byte[] labelBufferBytes, Byte[] featuresBufferBytes, Int32 labelBufferSizeInBytes, Int32 featureBufferSizeInBytes, Int32 featureFileRecordSize, LearningRateScheduler learningRateScheduler, DnnTrainState trainState, Runner runner, IntPtr featureBufferPtr, IntPtr labelBufferPtr, Action`2 metricsAggregator) in D:\a\1\s\src\Microsoft.ML.Vision\ImageClassificationTrainer.cs:line 1115
2020-01-21T18:41:13.4478827Z at Microsoft.ML.Vision.ImageClassificationTrainer.TrainAndEvaluateClassificationLayer(String trainBottleneckFilePath, String validationSetBottleneckFilePath) in D:\a\1\s\src\Microsoft.ML.Vision\ImageClassificationTrainer.cs:line 999
2020-01-21T18:41:13.4479684Z at Microsoft.ML.Vision.ImageClassificationTrainer.TrainModelCore(TrainContext trainContext) in D:\a\1\s\src\Microsoft.ML.Vision\ImageClassificationTrainer.cs:line 710
2020-01-21T18:41:13.4480104Z at Microsoft.ML.Trainers.TrainerEstimatorBase`2.TrainTransformer(IDataView trainSet, IDataView validationSet, IPredictor initPredictor) in D:\a\1\s\src\Microsoft.ML.Data\Training\TrainerEstimatorBase.cs:line 157
2020-01-21T18:41:13.4480343Z at Microsoft.ML.Trainers.TrainerEstimatorBase`2.Fit(IDataView input) in D:\a\1\s\src\Microsoft.ML.Data\Training\TrainerEstimatorBase.cs:line 77
2020-01-21T18:41:13.4480554Z at Microsoft.ML.Data.EstimatorChain`1.Fit(IDataView input) in D:\a\1\s\src\Microsoft.ML.Data\DataLoadSave\EstimatorChain.cs:line 67
2020-01-21T18:41:13.4480901Z at Microsoft.ML.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassificationWithLRScheduling(LearningRateScheduler learningRateScheduler, Int32 epoch) in D:\a\1\s\test\Microsoft.ML.Tests\ScenariosWithDirectInstantiation\TensorflowTests.cs:line 1544
2020-01-21T18:41:13.4481204Z at Microsoft.ML.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassificationWithPolynomialLRScheduling() in D:\a\1\s\test\Microsoft.ML.Tests\ScenariosWithDirectInstantiation\TensorflowTests.cs:line 1474

https://dev.azure.com/dnceng/public/_build/results?buildId=491199&view=logs&j=dd8eddb6-ecc6-5f65-73e6-df90e5693b94
It's only failing under Windows_x64_NetCoreApp21 Debug_Build, so it's unlikely to be deterministic.
I'll rerun the failing CI leg and hopefully the randomness falls in our favor. The previous Git issue was deterministic and restarting the CI seems to cause it to pick up the right commit ID.
@LittleLittleCloud
LittleLittleCloud
merged commit fa75f4f
into
dotnet:masterJan 21, 2020
@LittleLittleCloudLittleLittleCloud
mentioned this pull request
Feb 6, 2020
 Merged

@justinormontjustinormont
mentioned this pull request
Mar 3, 2020

Closed


This was referenced May 25, 2020

Closed

 Merged

@justinormontjustinormont
mentioned this pull request
Jul 23, 2020

Closed

@ghostghost
locked as resolved and limited conversation to collaborators
Mar 19, 2022
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@tannergoodingtannergoodingAwaiting requested review from tannergooding

1 more reviewer

@justinormontjustinormontjustinormont approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees


No one assigned


Labels

AutoML.NETAutomating various steps of the machine learning process

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

4 participants

@LittleLittleCloud@justinormont@sharwell@tannergooding

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default");
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Using invariance culture when converting to string - #4635
Merged
LittleLittleCloud merged 2 commits into
dotnet:masterfrom 
LittleLittleCloud:u/xiaoyun/removeCultureInfo
Jan 21, 2020
Merged
Using invariance culture when converting to string#4635
LittleLittleCloud merged 2 commits into
dotnet:masterfrom 
LittleLittleCloud:u/xiaoyun/removeCultureInfo
Conversation
@LittleLittleCloud
@LittleLittleCloudLittleLittleCloud
commented
Jan 7, 2020

Copy link


Copy Markdown

Member
@LittleLittleCloud
LittleLittleCloud
requested a review
from a team
as a code ownerJanuary 7, 2020 22:38
@LittleLittleCloudLittleLittleCloud
added
the 
AutoML.NET
Automating various steps of the machine learning process
label
Jan 7, 2020
justinormont
justinormont
reviewed
Jan 7, 2020
View reviewed changes
Comment threadsrc/Microsoft.ML.AutoML/Utils/SweepableParamAttributes.cs
Outdated

 public override void SetUsingValueText(string valueText)
 {
 RawValue = float.Parse(valueText, CultureInfo.InvariantCulture);
 RawValue = float.Parse(valueText);
@justinormontjustinormontJan 7, 2020

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would instead push the culture invariant value into the sweepable parameter:
machinelearning/src/Microsoft.ML.Sweeper/Parameters.cs

Line 140
in
b861b5d
_valueText=_value.ToString("R");
@codecov
codecovBot
commented
Jan 9, 2020•

edited

Loading

Copy link


Copy Markdown

Codecov Report
Merging #4635 into master will increase coverage by 0.39%.

The diff coverage is 100%.
@@ Coverage Diff @@## master #4635 +/- ##
==========================================
+ Coverage 75.63% 76.02% +0.39% 
==========================================
Files 938 951 +13 Lines 168669 173945 +5276 Branches 18217 18885 +668 ==========================================
+ Hits 127574 132247 +4673 - Misses 36065 36541 +476 - Partials 5030 5157 +127
FlagCoverage Δ
#Debug76.02% <100%> (+0.39%)⬆️
#production71.48% <100%> (+0.24%)⬆️
#test90.78% <ø> (+0.33%)⬆️
Impacted FilesCoverage Δ
src/Microsoft.ML.AutoML/Sweepers/Parameters.cs84.32% <100%> (ø)⬆️
src/Microsoft.ML.Data/Utilities/SlotDropper.cs96.5% <0%> (-2.75%)⬇️
...rc/Microsoft.ML.Featurizers/DateTimeTransformer.cs89.42% <0%> (-1.16%)⬇️
...s/Api/CookbookSamples/CookbookSamplesDynamicApi.cs95.28% <0%> (-0.2%)⬇️
...ML.Transforms/Text/StopWordsRemovingTransformer.cs86.1% <0%> (-0.16%)⬇️
...Microsoft.ML.Tests/Transformers/NormalizerTests.cs100% <0%> (ø)⬆️
...soft.ML.Tests/PermutationFeatureImportanceTests.cs100% <0%> (ø)⬆️
...ft.ML.Tests/Transformers/TimeSeriesImputerTests.cs100% <0%> (ø)
src/Microsoft.ML.Featurizers/TimeSeriesImputer.cs87.08% <0%> (ø)
....ML.Tests/Transformers/ToStringTransformerTests.cs100% <0%> (ø)
... and 35 more
@justinormont
justinormont
commented
Jan 21, 2020

Copy link


Copy Markdown

Contributor
Closing/reopening to restart CI tests.
CI is failing w/ an odd Git checkout issue:
2020-01-21T17:01:18.2670411Z ##[command]git checkout --progress --force 36a6ca3

2020-01-21T17:01:18.2671965Z fatal: reference is not a tree: 36a6ca3

2020-01-21T17:01:18.2722749Z ##[error]Git checkout failed with exit code: 128
https://dev.azure.com/dnceng/public/_build/results?buildId=477842&view=logs&j=2302d3fe-129a-52d2-3973-322aaec0c902&t=12a19c0e-c998-51b7-645f-93b51cf4ff66&l=291
It sounds like it its trying to read from an non-existent commit hash. Perhaps restarting the CI will get the CI to read the current commit ID.
@sharwell
sharwell
commented
Jan 21, 2020

Copy link


Copy Markdown

Contributor
Requesting a review from @tannergooding
@tannergooding
tannergooding
commented
Jan 21, 2020

Copy link


Copy Markdown

Member
This will break +/-Infinity as Invariant Culture is Infinity while other cultures may use things like Inf or ∞ (the latter is the default for en-us).
It is also likely to break users who expect or depend on the invariant culture working (a culture which uses , rather than . but gets 0.1 as the input may fail to parse or give an unexpected result).
I would recommend just updating the ToString call to also print using InvariantCulture as @justinormont suggested here: #4635 (comment)
@tannergooding
tannergooding
commented
Jan 21, 2020

Copy link


Copy Markdown

Member
Ah, nevermind. I see that the title no longer matches the change being provided. The actual change is updating ToString which should work as expected. 👍
@LittleLittleCloudLittleLittleCloud
changed the title
remove culture infoUsing invariance culture in toStringJan 21, 2020
@LittleLittleCloudLittleLittleCloud
changed the title
Using invariance culture in toStringUsing invariance culture when converting to stringJan 21, 2020
@justinormont
justinormont
commented
Jan 21, 2020

Copy link


Copy Markdown

Contributor
For the required MachineLearning-CI, one unit test is failing (randomly). The current unstable one is a TensorFlow test (TensorFlowImageClassificationDefault):
2020-01-21T18:41:12.4604444Z Starting test: Microsoft.ML.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassificationDefault
2020-01-21T18:41:13.0889399Z X Microsoft.ML.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassificationWithPolynomialLRScheduling [17s 474ms]
2020-01-21T18:41:13.3391570Z Error Message:
2020-01-21T18:41:13.3878708Z System.Runtime.InteropServices.SEHException : External component has thrown an exception.
2020-01-21T18:41:13.3881327Z Stack Trace:
2020-01-21T18:41:13.4477236Z at Tensorflow.c_api.TF_SessionRun(IntPtr session, TF_Buffer* run_options, TF_Output[] inputs, IntPtr[] input_values, Int32 ninputs, TF_Output[] outputs, IntPtr[] output_values, Int32 noutputs, IntPtr[] target_opers, Int32 ntargets, IntPtr run_metadata, IntPtr status)
2020-01-21T18:41:13.4477710Z at Microsoft.ML.TensorFlow.TensorFlowUtils.Runner.Run() in D:\a\1\s\src\Microsoft.ML.TensorFlow\TensorflowUtils.cs:line 491
2020-01-21T18:41:13.4478074Z at Microsoft.ML.Vision.ImageClassificationTrainer.TrainAndEvaluateClassificationLayerCore(Int32 epoch, Single learningRate, Int32 featureFileStartOffset, ImageClassificationMetrics metrics, Int64[] labelTensorShape, Int64[] featureTensorShape, Int32 batchSize, Stream trainSetLabelReader, Stream trainSetFeatureReader, Byte[] labelBufferBytes, Byte[] featuresBufferBytes, Int32 labelBufferSizeInBytes, Int32 featureBufferSizeInBytes, Int32 featureFileRecordSize, LearningRateScheduler learningRateScheduler, DnnTrainState trainState, Runner runner, IntPtr featureBufferPtr, IntPtr labelBufferPtr, Action`2 metricsAggregator) in D:\a\1\s\src\Microsoft.ML.Vision\ImageClassificationTrainer.cs:line 1115
2020-01-21T18:41:13.4478827Z at Microsoft.ML.Vision.ImageClassificationTrainer.TrainAndEvaluateClassificationLayer(String trainBottleneckFilePath, String validationSetBottleneckFilePath) in D:\a\1\s\src\Microsoft.ML.Vision\ImageClassificationTrainer.cs:line 999
2020-01-21T18:41:13.4479684Z at Microsoft.ML.Vision.ImageClassificationTrainer.TrainModelCore(TrainContext trainContext) in D:\a\1\s\src\Microsoft.ML.Vision\ImageClassificationTrainer.cs:line 710
2020-01-21T18:41:13.4480104Z at Microsoft.ML.Trainers.TrainerEstimatorBase`2.TrainTransformer(IDataView trainSet, IDataView validationSet, IPredictor initPredictor) in D:\a\1\s\src\Microsoft.ML.Data\Training\TrainerEstimatorBase.cs:line 157
2020-01-21T18:41:13.4480343Z at Microsoft.ML.Trainers.TrainerEstimatorBase`2.Fit(IDataView input) in D:\a\1\s\src\Microsoft.ML.Data\Training\TrainerEstimatorBase.cs:line 77
2020-01-21T18:41:13.4480554Z at Microsoft.ML.Data.EstimatorChain`1.Fit(IDataView input) in D:\a\1\s\src\Microsoft.ML.Data\DataLoadSave\EstimatorChain.cs:line 67
2020-01-21T18:41:13.4480901Z at Microsoft.ML.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassificationWithLRScheduling(LearningRateScheduler learningRateScheduler, Int32 epoch) in D:\a\1\s\test\Microsoft.ML.Tests\ScenariosWithDirectInstantiation\TensorflowTests.cs:line 1544
2020-01-21T18:41:13.4481204Z at Microsoft.ML.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassificationWithPolynomialLRScheduling() in D:\a\1\s\test\Microsoft.ML.Tests\ScenariosWithDirectInstantiation\TensorflowTests.cs:line 1474

https://dev.azure.com/dnceng/public/_build/results?buildId=491199&view=logs&j=dd8eddb6-ecc6-5f65-73e6-df90e5693b94
It's only failing under Windows_x64_NetCoreApp21 Debug_Build, so it's unlikely to be deterministic.
I'll rerun the failing CI leg and hopefully the randomness falls in our favor. The previous Git issue was deterministic and restarting the CI seems to cause it to pick up the right commit ID.
@LittleLittleCloud
LittleLittleCloud
merged commit fa75f4f
into
dotnet:masterJan 21, 2020
@LittleLittleCloudLittleLittleCloud
mentioned this pull request
Feb 6, 2020
 Merged

@justinormontjustinormont
mentioned this pull request
Mar 3, 2020

Closed


This was referenced May 25, 2020

Closed

 Merged

@justinormontjustinormont
mentioned this pull request
Jul 23, 2020

Closed

@ghostghost
locked as resolved and limited conversation to collaborators
Mar 19, 2022
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@tannergoodingtannergoodingAwaiting requested review from tannergooding

1 more reviewer

@justinormontjustinormontjustinormont approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees


No one assigned


Labels

AutoML.NETAutomating various steps of the machine learning process

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

4 participants

@LittleLittleCloud@justinormont@sharwell@tannergooding

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers");
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Using invariance culture when converting to string - #4635
Merged
LittleLittleCloud merged 2 commits into
dotnet:masterfrom 
LittleLittleCloud:u/xiaoyun/removeCultureInfo
Jan 21, 2020
Merged
Using invariance culture when converting to string#4635
LittleLittleCloud merged 2 commits into
dotnet:masterfrom 
LittleLittleCloud:u/xiaoyun/removeCultureInfo
Conversation
@LittleLittleCloud
@LittleLittleCloudLittleLittleCloud
commented
Jan 7, 2020

Copy link


Copy Markdown

Member
@LittleLittleCloud
LittleLittleCloud
requested a review
from a team
as a code ownerJanuary 7, 2020 22:38
@LittleLittleCloudLittleLittleCloud
added
the 
AutoML.NET
Automating various steps of the machine learning process
label
Jan 7, 2020
justinormont
justinormont
reviewed
Jan 7, 2020
View reviewed changes
Comment threadsrc/Microsoft.ML.AutoML/Utils/SweepableParamAttributes.cs
Outdated

 public override void SetUsingValueText(string valueText)
 {
 RawValue = float.Parse(valueText, CultureInfo.InvariantCulture);
 RawValue = float.Parse(valueText);
@justinormontjustinormontJan 7, 2020

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would instead push the culture invariant value into the sweepable parameter:
machinelearning/src/Microsoft.ML.Sweeper/Parameters.cs

Line 140
in
b861b5d
_valueText=_value.ToString("R");
@codecov
codecovBot
commented
Jan 9, 2020•

edited

Loading

Copy link


Copy Markdown

Codecov Report
Merging #4635 into master will increase coverage by 0.39%.

The diff coverage is 100%.
@@ Coverage Diff @@## master #4635 +/- ##
==========================================
+ Coverage 75.63% 76.02% +0.39% 
==========================================
Files 938 951 +13 Lines 168669 173945 +5276 Branches 18217 18885 +668 ==========================================
+ Hits 127574 132247 +4673 - Misses 36065 36541 +476 - Partials 5030 5157 +127
FlagCoverage Δ
#Debug76.02% <100%> (+0.39%)⬆️
#production71.48% <100%> (+0.24%)⬆️
#test90.78% <ø> (+0.33%)⬆️
Impacted FilesCoverage Δ
src/Microsoft.ML.AutoML/Sweepers/Parameters.cs84.32% <100%> (ø)⬆️
src/Microsoft.ML.Data/Utilities/SlotDropper.cs96.5% <0%> (-2.75%)⬇️
...rc/Microsoft.ML.Featurizers/DateTimeTransformer.cs89.42% <0%> (-1.16%)⬇️
...s/Api/CookbookSamples/CookbookSamplesDynamicApi.cs95.28% <0%> (-0.2%)⬇️
...ML.Transforms/Text/StopWordsRemovingTransformer.cs86.1% <0%> (-0.16%)⬇️
...Microsoft.ML.Tests/Transformers/NormalizerTests.cs100% <0%> (ø)⬆️
...soft.ML.Tests/PermutationFeatureImportanceTests.cs100% <0%> (ø)⬆️
...ft.ML.Tests/Transformers/TimeSeriesImputerTests.cs100% <0%> (ø)
src/Microsoft.ML.Featurizers/TimeSeriesImputer.cs87.08% <0%> (ø)
....ML.Tests/Transformers/ToStringTransformerTests.cs100% <0%> (ø)
... and 35 more
@justinormont
justinormont
commented
Jan 21, 2020

Copy link


Copy Markdown

Contributor
Closing/reopening to restart CI tests.
CI is failing w/ an odd Git checkout issue:
2020-01-21T17:01:18.2670411Z ##[command]git checkout --progress --force 36a6ca3

2020-01-21T17:01:18.2671965Z fatal: reference is not a tree: 36a6ca3

2020-01-21T17:01:18.2722749Z ##[error]Git checkout failed with exit code: 128
https://dev.azure.com/dnceng/public/_build/results?buildId=477842&view=logs&j=2302d3fe-129a-52d2-3973-322aaec0c902&t=12a19c0e-c998-51b7-645f-93b51cf4ff66&l=291
It sounds like it its trying to read from an non-existent commit hash. Perhaps restarting the CI will get the CI to read the current commit ID.
@sharwell
sharwell
commented
Jan 21, 2020

Copy link


Copy Markdown

Contributor
Requesting a review from @tannergooding
@tannergooding
tannergooding
commented
Jan 21, 2020

Copy link


Copy Markdown

Member
This will break +/-Infinity as Invariant Culture is Infinity while other cultures may use things like Inf or ∞ (the latter is the default for en-us).
It is also likely to break users who expect or depend on the invariant culture working (a culture which uses , rather than . but gets 0.1 as the input may fail to parse or give an unexpected result).
I would recommend just updating the ToString call to also print using InvariantCulture as @justinormont suggested here: #4635 (comment)
@tannergooding
tannergooding
commented
Jan 21, 2020

Copy link


Copy Markdown

Member
Ah, nevermind. I see that the title no longer matches the change being provided. The actual change is updating ToString which should work as expected. 👍
@LittleLittleCloudLittleLittleCloud
changed the title
remove culture infoUsing invariance culture in toStringJan 21, 2020
@LittleLittleCloudLittleLittleCloud
changed the title
Using invariance culture in toStringUsing invariance culture when converting to stringJan 21, 2020
@justinormont
justinormont
commented
Jan 21, 2020

Copy link


Copy Markdown

Contributor
For the required MachineLearning-CI, one unit test is failing (randomly). The current unstable one is a TensorFlow test (TensorFlowImageClassificationDefault):
2020-01-21T18:41:12.4604444Z Starting test: Microsoft.ML.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassificationDefault
2020-01-21T18:41:13.0889399Z X Microsoft.ML.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassificationWithPolynomialLRScheduling [17s 474ms]
2020-01-21T18:41:13.3391570Z Error Message:
2020-01-21T18:41:13.3878708Z System.Runtime.InteropServices.SEHException : External component has thrown an exception.
2020-01-21T18:41:13.3881327Z Stack Trace:
2020-01-21T18:41:13.4477236Z at Tensorflow.c_api.TF_SessionRun(IntPtr session, TF_Buffer* run_options, TF_Output[] inputs, IntPtr[] input_values, Int32 ninputs, TF_Output[] outputs, IntPtr[] output_values, Int32 noutputs, IntPtr[] target_opers, Int32 ntargets, IntPtr run_metadata, IntPtr status)
2020-01-21T18:41:13.4477710Z at Microsoft.ML.TensorFlow.TensorFlowUtils.Runner.Run() in D:\a\1\s\src\Microsoft.ML.TensorFlow\TensorflowUtils.cs:line 491
2020-01-21T18:41:13.4478074Z at Microsoft.ML.Vision.ImageClassificationTrainer.TrainAndEvaluateClassificationLayerCore(Int32 epoch, Single learningRate, Int32 featureFileStartOffset, ImageClassificationMetrics metrics, Int64[] labelTensorShape, Int64[] featureTensorShape, Int32 batchSize, Stream trainSetLabelReader, Stream trainSetFeatureReader, Byte[] labelBufferBytes, Byte[] featuresBufferBytes, Int32 labelBufferSizeInBytes, Int32 featureBufferSizeInBytes, Int32 featureFileRecordSize, LearningRateScheduler learningRateScheduler, DnnTrainState trainState, Runner runner, IntPtr featureBufferPtr, IntPtr labelBufferPtr, Action`2 metricsAggregator) in D:\a\1\s\src\Microsoft.ML.Vision\ImageClassificationTrainer.cs:line 1115
2020-01-21T18:41:13.4478827Z at Microsoft.ML.Vision.ImageClassificationTrainer.TrainAndEvaluateClassificationLayer(String trainBottleneckFilePath, String validationSetBottleneckFilePath) in D:\a\1\s\src\Microsoft.ML.Vision\ImageClassificationTrainer.cs:line 999
2020-01-21T18:41:13.4479684Z at Microsoft.ML.Vision.ImageClassificationTrainer.TrainModelCore(TrainContext trainContext) in D:\a\1\s\src\Microsoft.ML.Vision\ImageClassificationTrainer.cs:line 710
2020-01-21T18:41:13.4480104Z at Microsoft.ML.Trainers.TrainerEstimatorBase`2.TrainTransformer(IDataView trainSet, IDataView validationSet, IPredictor initPredictor) in D:\a\1\s\src\Microsoft.ML.Data\Training\TrainerEstimatorBase.cs:line 157
2020-01-21T18:41:13.4480343Z at Microsoft.ML.Trainers.TrainerEstimatorBase`2.Fit(IDataView input) in D:\a\1\s\src\Microsoft.ML.Data\Training\TrainerEstimatorBase.cs:line 77
2020-01-21T18:41:13.4480554Z at Microsoft.ML.Data.EstimatorChain`1.Fit(IDataView input) in D:\a\1\s\src\Microsoft.ML.Data\DataLoadSave\EstimatorChain.cs:line 67
2020-01-21T18:41:13.4480901Z at Microsoft.ML.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassificationWithLRScheduling(LearningRateScheduler learningRateScheduler, Int32 epoch) in D:\a\1\s\test\Microsoft.ML.Tests\ScenariosWithDirectInstantiation\TensorflowTests.cs:line 1544
2020-01-21T18:41:13.4481204Z at Microsoft.ML.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassificationWithPolynomialLRScheduling() in D:\a\1\s\test\Microsoft.ML.Tests\ScenariosWithDirectInstantiation\TensorflowTests.cs:line 1474

https://dev.azure.com/dnceng/public/_build/results?buildId=491199&view=logs&j=dd8eddb6-ecc6-5f65-73e6-df90e5693b94
It's only failing under Windows_x64_NetCoreApp21 Debug_Build, so it's unlikely to be deterministic.
I'll rerun the failing CI leg and hopefully the randomness falls in our favor. The previous Git issue was deterministic and restarting the CI seems to cause it to pick up the right commit ID.
@LittleLittleCloud
LittleLittleCloud
merged commit fa75f4f
into
dotnet:masterJan 21, 2020
@LittleLittleCloudLittleLittleCloud
mentioned this pull request
Feb 6, 2020
 Merged

@justinormontjustinormont
mentioned this pull request
Mar 3, 2020

Closed


This was referenced May 25, 2020

Closed

 Merged

@justinormontjustinormont
mentioned this pull request
Jul 23, 2020

Closed

@ghostghost
locked as resolved and limited conversation to collaborators
Mar 19, 2022
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@tannergoodingtannergoodingAwaiting requested review from tannergooding

1 more reviewer

@justinormontjustinormontjustinormont approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees


No one assigned


Labels

AutoML.NETAutomating various steps of the machine learning process

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

4 participants

@LittleLittleCloud@justinormont@sharwell@tannergooding

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode");
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();

Skip to content
Using invariance culture when converting to string - #4635
Merged
LittleLittleCloud merged 2 commits into
dotnet:masterfrom 
LittleLittleCloud:u/xiaoyun/removeCultureInfo
Jan 21, 2020
Merged
Using invariance culture when converting to string#4635
LittleLittleCloud merged 2 commits into
dotnet:masterfrom 
LittleLittleCloud:u/xiaoyun/removeCultureInfo
Conversation
@LittleLittleCloud
@LittleLittleCloudLittleLittleCloud
commented
Jan 7, 2020

Copy link


Copy Markdown

Member
@LittleLittleCloud
LittleLittleCloud
requested a review
from a team
as a code ownerJanuary 7, 2020 22:38
@LittleLittleCloudLittleLittleCloud
added
the 
AutoML.NET
Automating various steps of the machine learning process
label
Jan 7, 2020
justinormont
justinormont
reviewed
Jan 7, 2020
View reviewed changes
Comment threadsrc/Microsoft.ML.AutoML/Utils/SweepableParamAttributes.cs
Outdated

 public override void SetUsingValueText(string valueText)
 {
 RawValue = float.Parse(valueText, CultureInfo.InvariantCulture);
 RawValue = float.Parse(valueText);
@justinormontjustinormontJan 7, 2020

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would instead push the culture invariant value into the sweepable parameter:
machinelearning/src/Microsoft.ML.Sweeper/Parameters.cs

Line 140
in
b861b5d
_valueText=_value.ToString("R");
@codecov
codecovBot
commented
Jan 9, 2020•

edited

Loading

Copy link


Copy Markdown

Codecov Report
Merging #4635 into master will increase coverage by 0.39%.

The diff coverage is 100%.
@@ Coverage Diff @@## master #4635 +/- ##
==========================================
+ Coverage 75.63% 76.02% +0.39% 
==========================================
Files 938 951 +13 Lines 168669 173945 +5276 Branches 18217 18885 +668 ==========================================
+ Hits 127574 132247 +4673 - Misses 36065 36541 +476 - Partials 5030 5157 +127
FlagCoverage Δ
#Debug76.02% <100%> (+0.39%)⬆️
#production71.48% <100%> (+0.24%)⬆️
#test90.78% <ø> (+0.33%)⬆️
Impacted FilesCoverage Δ
src/Microsoft.ML.AutoML/Sweepers/Parameters.cs84.32% <100%> (ø)⬆️
src/Microsoft.ML.Data/Utilities/SlotDropper.cs96.5% <0%> (-2.75%)⬇️
...rc/Microsoft.ML.Featurizers/DateTimeTransformer.cs89.42% <0%> (-1.16%)⬇️
...s/Api/CookbookSamples/CookbookSamplesDynamicApi.cs95.28% <0%> (-0.2%)⬇️
...ML.Transforms/Text/StopWordsRemovingTransformer.cs86.1% <0%> (-0.16%)⬇️
...Microsoft.ML.Tests/Transformers/NormalizerTests.cs100% <0%> (ø)⬆️
...soft.ML.Tests/PermutationFeatureImportanceTests.cs100% <0%> (ø)⬆️
...ft.ML.Tests/Transformers/TimeSeriesImputerTests.cs100% <0%> (ø)
src/Microsoft.ML.Featurizers/TimeSeriesImputer.cs87.08% <0%> (ø)
....ML.Tests/Transformers/ToStringTransformerTests.cs100% <0%> (ø)
... and 35 more
@justinormont
justinormont
commented
Jan 21, 2020

Copy link


Copy Markdown

Contributor
Closing/reopening to restart CI tests.
CI is failing w/ an odd Git checkout issue:
2020-01-21T17:01:18.2670411Z ##[command]git checkout --progress --force 36a6ca3

2020-01-21T17:01:18.2671965Z fatal: reference is not a tree: 36a6ca3

2020-01-21T17:01:18.2722749Z ##[error]Git checkout failed with exit code: 128
https://dev.azure.com/dnceng/public/_build/results?buildId=477842&view=logs&j=2302d3fe-129a-52d2-3973-322aaec0c902&t=12a19c0e-c998-51b7-645f-93b51cf4ff66&l=291
It sounds like it its trying to read from an non-existent commit hash. Perhaps restarting the CI will get the CI to read the current commit ID.
@sharwell
sharwell
commented
Jan 21, 2020

Copy link


Copy Markdown

Contributor
Requesting a review from @tannergooding
@tannergooding
tannergooding
commented
Jan 21, 2020

Copy link


Copy Markdown

Member
This will break +/-Infinity as Invariant Culture is Infinity while other cultures may use things like Inf or ∞ (the latter is the default for en-us).
It is also likely to break users who expect or depend on the invariant culture working (a culture which uses , rather than . but gets 0.1 as the input may fail to parse or give an unexpected result).
I would recommend just updating the ToString call to also print using InvariantCulture as @justinormont suggested here: #4635 (comment)
@tannergooding
tannergooding
commented
Jan 21, 2020

Copy link


Copy Markdown

Member
Ah, nevermind. I see that the title no longer matches the change being provided. The actual change is updating ToString which should work as expected. 👍
@LittleLittleCloudLittleLittleCloud
changed the title
remove culture infoUsing invariance culture in toStringJan 21, 2020
@LittleLittleCloudLittleLittleCloud
changed the title
Using invariance culture in toStringUsing invariance culture when converting to stringJan 21, 2020
@justinormont
justinormont
commented
Jan 21, 2020

Copy link


Copy Markdown

Contributor
For the required MachineLearning-CI, one unit test is failing (randomly). The current unstable one is a TensorFlow test (TensorFlowImageClassificationDefault):
2020-01-21T18:41:12.4604444Z Starting test: Microsoft.ML.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassificationDefault
2020-01-21T18:41:13.0889399Z X Microsoft.ML.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassificationWithPolynomialLRScheduling [17s 474ms]
2020-01-21T18:41:13.3391570Z Error Message:
2020-01-21T18:41:13.3878708Z System.Runtime.InteropServices.SEHException : External component has thrown an exception.
2020-01-21T18:41:13.3881327Z Stack Trace:
2020-01-21T18:41:13.4477236Z at Tensorflow.c_api.TF_SessionRun(IntPtr session, TF_Buffer* run_options, TF_Output[] inputs, IntPtr[] input_values, Int32 ninputs, TF_Output[] outputs, IntPtr[] output_values, Int32 noutputs, IntPtr[] target_opers, Int32 ntargets, IntPtr run_metadata, IntPtr status)
2020-01-21T18:41:13.4477710Z at Microsoft.ML.TensorFlow.TensorFlowUtils.Runner.Run() in D:\a\1\s\src\Microsoft.ML.TensorFlow\TensorflowUtils.cs:line 491
2020-01-21T18:41:13.4478074Z at Microsoft.ML.Vision.ImageClassificationTrainer.TrainAndEvaluateClassificationLayerCore(Int32 epoch, Single learningRate, Int32 featureFileStartOffset, ImageClassificationMetrics metrics, Int64[] labelTensorShape, Int64[] featureTensorShape, Int32 batchSize, Stream trainSetLabelReader, Stream trainSetFeatureReader, Byte[] labelBufferBytes, Byte[] featuresBufferBytes, Int32 labelBufferSizeInBytes, Int32 featureBufferSizeInBytes, Int32 featureFileRecordSize, LearningRateScheduler learningRateScheduler, DnnTrainState trainState, Runner runner, IntPtr featureBufferPtr, IntPtr labelBufferPtr, Action`2 metricsAggregator) in D:\a\1\s\src\Microsoft.ML.Vision\ImageClassificationTrainer.cs:line 1115
2020-01-21T18:41:13.4478827Z at Microsoft.ML.Vision.ImageClassificationTrainer.TrainAndEvaluateClassificationLayer(String trainBottleneckFilePath, String validationSetBottleneckFilePath) in D:\a\1\s\src\Microsoft.ML.Vision\ImageClassificationTrainer.cs:line 999
2020-01-21T18:41:13.4479684Z at Microsoft.ML.Vision.ImageClassificationTrainer.TrainModelCore(TrainContext trainContext) in D:\a\1\s\src\Microsoft.ML.Vision\ImageClassificationTrainer.cs:line 710
2020-01-21T18:41:13.4480104Z at Microsoft.ML.Trainers.TrainerEstimatorBase`2.TrainTransformer(IDataView trainSet, IDataView validationSet, IPredictor initPredictor) in D:\a\1\s\src\Microsoft.ML.Data\Training\TrainerEstimatorBase.cs:line 157
2020-01-21T18:41:13.4480343Z at Microsoft.ML.Trainers.TrainerEstimatorBase`2.Fit(IDataView input) in D:\a\1\s\src\Microsoft.ML.Data\Training\TrainerEstimatorBase.cs:line 77
2020-01-21T18:41:13.4480554Z at Microsoft.ML.Data.EstimatorChain`1.Fit(IDataView input) in D:\a\1\s\src\Microsoft.ML.Data\DataLoadSave\EstimatorChain.cs:line 67
2020-01-21T18:41:13.4480901Z at Microsoft.ML.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassificationWithLRScheduling(LearningRateScheduler learningRateScheduler, Int32 epoch) in D:\a\1\s\test\Microsoft.ML.Tests\ScenariosWithDirectInstantiation\TensorflowTests.cs:line 1544
2020-01-21T18:41:13.4481204Z at Microsoft.ML.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassificationWithPolynomialLRScheduling() in D:\a\1\s\test\Microsoft.ML.Tests\ScenariosWithDirectInstantiation\TensorflowTests.cs:line 1474

https://dev.azure.com/dnceng/public/_build/results?buildId=491199&view=logs&j=dd8eddb6-ecc6-5f65-73e6-df90e5693b94
It's only failing under Windows_x64_NetCoreApp21 Debug_Build, so it's unlikely to be deterministic.
I'll rerun the failing CI leg and hopefully the randomness falls in our favor. The previous Git issue was deterministic and restarting the CI seems to cause it to pick up the right commit ID.
@LittleLittleCloud
LittleLittleCloud
merged commit fa75f4f
into
dotnet:masterJan 21, 2020
@LittleLittleCloudLittleLittleCloud
mentioned this pull request
Feb 6, 2020
 Merged

@justinormontjustinormont
mentioned this pull request
Mar 3, 2020

Closed


This was referenced May 25, 2020

Closed

 Merged

@justinormontjustinormont
mentioned this pull request
Jul 23, 2020

Closed

@ghostghost
locked as resolved and limited conversation to collaborators
Mar 19, 2022
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@tannergoodingtannergoodingAwaiting requested review from tannergooding

1 more reviewer

@justinormontjustinormontjustinormont approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees


No one assigned


Labels

AutoML.NETAutomating various steps of the machine learning process

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

4 participants

@LittleLittleCloud@justinormont@sharwell@tannergooding