Adds CodeGen to master (from features/automl) - #4365

Merged
codemzs merged 1 commit into
dotnet:masterfrom
maryamariyan:codegen-to-master
Oct 25, 2019
Merged

Adds CodeGen to master (from features/automl)#4365
codemzs merged 1 commit into
dotnet:masterfrom
maryamariyan:codegen-to-master

Conversation

@maryamariyan

@maryamariyanmaryamariyan commented Oct 22, 2019

Copy link
Copy Markdown
Contributor
  • Add src project for CodeGen
  • Add test project for CodeGen
  • Add packaging for CodeGen

cc: @eerhardt@LittleLittleCloud

@maryamariyanmaryamariyan self-assigned this Oct 22, 2019
Comment threadsrc/Microsoft.ML.CodeGen/Microsoft.ML.CodeGenerator.csproj Outdated
@maryamariyanmaryamariyan changed the title [WIP] Adds CodeGen to master (from features/automl)Adds CodeGen to master (from features/automl)Oct 22, 2019
Comment threadsrc/mlnet/mlnet.Build.props Outdated
Comment threadsrc/Microsoft.ML.CodeGen/Utils.cs Outdated
Comment threadsrc/Microsoft.ML.CodeGen/Microsoft.ML.CodeGenerator.csproj Outdated
Comment threadsrc/Microsoft.ML.CodeGen/Templates/Console/Annotation.txt Outdated
Comment threadsrc/Microsoft.ML.AutoML/Assembly.cs
@maryamariyan
maryamariyan marked this pull request as ready for review October 22, 2019 23:55
@maryamariyan
maryamariyan requested review from a team as code ownersOctober 22, 2019 23:55
Comment threadsrc/Microsoft.ML.CodeGenerator/Microsoft.ML.CodeGenerator.csproj Outdated
Comment threadsrc/Microsoft.ML.CodeGenerator/Microsoft.ML.CodeGenerator.csproj Outdated
Comment threadsrc/Microsoft.ML.CodeGenerator/Microsoft.ML.CodeGenerator.csproj Outdated
Comment threadpkg/Microsoft.ML.CodeGen/Microsoft.ML.CodeGen.nupkgproj
Comment threadpkg/Microsoft.ML.CodeGen/Microsoft.ML.CodeGen.nupkgproj
Comment threadsrc/Microsoft.ML.CodeGenerator/CodeGenerator/CSharp/CodeGenerator.cs Outdated
@eerhardt

Copy link
Copy Markdown
Member

<#@ template language="C#" #>

We should put a visibility="internal" on these, so the class that is generated is internal instead of public.

<#@ template language="C#" visibility="internal" #>

Refers to: src/Microsoft.ML.CodeGenerator/Templates/Console/ConsumeModel.tt:1 in a98fa15. [](commit_id = a98fa15, deletion_comment = False)

@maryamariyan

maryamariyan commented Oct 23, 2019

Copy link
Copy Markdown
ContributorAuthor

I'm looking at this comment: #4365 (comment)

Update: addressed in 2f894f4

Comment threadsrc/Microsoft.ML.AutoML/Assembly.cs
Comment threadsrc/Microsoft.ML.AutoML/Assembly.cs
{#>
//*****************************************************************************************
//* *
//* This is an auto-generated file by Microsoft ML.NET CLI (Command-Line Interface) tool. *

@sharwellsharwellOct 23, 2019

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ This is missing the <auto-generated/> comment, which means code style analyzer diagnostics will be reported in the file even though it is generated.

@eerhardteerhardtOct 24, 2019

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My thinking is that these code files aren't true "auto generated" files. I expect users to update them. Especially considering that we are putting hard-coded paths in some of the files, and we are making public types, that I assume most users will want to make internal. Or, if they do want them to be public - they would want to put XML docs on them.

So what is the appropriate pattern here? I think it is similar to "scaffolding" in ASP.NET. Or even "Add -> new Item" in VS. Do they put any "this was generated" comments at the top there?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For this case, the ideal header would be more like a template, with links explaining what the file is and where to find documentation of expected changes users would need to make. You would not need the <auto-generated/> comment for templates (one-time code generation).

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@maryamariyan - can you log an issue to address this? I think we should make that change separately and we will need @JakeRadMSFT@LittleLittleCloud and @rustd input as well.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Created #4387

<#+
void MB_Annotation()
{#>
// This file was auto-generated by ML.NET Model Builder.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ This is missing the <auto-generated/> comment, which means diagnostics will be reported in the file even though it is generated.

Comment threadsrc/Microsoft.ML.CodeGenerator/Templates/Console/PredictProject.cs Outdated

@eerhardteerhardt left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this change is ready to be merged.

We can address @sharwell's concern in a follow-up change. But let's not let that block us from moving this code from features/automl into the master branch. It is an existing issue in the code, and there isn't a clear direction we should take on it right now.

@codecov

codecovBot commented Oct 24, 2019

Copy link
Copy Markdown

Codecov Report

Merging #4365 into master will increase coverage by <.01%.
The diff coverage is 74.36%.

@@ Coverage Diff @@## master #4365 +/- ##
==========================================
+ Coverage 74.65% 74.65% +<.01% 
==========================================
Files 883 903 +20 Lines 155117 158562 +3445 Branches 16931 17136 +205 ==========================================
+ Hits 115795 118378 +2583 - Misses 34572 35362 +790 - Partials 4750 4822 +72
FlagCoverage Δ
#Debug74.65% <74.36%> (ø)⬆️
#production70.11% <64.23%> (-0.11%)⬇️
#test89.84% <100%> (+0.28%)⬆️
Impacted FilesCoverage Δ
...tor/CodeGenerator/CSharp/TransformGeneratorBase.cs100% <100%> (ø)
...r.Tests/ApprovalTests/ConsoleCodeGeneratorTests.cs100% <100%> (ø)
....ML.CodeGenerator.Tests/TransformGeneratorTests.cs100% <100%> (ø)
...t/Microsoft.ML.CodeGenerator.Tests/CodeGenTests.cs100% <100%> (ø)
...ator/CodeGenerator/CSharp/CodeGeneratorSettings.cs100% <100%> (ø)
...t.ML.CodeGenerator/CodeGenerator/CSharp/Symbols.cs100% <100%> (ø)
...ft.ML.CodeGenerator.Tests/TrainerGeneratorTests.cs100% <100%> (ø)
src/Microsoft.ML.CodeGenerator/Utils.cs18.3% <18.3%> (ø)
...ML.CodeGenerator/Templates/Console/ModelProject.cs29.29% <29.29%> (ø)
...odeGenerator/Templates/Console/ModelOutputClass.cs35.19% <35.19%> (ø)
... and 33 more

Comment threadsrc/Microsoft.ML.CodeGenerator/Microsoft.ML.CodeGenerator.csproj Outdated
Comment threadsrc/Microsoft.ML.CodeGenerator/Microsoft.ML.CodeGenerator.csproj Outdated
Comment threadsrc/Microsoft.ML.CodeGenerator/Templates/Console/Annotation.cs Outdated
@maryamariyan

Copy link
Copy Markdown
ContributorAuthor

NOTE:
@eerhardt@LittleLittleCloud since you both had already approved the PR I just squash rebased it with master, to get the latest changes from #4377.

The diff should be the same as you both reviewed.

@justinormontjustinormont left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@codemzs
codemzs merged commit 4f6de52 into dotnet:masterOct 25, 2019
frank-dong-ms-zz pushed a commit to frank-dong-ms-zz/machinelearning that referenced this pull request Nov 4, 2019
)
All legs are passing except for one which appears to be a random failure and likely will not break the CI. This PR is blocking a few other important PRs that need to be in today hence I'm merging it. Spoke with @eerhardt who also agrees is ok to merge it with one leg still building and all greens.
@ghostghost locked as resolved and limited conversation to collaborators Mar 20, 2022
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants

@maryamariyan@eerhardt@sharwell@justinormont@LittleLittleCloud@codemzs
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

Adds CodeGen to master (from features/automl) - #4365

Merged
codemzs merged 1 commit into
dotnet:masterfrom
maryamariyan:codegen-to-master
Oct 25, 2019
Merged

Adds CodeGen to master (from features/automl)#4365
codemzs merged 1 commit into
dotnet:masterfrom
maryamariyan:codegen-to-master

Conversation

@maryamariyan

@maryamariyanmaryamariyan commented Oct 22, 2019

Copy link
Copy Markdown
Contributor
  • Add src project for CodeGen
  • Add test project for CodeGen
  • Add packaging for CodeGen

cc: @eerhardt@LittleLittleCloud

@maryamariyanmaryamariyan self-assigned this Oct 22, 2019
Comment threadsrc/Microsoft.ML.CodeGen/Microsoft.ML.CodeGenerator.csproj Outdated
@maryamariyanmaryamariyan changed the title [WIP] Adds CodeGen to master (from features/automl)Adds CodeGen to master (from features/automl)Oct 22, 2019
Comment threadsrc/mlnet/mlnet.Build.props Outdated
Comment threadsrc/Microsoft.ML.CodeGen/Utils.cs Outdated
Comment threadsrc/Microsoft.ML.CodeGen/Microsoft.ML.CodeGenerator.csproj Outdated
Comment threadsrc/Microsoft.ML.CodeGen/Templates/Console/Annotation.txt Outdated
Comment threadsrc/Microsoft.ML.AutoML/Assembly.cs
@maryamariyan
maryamariyan marked this pull request as ready for review October 22, 2019 23:55
@maryamariyan
maryamariyan requested review from a team as code ownersOctober 22, 2019 23:55
Comment threadsrc/Microsoft.ML.CodeGenerator/Microsoft.ML.CodeGenerator.csproj Outdated
Comment threadsrc/Microsoft.ML.CodeGenerator/Microsoft.ML.CodeGenerator.csproj Outdated
Comment threadsrc/Microsoft.ML.CodeGenerator/Microsoft.ML.CodeGenerator.csproj Outdated
Comment threadpkg/Microsoft.ML.CodeGen/Microsoft.ML.CodeGen.nupkgproj
Comment threadpkg/Microsoft.ML.CodeGen/Microsoft.ML.CodeGen.nupkgproj
Comment threadsrc/Microsoft.ML.CodeGenerator/CodeGenerator/CSharp/CodeGenerator.cs Outdated
@eerhardt

Copy link
Copy Markdown
Member

<#@ template language="C#" #>

We should put a visibility="internal" on these, so the class that is generated is internal instead of public.

<#@ template language="C#" visibility="internal" #>

Refers to: src/Microsoft.ML.CodeGenerator/Templates/Console/ConsumeModel.tt:1 in a98fa15. [](commit_id = a98fa15, deletion_comment = False)

@maryamariyan

maryamariyan commented Oct 23, 2019

Copy link
Copy Markdown
ContributorAuthor

I'm looking at this comment: #4365 (comment)

Update: addressed in 2f894f4

Comment threadsrc/Microsoft.ML.AutoML/Assembly.cs
Comment threadsrc/Microsoft.ML.AutoML/Assembly.cs
{#>
//*****************************************************************************************
//* *
//* This is an auto-generated file by Microsoft ML.NET CLI (Command-Line Interface) tool. *

@sharwellsharwellOct 23, 2019

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ This is missing the <auto-generated/> comment, which means code style analyzer diagnostics will be reported in the file even though it is generated.

@eerhardteerhardtOct 24, 2019

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My thinking is that these code files aren't true "auto generated" files. I expect users to update them. Especially considering that we are putting hard-coded paths in some of the files, and we are making public types, that I assume most users will want to make internal. Or, if they do want them to be public - they would want to put XML docs on them.

So what is the appropriate pattern here? I think it is similar to "scaffolding" in ASP.NET. Or even "Add -> new Item" in VS. Do they put any "this was generated" comments at the top there?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For this case, the ideal header would be more like a template, with links explaining what the file is and where to find documentation of expected changes users would need to make. You would not need the <auto-generated/> comment for templates (one-time code generation).

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@maryamariyan - can you log an issue to address this? I think we should make that change separately and we will need @JakeRadMSFT@LittleLittleCloud and @rustd input as well.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Created #4387

<#+
void MB_Annotation()
{#>
// This file was auto-generated by ML.NET Model Builder.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ This is missing the <auto-generated/> comment, which means diagnostics will be reported in the file even though it is generated.

Comment threadsrc/Microsoft.ML.CodeGenerator/Templates/Console/PredictProject.cs Outdated

@eerhardteerhardt left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this change is ready to be merged.

We can address @sharwell's concern in a follow-up change. But let's not let that block us from moving this code from features/automl into the master branch. It is an existing issue in the code, and there isn't a clear direction we should take on it right now.

@codecov

codecovBot commented Oct 24, 2019

Copy link
Copy Markdown

Codecov Report

Merging #4365 into master will increase coverage by <.01%.
The diff coverage is 74.36%.

@@ Coverage Diff @@## master #4365 +/- ##
==========================================
+ Coverage 74.65% 74.65% +<.01% 
==========================================
Files 883 903 +20 Lines 155117 158562 +3445 Branches 16931 17136 +205 ==========================================
+ Hits 115795 118378 +2583 - Misses 34572 35362 +790 - Partials 4750 4822 +72
FlagCoverage Δ
#Debug74.65% <74.36%> (ø)⬆️
#production70.11% <64.23%> (-0.11%)⬇️
#test89.84% <100%> (+0.28%)⬆️
Impacted FilesCoverage Δ
...tor/CodeGenerator/CSharp/TransformGeneratorBase.cs100% <100%> (ø)
...r.Tests/ApprovalTests/ConsoleCodeGeneratorTests.cs100% <100%> (ø)
....ML.CodeGenerator.Tests/TransformGeneratorTests.cs100% <100%> (ø)
...t/Microsoft.ML.CodeGenerator.Tests/CodeGenTests.cs100% <100%> (ø)
...ator/CodeGenerator/CSharp/CodeGeneratorSettings.cs100% <100%> (ø)
...t.ML.CodeGenerator/CodeGenerator/CSharp/Symbols.cs100% <100%> (ø)
...ft.ML.CodeGenerator.Tests/TrainerGeneratorTests.cs100% <100%> (ø)
src/Microsoft.ML.CodeGenerator/Utils.cs18.3% <18.3%> (ø)
...ML.CodeGenerator/Templates/Console/ModelProject.cs29.29% <29.29%> (ø)
...odeGenerator/Templates/Console/ModelOutputClass.cs35.19% <35.19%> (ø)
... and 33 more

Comment threadsrc/Microsoft.ML.CodeGenerator/Microsoft.ML.CodeGenerator.csproj Outdated
Comment threadsrc/Microsoft.ML.CodeGenerator/Microsoft.ML.CodeGenerator.csproj Outdated
Comment threadsrc/Microsoft.ML.CodeGenerator/Templates/Console/Annotation.cs Outdated
@maryamariyan

Copy link
Copy Markdown
ContributorAuthor

NOTE:
@eerhardt@LittleLittleCloud since you both had already approved the PR I just squash rebased it with master, to get the latest changes from #4377.

The diff should be the same as you both reviewed.

@justinormontjustinormont left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@codemzs
codemzs merged commit 4f6de52 into dotnet:masterOct 25, 2019
frank-dong-ms-zz pushed a commit to frank-dong-ms-zz/machinelearning that referenced this pull request Nov 4, 2019
)
All legs are passing except for one which appears to be a random failure and likely will not break the CI. This PR is blocking a few other important PRs that need to be in today hence I'm merging it. Spoke with @eerhardt who also agrees is ok to merge it with one leg still building and all greens.
@ghostghost locked as resolved and limited conversation to collaborators Mar 20, 2022
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants

@maryamariyan@eerhardt@sharwell@justinormont@LittleLittleCloud@codemzs
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Adds CodeGen to master (from features/automl) - #4365

Merged
codemzs merged 1 commit into
dotnet:masterfrom
maryamariyan:codegen-to-master
Oct 25, 2019
Merged

Adds CodeGen to master (from features/automl)#4365
codemzs merged 1 commit into
dotnet:masterfrom
maryamariyan:codegen-to-master

Conversation

@maryamariyan

@maryamariyanmaryamariyan commented Oct 22, 2019

Copy link
Copy Markdown
Contributor
  • Add src project for CodeGen
  • Add test project for CodeGen
  • Add packaging for CodeGen

cc: @eerhardt@LittleLittleCloud

@maryamariyanmaryamariyan self-assigned this Oct 22, 2019
Comment threadsrc/Microsoft.ML.CodeGen/Microsoft.ML.CodeGenerator.csproj Outdated
@maryamariyanmaryamariyan changed the title [WIP] Adds CodeGen to master (from features/automl)Adds CodeGen to master (from features/automl)Oct 22, 2019
Comment threadsrc/mlnet/mlnet.Build.props Outdated
Comment threadsrc/Microsoft.ML.CodeGen/Utils.cs Outdated
Comment threadsrc/Microsoft.ML.CodeGen/Microsoft.ML.CodeGenerator.csproj Outdated
Comment threadsrc/Microsoft.ML.CodeGen/Templates/Console/Annotation.txt Outdated
Comment threadsrc/Microsoft.ML.AutoML/Assembly.cs
@maryamariyan
maryamariyan marked this pull request as ready for review October 22, 2019 23:55
@maryamariyan
maryamariyan requested review from a team as code ownersOctober 22, 2019 23:55
Comment threadsrc/Microsoft.ML.CodeGenerator/Microsoft.ML.CodeGenerator.csproj Outdated
Comment threadsrc/Microsoft.ML.CodeGenerator/Microsoft.ML.CodeGenerator.csproj Outdated
Comment threadsrc/Microsoft.ML.CodeGenerator/Microsoft.ML.CodeGenerator.csproj Outdated
Comment threadpkg/Microsoft.ML.CodeGen/Microsoft.ML.CodeGen.nupkgproj
Comment threadpkg/Microsoft.ML.CodeGen/Microsoft.ML.CodeGen.nupkgproj
Comment threadsrc/Microsoft.ML.CodeGenerator/CodeGenerator/CSharp/CodeGenerator.cs Outdated
@eerhardt

Copy link
Copy Markdown
Member

<#@ template language="C#" #>

We should put a visibility="internal" on these, so the class that is generated is internal instead of public.

<#@ template language="C#" visibility="internal" #>

Refers to: src/Microsoft.ML.CodeGenerator/Templates/Console/ConsumeModel.tt:1 in a98fa15. [](commit_id = a98fa15, deletion_comment = False)

@maryamariyan

maryamariyan commented Oct 23, 2019

Copy link
Copy Markdown
ContributorAuthor

I'm looking at this comment: #4365 (comment)

Update: addressed in 2f894f4

Comment threadsrc/Microsoft.ML.AutoML/Assembly.cs
Comment threadsrc/Microsoft.ML.AutoML/Assembly.cs
{#>
//*****************************************************************************************
//* *
//* This is an auto-generated file by Microsoft ML.NET CLI (Command-Line Interface) tool. *

@sharwellsharwellOct 23, 2019

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ This is missing the <auto-generated/> comment, which means code style analyzer diagnostics will be reported in the file even though it is generated.

@eerhardteerhardtOct 24, 2019

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My thinking is that these code files aren't true "auto generated" files. I expect users to update them. Especially considering that we are putting hard-coded paths in some of the files, and we are making public types, that I assume most users will want to make internal. Or, if they do want them to be public - they would want to put XML docs on them.

So what is the appropriate pattern here? I think it is similar to "scaffolding" in ASP.NET. Or even "Add -> new Item" in VS. Do they put any "this was generated" comments at the top there?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For this case, the ideal header would be more like a template, with links explaining what the file is and where to find documentation of expected changes users would need to make. You would not need the <auto-generated/> comment for templates (one-time code generation).

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@maryamariyan - can you log an issue to address this? I think we should make that change separately and we will need @JakeRadMSFT@LittleLittleCloud and @rustd input as well.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Created #4387

<#+
void MB_Annotation()
{#>
// This file was auto-generated by ML.NET Model Builder.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ This is missing the <auto-generated/> comment, which means diagnostics will be reported in the file even though it is generated.

Comment threadsrc/Microsoft.ML.CodeGenerator/Templates/Console/PredictProject.cs Outdated

@eerhardteerhardt left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this change is ready to be merged.

We can address @sharwell's concern in a follow-up change. But let's not let that block us from moving this code from features/automl into the master branch. It is an existing issue in the code, and there isn't a clear direction we should take on it right now.

@codecov

codecovBot commented Oct 24, 2019

Copy link
Copy Markdown

Codecov Report

Merging #4365 into master will increase coverage by <.01%.
The diff coverage is 74.36%.

@@ Coverage Diff @@## master #4365 +/- ##
==========================================
+ Coverage 74.65% 74.65% +<.01% 
==========================================
Files 883 903 +20 Lines 155117 158562 +3445 Branches 16931 17136 +205 ==========================================
+ Hits 115795 118378 +2583 - Misses 34572 35362 +790 - Partials 4750 4822 +72
FlagCoverage Δ
#Debug74.65% <74.36%> (ø)⬆️
#production70.11% <64.23%> (-0.11%)⬇️
#test89.84% <100%> (+0.28%)⬆️
Impacted FilesCoverage Δ
...tor/CodeGenerator/CSharp/TransformGeneratorBase.cs100% <100%> (ø)
...r.Tests/ApprovalTests/ConsoleCodeGeneratorTests.cs100% <100%> (ø)
....ML.CodeGenerator.Tests/TransformGeneratorTests.cs100% <100%> (ø)
...t/Microsoft.ML.CodeGenerator.Tests/CodeGenTests.cs100% <100%> (ø)
...ator/CodeGenerator/CSharp/CodeGeneratorSettings.cs100% <100%> (ø)
...t.ML.CodeGenerator/CodeGenerator/CSharp/Symbols.cs100% <100%> (ø)
...ft.ML.CodeGenerator.Tests/TrainerGeneratorTests.cs100% <100%> (ø)
src/Microsoft.ML.CodeGenerator/Utils.cs18.3% <18.3%> (ø)
...ML.CodeGenerator/Templates/Console/ModelProject.cs29.29% <29.29%> (ø)
...odeGenerator/Templates/Console/ModelOutputClass.cs35.19% <35.19%> (ø)
... and 33 more

Comment threadsrc/Microsoft.ML.CodeGenerator/Microsoft.ML.CodeGenerator.csproj Outdated
Comment threadsrc/Microsoft.ML.CodeGenerator/Microsoft.ML.CodeGenerator.csproj Outdated
Comment threadsrc/Microsoft.ML.CodeGenerator/Templates/Console/Annotation.cs Outdated
@maryamariyan

Copy link
Copy Markdown
ContributorAuthor

NOTE:
@eerhardt@LittleLittleCloud since you both had already approved the PR I just squash rebased it with master, to get the latest changes from #4377.

The diff should be the same as you both reviewed.

@justinormontjustinormont left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@codemzs
codemzs merged commit 4f6de52 into dotnet:masterOct 25, 2019
frank-dong-ms-zz pushed a commit to frank-dong-ms-zz/machinelearning that referenced this pull request Nov 4, 2019
)
All legs are passing except for one which appears to be a random failure and likely will not break the CI. This PR is blocking a few other important PRs that need to be in today hence I'm merging it. Spoke with @eerhardt who also agrees is ok to merge it with one leg still building and all greens.
@ghostghost locked as resolved and limited conversation to collaborators Mar 20, 2022
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants

@maryamariyan@eerhardt@sharwell@justinormont@LittleLittleCloud@codemzs
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Adds CodeGen to master (from features/automl) - #4365

Merged
codemzs merged 1 commit into
dotnet:masterfrom
maryamariyan:codegen-to-master
Oct 25, 2019
Merged

Adds CodeGen to master (from features/automl)#4365
codemzs merged 1 commit into
dotnet:masterfrom
maryamariyan:codegen-to-master

Conversation

@maryamariyan

@maryamariyanmaryamariyan commented Oct 22, 2019

Copy link
Copy Markdown
Contributor
  • Add src project for CodeGen
  • Add test project for CodeGen
  • Add packaging for CodeGen

cc: @eerhardt@LittleLittleCloud

@maryamariyanmaryamariyan self-assigned this Oct 22, 2019
Comment threadsrc/Microsoft.ML.CodeGen/Microsoft.ML.CodeGenerator.csproj Outdated
@maryamariyanmaryamariyan changed the title [WIP] Adds CodeGen to master (from features/automl)Adds CodeGen to master (from features/automl)Oct 22, 2019
Comment threadsrc/mlnet/mlnet.Build.props Outdated
Comment threadsrc/Microsoft.ML.CodeGen/Utils.cs Outdated
Comment threadsrc/Microsoft.ML.CodeGen/Microsoft.ML.CodeGenerator.csproj Outdated
Comment threadsrc/Microsoft.ML.CodeGen/Templates/Console/Annotation.txt Outdated
Comment threadsrc/Microsoft.ML.AutoML/Assembly.cs
@maryamariyan
maryamariyan marked this pull request as ready for review October 22, 2019 23:55
@maryamariyan
maryamariyan requested review from a team as code ownersOctober 22, 2019 23:55
Comment threadsrc/Microsoft.ML.CodeGenerator/Microsoft.ML.CodeGenerator.csproj Outdated
Comment threadsrc/Microsoft.ML.CodeGenerator/Microsoft.ML.CodeGenerator.csproj Outdated
Comment threadsrc/Microsoft.ML.CodeGenerator/Microsoft.ML.CodeGenerator.csproj Outdated
Comment threadpkg/Microsoft.ML.CodeGen/Microsoft.ML.CodeGen.nupkgproj
Comment threadpkg/Microsoft.ML.CodeGen/Microsoft.ML.CodeGen.nupkgproj
Comment threadsrc/Microsoft.ML.CodeGenerator/CodeGenerator/CSharp/CodeGenerator.cs Outdated
@eerhardt

Copy link
Copy Markdown
Member

<#@ template language="C#" #>

We should put a visibility="internal" on these, so the class that is generated is internal instead of public.

<#@ template language="C#" visibility="internal" #>

Refers to: src/Microsoft.ML.CodeGenerator/Templates/Console/ConsumeModel.tt:1 in a98fa15. [](commit_id = a98fa15, deletion_comment = False)

@maryamariyan

maryamariyan commented Oct 23, 2019

Copy link
Copy Markdown
ContributorAuthor

I'm looking at this comment: #4365 (comment)

Update: addressed in 2f894f4

Comment threadsrc/Microsoft.ML.AutoML/Assembly.cs
Comment threadsrc/Microsoft.ML.AutoML/Assembly.cs
{#>
//*****************************************************************************************
//* *
//* This is an auto-generated file by Microsoft ML.NET CLI (Command-Line Interface) tool. *

@sharwellsharwellOct 23, 2019

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ This is missing the <auto-generated/> comment, which means code style analyzer diagnostics will be reported in the file even though it is generated.

@eerhardteerhardtOct 24, 2019

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My thinking is that these code files aren't true "auto generated" files. I expect users to update them. Especially considering that we are putting hard-coded paths in some of the files, and we are making public types, that I assume most users will want to make internal. Or, if they do want them to be public - they would want to put XML docs on them.

So what is the appropriate pattern here? I think it is similar to "scaffolding" in ASP.NET. Or even "Add -> new Item" in VS. Do they put any "this was generated" comments at the top there?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For this case, the ideal header would be more like a template, with links explaining what the file is and where to find documentation of expected changes users would need to make. You would not need the <auto-generated/> comment for templates (one-time code generation).

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@maryamariyan - can you log an issue to address this? I think we should make that change separately and we will need @JakeRadMSFT@LittleLittleCloud and @rustd input as well.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Created #4387

<#+
void MB_Annotation()
{#>
// This file was auto-generated by ML.NET Model Builder.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ This is missing the <auto-generated/> comment, which means diagnostics will be reported in the file even though it is generated.

Comment threadsrc/Microsoft.ML.CodeGenerator/Templates/Console/PredictProject.cs Outdated

@eerhardteerhardt left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this change is ready to be merged.

We can address @sharwell's concern in a follow-up change. But let's not let that block us from moving this code from features/automl into the master branch. It is an existing issue in the code, and there isn't a clear direction we should take on it right now.

@codecov

codecovBot commented Oct 24, 2019

Copy link
Copy Markdown

Codecov Report

Merging #4365 into master will increase coverage by <.01%.
The diff coverage is 74.36%.

@@ Coverage Diff @@## master #4365 +/- ##
==========================================
+ Coverage 74.65% 74.65% +<.01% 
==========================================
Files 883 903 +20 Lines 155117 158562 +3445 Branches 16931 17136 +205 ==========================================
+ Hits 115795 118378 +2583 - Misses 34572 35362 +790 - Partials 4750 4822 +72
FlagCoverage Δ
#Debug74.65% <74.36%> (ø)⬆️
#production70.11% <64.23%> (-0.11%)⬇️
#test89.84% <100%> (+0.28%)⬆️
Impacted FilesCoverage Δ
...tor/CodeGenerator/CSharp/TransformGeneratorBase.cs100% <100%> (ø)
...r.Tests/ApprovalTests/ConsoleCodeGeneratorTests.cs100% <100%> (ø)
....ML.CodeGenerator.Tests/TransformGeneratorTests.cs100% <100%> (ø)
...t/Microsoft.ML.CodeGenerator.Tests/CodeGenTests.cs100% <100%> (ø)
...ator/CodeGenerator/CSharp/CodeGeneratorSettings.cs100% <100%> (ø)
...t.ML.CodeGenerator/CodeGenerator/CSharp/Symbols.cs100% <100%> (ø)
...ft.ML.CodeGenerator.Tests/TrainerGeneratorTests.cs100% <100%> (ø)
src/Microsoft.ML.CodeGenerator/Utils.cs18.3% <18.3%> (ø)
...ML.CodeGenerator/Templates/Console/ModelProject.cs29.29% <29.29%> (ø)
...odeGenerator/Templates/Console/ModelOutputClass.cs35.19% <35.19%> (ø)
... and 33 more

Comment threadsrc/Microsoft.ML.CodeGenerator/Microsoft.ML.CodeGenerator.csproj Outdated
Comment threadsrc/Microsoft.ML.CodeGenerator/Microsoft.ML.CodeGenerator.csproj Outdated
Comment threadsrc/Microsoft.ML.CodeGenerator/Templates/Console/Annotation.cs Outdated
@maryamariyan

Copy link
Copy Markdown
ContributorAuthor

NOTE:
@eerhardt@LittleLittleCloud since you both had already approved the PR I just squash rebased it with master, to get the latest changes from #4377.

The diff should be the same as you both reviewed.

@justinormontjustinormont left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@codemzs
codemzs merged commit 4f6de52 into dotnet:masterOct 25, 2019
frank-dong-ms-zz pushed a commit to frank-dong-ms-zz/machinelearning that referenced this pull request Nov 4, 2019
)
All legs are passing except for one which appears to be a random failure and likely will not break the CI. This PR is blocking a few other important PRs that need to be in today hence I'm merging it. Spoke with @eerhardt who also agrees is ok to merge it with one leg still building and all greens.
@ghostghost locked as resolved and limited conversation to collaborators Mar 20, 2022
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants

@maryamariyan@eerhardt@sharwell@justinormont@LittleLittleCloud@codemzs
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

Adds CodeGen to master (from features/automl) - #4365

Merged
codemzs merged 1 commit into
dotnet:masterfrom
maryamariyan:codegen-to-master
Oct 25, 2019
Merged

Adds CodeGen to master (from features/automl)#4365
codemzs merged 1 commit into
dotnet:masterfrom
maryamariyan:codegen-to-master

Conversation

@maryamariyan

@maryamariyanmaryamariyan commented Oct 22, 2019

Copy link
Copy Markdown
Contributor
  • Add src project for CodeGen
  • Add test project for CodeGen
  • Add packaging for CodeGen

cc: @eerhardt@LittleLittleCloud

@maryamariyanmaryamariyan self-assigned this Oct 22, 2019
Comment threadsrc/Microsoft.ML.CodeGen/Microsoft.ML.CodeGenerator.csproj Outdated
@maryamariyanmaryamariyan changed the title [WIP] Adds CodeGen to master (from features/automl)Adds CodeGen to master (from features/automl)Oct 22, 2019
Comment threadsrc/mlnet/mlnet.Build.props Outdated
Comment threadsrc/Microsoft.ML.CodeGen/Utils.cs Outdated
Comment threadsrc/Microsoft.ML.CodeGen/Microsoft.ML.CodeGenerator.csproj Outdated
Comment threadsrc/Microsoft.ML.CodeGen/Templates/Console/Annotation.txt Outdated
Comment threadsrc/Microsoft.ML.AutoML/Assembly.cs
@maryamariyan
maryamariyan marked this pull request as ready for review October 22, 2019 23:55
@maryamariyan
maryamariyan requested review from a team as code ownersOctober 22, 2019 23:55
Comment threadsrc/Microsoft.ML.CodeGenerator/Microsoft.ML.CodeGenerator.csproj Outdated
Comment threadsrc/Microsoft.ML.CodeGenerator/Microsoft.ML.CodeGenerator.csproj Outdated
Comment threadsrc/Microsoft.ML.CodeGenerator/Microsoft.ML.CodeGenerator.csproj Outdated
Comment threadpkg/Microsoft.ML.CodeGen/Microsoft.ML.CodeGen.nupkgproj
Comment threadpkg/Microsoft.ML.CodeGen/Microsoft.ML.CodeGen.nupkgproj
Comment threadsrc/Microsoft.ML.CodeGenerator/CodeGenerator/CSharp/CodeGenerator.cs Outdated
@eerhardt

Copy link
Copy Markdown
Member

<#@ template language="C#" #>

We should put a visibility="internal" on these, so the class that is generated is internal instead of public.

<#@ template language="C#" visibility="internal" #>

Refers to: src/Microsoft.ML.CodeGenerator/Templates/Console/ConsumeModel.tt:1 in a98fa15. [](commit_id = a98fa15, deletion_comment = False)

@maryamariyan

maryamariyan commented Oct 23, 2019

Copy link
Copy Markdown
ContributorAuthor

I'm looking at this comment: #4365 (comment)

Update: addressed in 2f894f4

Comment threadsrc/Microsoft.ML.AutoML/Assembly.cs
Comment threadsrc/Microsoft.ML.AutoML/Assembly.cs
{#>
//*****************************************************************************************
//* *
//* This is an auto-generated file by Microsoft ML.NET CLI (Command-Line Interface) tool. *

@sharwellsharwellOct 23, 2019

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ This is missing the <auto-generated/> comment, which means code style analyzer diagnostics will be reported in the file even though it is generated.

@eerhardteerhardtOct 24, 2019

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My thinking is that these code files aren't true "auto generated" files. I expect users to update them. Especially considering that we are putting hard-coded paths in some of the files, and we are making public types, that I assume most users will want to make internal. Or, if they do want them to be public - they would want to put XML docs on them.

So what is the appropriate pattern here? I think it is similar to "scaffolding" in ASP.NET. Or even "Add -> new Item" in VS. Do they put any "this was generated" comments at the top there?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For this case, the ideal header would be more like a template, with links explaining what the file is and where to find documentation of expected changes users would need to make. You would not need the <auto-generated/> comment for templates (one-time code generation).

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@maryamariyan - can you log an issue to address this? I think we should make that change separately and we will need @JakeRadMSFT@LittleLittleCloud and @rustd input as well.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Created #4387

<#+
void MB_Annotation()
{#>
// This file was auto-generated by ML.NET Model Builder.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ This is missing the <auto-generated/> comment, which means diagnostics will be reported in the file even though it is generated.

Comment threadsrc/Microsoft.ML.CodeGenerator/Templates/Console/PredictProject.cs Outdated

@eerhardteerhardt left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this change is ready to be merged.

We can address @sharwell's concern in a follow-up change. But let's not let that block us from moving this code from features/automl into the master branch. It is an existing issue in the code, and there isn't a clear direction we should take on it right now.

@codecov

codecovBot commented Oct 24, 2019

Copy link
Copy Markdown

Codecov Report

Merging #4365 into master will increase coverage by <.01%.
The diff coverage is 74.36%.

@@ Coverage Diff @@## master #4365 +/- ##
==========================================
+ Coverage 74.65% 74.65% +<.01% 
==========================================
Files 883 903 +20 Lines 155117 158562 +3445 Branches 16931 17136 +205 ==========================================
+ Hits 115795 118378 +2583 - Misses 34572 35362 +790 - Partials 4750 4822 +72
FlagCoverage Δ
#Debug74.65% <74.36%> (ø)⬆️
#production70.11% <64.23%> (-0.11%)⬇️
#test89.84% <100%> (+0.28%)⬆️
Impacted FilesCoverage Δ
...tor/CodeGenerator/CSharp/TransformGeneratorBase.cs100% <100%> (ø)
...r.Tests/ApprovalTests/ConsoleCodeGeneratorTests.cs100% <100%> (ø)
....ML.CodeGenerator.Tests/TransformGeneratorTests.cs100% <100%> (ø)
...t/Microsoft.ML.CodeGenerator.Tests/CodeGenTests.cs100% <100%> (ø)
...ator/CodeGenerator/CSharp/CodeGeneratorSettings.cs100% <100%> (ø)
...t.ML.CodeGenerator/CodeGenerator/CSharp/Symbols.cs100% <100%> (ø)
...ft.ML.CodeGenerator.Tests/TrainerGeneratorTests.cs100% <100%> (ø)
src/Microsoft.ML.CodeGenerator/Utils.cs18.3% <18.3%> (ø)
...ML.CodeGenerator/Templates/Console/ModelProject.cs29.29% <29.29%> (ø)
...odeGenerator/Templates/Console/ModelOutputClass.cs35.19% <35.19%> (ø)
... and 33 more

Comment threadsrc/Microsoft.ML.CodeGenerator/Microsoft.ML.CodeGenerator.csproj Outdated
Comment threadsrc/Microsoft.ML.CodeGenerator/Microsoft.ML.CodeGenerator.csproj Outdated
Comment threadsrc/Microsoft.ML.CodeGenerator/Templates/Console/Annotation.cs Outdated
@maryamariyan

Copy link
Copy Markdown
ContributorAuthor

NOTE:
@eerhardt@LittleLittleCloud since you both had already approved the PR I just squash rebased it with master, to get the latest changes from #4377.

The diff should be the same as you both reviewed.

@justinormontjustinormont left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@codemzs
codemzs merged commit 4f6de52 into dotnet:masterOct 25, 2019
frank-dong-ms-zz pushed a commit to frank-dong-ms-zz/machinelearning that referenced this pull request Nov 4, 2019
)
All legs are passing except for one which appears to be a random failure and likely will not break the CI. This PR is blocking a few other important PRs that need to be in today hence I'm merging it. Spoke with @eerhardt who also agrees is ok to merge it with one leg still building and all greens.
@ghostghost locked as resolved and limited conversation to collaborators Mar 20, 2022
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants

@maryamariyan@eerhardt@sharwell@justinormont@LittleLittleCloud@codemzs
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Adds CodeGen to master (from features/automl) - #4365

Merged
codemzs merged 1 commit into
dotnet:masterfrom
maryamariyan:codegen-to-master
Oct 25, 2019
Merged

Adds CodeGen to master (from features/automl)#4365
codemzs merged 1 commit into
dotnet:masterfrom
maryamariyan:codegen-to-master

Conversation

@maryamariyan

@maryamariyanmaryamariyan commented Oct 22, 2019

Copy link
Copy Markdown
Contributor
  • Add src project for CodeGen
  • Add test project for CodeGen
  • Add packaging for CodeGen

cc: @eerhardt@LittleLittleCloud

@maryamariyanmaryamariyan self-assigned this Oct 22, 2019
Comment threadsrc/Microsoft.ML.CodeGen/Microsoft.ML.CodeGenerator.csproj Outdated
@maryamariyanmaryamariyan changed the title [WIP] Adds CodeGen to master (from features/automl)Adds CodeGen to master (from features/automl)Oct 22, 2019
Comment threadsrc/mlnet/mlnet.Build.props Outdated
Comment threadsrc/Microsoft.ML.CodeGen/Utils.cs Outdated
Comment threadsrc/Microsoft.ML.CodeGen/Microsoft.ML.CodeGenerator.csproj Outdated
Comment threadsrc/Microsoft.ML.CodeGen/Templates/Console/Annotation.txt Outdated
Comment threadsrc/Microsoft.ML.AutoML/Assembly.cs
@maryamariyan
maryamariyan marked this pull request as ready for review October 22, 2019 23:55
@maryamariyan
maryamariyan requested review from a team as code ownersOctober 22, 2019 23:55
Comment threadsrc/Microsoft.ML.CodeGenerator/Microsoft.ML.CodeGenerator.csproj Outdated
Comment threadsrc/Microsoft.ML.CodeGenerator/Microsoft.ML.CodeGenerator.csproj Outdated
Comment threadsrc/Microsoft.ML.CodeGenerator/Microsoft.ML.CodeGenerator.csproj Outdated
Comment threadpkg/Microsoft.ML.CodeGen/Microsoft.ML.CodeGen.nupkgproj
Comment threadpkg/Microsoft.ML.CodeGen/Microsoft.ML.CodeGen.nupkgproj
Comment threadsrc/Microsoft.ML.CodeGenerator/CodeGenerator/CSharp/CodeGenerator.cs Outdated
@eerhardt

Copy link
Copy Markdown
Member

<#@ template language="C#" #>

We should put a visibility="internal" on these, so the class that is generated is internal instead of public.

<#@ template language="C#" visibility="internal" #>

Refers to: src/Microsoft.ML.CodeGenerator/Templates/Console/ConsumeModel.tt:1 in a98fa15. [](commit_id = a98fa15, deletion_comment = False)

@maryamariyan

maryamariyan commented Oct 23, 2019

Copy link
Copy Markdown
ContributorAuthor

I'm looking at this comment: #4365 (comment)

Update: addressed in 2f894f4

Comment threadsrc/Microsoft.ML.AutoML/Assembly.cs
Comment threadsrc/Microsoft.ML.AutoML/Assembly.cs
{#>
//*****************************************************************************************
//* *
//* This is an auto-generated file by Microsoft ML.NET CLI (Command-Line Interface) tool. *

@sharwellsharwellOct 23, 2019

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ This is missing the <auto-generated/> comment, which means code style analyzer diagnostics will be reported in the file even though it is generated.

@eerhardteerhardtOct 24, 2019

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My thinking is that these code files aren't true "auto generated" files. I expect users to update them. Especially considering that we are putting hard-coded paths in some of the files, and we are making public types, that I assume most users will want to make internal. Or, if they do want them to be public - they would want to put XML docs on them.

So what is the appropriate pattern here? I think it is similar to "scaffolding" in ASP.NET. Or even "Add -> new Item" in VS. Do they put any "this was generated" comments at the top there?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For this case, the ideal header would be more like a template, with links explaining what the file is and where to find documentation of expected changes users would need to make. You would not need the <auto-generated/> comment for templates (one-time code generation).

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@maryamariyan - can you log an issue to address this? I think we should make that change separately and we will need @JakeRadMSFT@LittleLittleCloud and @rustd input as well.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Created #4387

<#+
void MB_Annotation()
{#>
// This file was auto-generated by ML.NET Model Builder.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ This is missing the <auto-generated/> comment, which means diagnostics will be reported in the file even though it is generated.

Comment threadsrc/Microsoft.ML.CodeGenerator/Templates/Console/PredictProject.cs Outdated

@eerhardteerhardt left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this change is ready to be merged.

We can address @sharwell's concern in a follow-up change. But let's not let that block us from moving this code from features/automl into the master branch. It is an existing issue in the code, and there isn't a clear direction we should take on it right now.

@codecov

codecovBot commented Oct 24, 2019

Copy link
Copy Markdown

Codecov Report

Merging #4365 into master will increase coverage by <.01%.
The diff coverage is 74.36%.

@@ Coverage Diff @@## master #4365 +/- ##
==========================================
+ Coverage 74.65% 74.65% +<.01% 
==========================================
Files 883 903 +20 Lines 155117 158562 +3445 Branches 16931 17136 +205 ==========================================
+ Hits 115795 118378 +2583 - Misses 34572 35362 +790 - Partials 4750 4822 +72
FlagCoverage Δ
#Debug74.65% <74.36%> (ø)⬆️
#production70.11% <64.23%> (-0.11%)⬇️
#test89.84% <100%> (+0.28%)⬆️
Impacted FilesCoverage Δ
...tor/CodeGenerator/CSharp/TransformGeneratorBase.cs100% <100%> (ø)
...r.Tests/ApprovalTests/ConsoleCodeGeneratorTests.cs100% <100%> (ø)
....ML.CodeGenerator.Tests/TransformGeneratorTests.cs100% <100%> (ø)
...t/Microsoft.ML.CodeGenerator.Tests/CodeGenTests.cs100% <100%> (ø)
...ator/CodeGenerator/CSharp/CodeGeneratorSettings.cs100% <100%> (ø)
...t.ML.CodeGenerator/CodeGenerator/CSharp/Symbols.cs100% <100%> (ø)
...ft.ML.CodeGenerator.Tests/TrainerGeneratorTests.cs100% <100%> (ø)
src/Microsoft.ML.CodeGenerator/Utils.cs18.3% <18.3%> (ø)
...ML.CodeGenerator/Templates/Console/ModelProject.cs29.29% <29.29%> (ø)
...odeGenerator/Templates/Console/ModelOutputClass.cs35.19% <35.19%> (ø)
... and 33 more

Comment threadsrc/Microsoft.ML.CodeGenerator/Microsoft.ML.CodeGenerator.csproj Outdated
Comment threadsrc/Microsoft.ML.CodeGenerator/Microsoft.ML.CodeGenerator.csproj Outdated
Comment threadsrc/Microsoft.ML.CodeGenerator/Templates/Console/Annotation.cs Outdated
@maryamariyan

Copy link
Copy Markdown
ContributorAuthor

NOTE:
@eerhardt@LittleLittleCloud since you both had already approved the PR I just squash rebased it with master, to get the latest changes from #4377.

The diff should be the same as you both reviewed.

@justinormontjustinormont left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@codemzs
codemzs merged commit 4f6de52 into dotnet:masterOct 25, 2019
frank-dong-ms-zz pushed a commit to frank-dong-ms-zz/machinelearning that referenced this pull request Nov 4, 2019
)
All legs are passing except for one which appears to be a random failure and likely will not break the CI. This PR is blocking a few other important PRs that need to be in today hence I'm merging it. Spoke with @eerhardt who also agrees is ok to merge it with one leg still building and all greens.
@ghostghost locked as resolved and limited conversation to collaborators Mar 20, 2022
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants

@maryamariyan@eerhardt@sharwell@justinormont@LittleLittleCloud@codemzs
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Adds CodeGen to master (from features/automl) - #4365

Merged
codemzs merged 1 commit into
dotnet:masterfrom
maryamariyan:codegen-to-master
Oct 25, 2019
Merged

Adds CodeGen to master (from features/automl)#4365
codemzs merged 1 commit into
dotnet:masterfrom
maryamariyan:codegen-to-master

Conversation

@maryamariyan

@maryamariyanmaryamariyan commented Oct 22, 2019

Copy link
Copy Markdown
Contributor
  • Add src project for CodeGen
  • Add test project for CodeGen
  • Add packaging for CodeGen

cc: @eerhardt@LittleLittleCloud

@maryamariyanmaryamariyan self-assigned this Oct 22, 2019
Comment threadsrc/Microsoft.ML.CodeGen/Microsoft.ML.CodeGenerator.csproj Outdated
@maryamariyanmaryamariyan changed the title [WIP] Adds CodeGen to master (from features/automl)Adds CodeGen to master (from features/automl)Oct 22, 2019
Comment threadsrc/mlnet/mlnet.Build.props Outdated
Comment threadsrc/Microsoft.ML.CodeGen/Utils.cs Outdated
Comment threadsrc/Microsoft.ML.CodeGen/Microsoft.ML.CodeGenerator.csproj Outdated
Comment threadsrc/Microsoft.ML.CodeGen/Templates/Console/Annotation.txt Outdated
Comment threadsrc/Microsoft.ML.AutoML/Assembly.cs
@maryamariyan
maryamariyan marked this pull request as ready for review October 22, 2019 23:55
@maryamariyan
maryamariyan requested review from a team as code ownersOctober 22, 2019 23:55
Comment threadsrc/Microsoft.ML.CodeGenerator/Microsoft.ML.CodeGenerator.csproj Outdated
Comment threadsrc/Microsoft.ML.CodeGenerator/Microsoft.ML.CodeGenerator.csproj Outdated
Comment threadsrc/Microsoft.ML.CodeGenerator/Microsoft.ML.CodeGenerator.csproj Outdated
Comment threadpkg/Microsoft.ML.CodeGen/Microsoft.ML.CodeGen.nupkgproj
Comment threadpkg/Microsoft.ML.CodeGen/Microsoft.ML.CodeGen.nupkgproj
Comment threadsrc/Microsoft.ML.CodeGenerator/CodeGenerator/CSharp/CodeGenerator.cs Outdated
@eerhardt

Copy link
Copy Markdown
Member

<#@ template language="C#" #>

We should put a visibility="internal" on these, so the class that is generated is internal instead of public.

<#@ template language="C#" visibility="internal" #>

Refers to: src/Microsoft.ML.CodeGenerator/Templates/Console/ConsumeModel.tt:1 in a98fa15. [](commit_id = a98fa15, deletion_comment = False)

@maryamariyan

maryamariyan commented Oct 23, 2019

Copy link
Copy Markdown
ContributorAuthor

I'm looking at this comment: #4365 (comment)

Update: addressed in 2f894f4

Comment threadsrc/Microsoft.ML.AutoML/Assembly.cs
Comment threadsrc/Microsoft.ML.AutoML/Assembly.cs
{#>
//*****************************************************************************************
//* *
//* This is an auto-generated file by Microsoft ML.NET CLI (Command-Line Interface) tool. *

@sharwellsharwellOct 23, 2019

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ This is missing the <auto-generated/> comment, which means code style analyzer diagnostics will be reported in the file even though it is generated.

@eerhardteerhardtOct 24, 2019

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My thinking is that these code files aren't true "auto generated" files. I expect users to update them. Especially considering that we are putting hard-coded paths in some of the files, and we are making public types, that I assume most users will want to make internal. Or, if they do want them to be public - they would want to put XML docs on them.

So what is the appropriate pattern here? I think it is similar to "scaffolding" in ASP.NET. Or even "Add -> new Item" in VS. Do they put any "this was generated" comments at the top there?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For this case, the ideal header would be more like a template, with links explaining what the file is and where to find documentation of expected changes users would need to make. You would not need the <auto-generated/> comment for templates (one-time code generation).

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@maryamariyan - can you log an issue to address this? I think we should make that change separately and we will need @JakeRadMSFT@LittleLittleCloud and @rustd input as well.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Created #4387

<#+
void MB_Annotation()
{#>
// This file was auto-generated by ML.NET Model Builder.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ This is missing the <auto-generated/> comment, which means diagnostics will be reported in the file even though it is generated.

Comment threadsrc/Microsoft.ML.CodeGenerator/Templates/Console/PredictProject.cs Outdated

@eerhardteerhardt left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this change is ready to be merged.

We can address @sharwell's concern in a follow-up change. But let's not let that block us from moving this code from features/automl into the master branch. It is an existing issue in the code, and there isn't a clear direction we should take on it right now.

@codecov

codecovBot commented Oct 24, 2019

Copy link
Copy Markdown

Codecov Report

Merging #4365 into master will increase coverage by <.01%.
The diff coverage is 74.36%.

@@ Coverage Diff @@## master #4365 +/- ##
==========================================
+ Coverage 74.65% 74.65% +<.01% 
==========================================
Files 883 903 +20 Lines 155117 158562 +3445 Branches 16931 17136 +205 ==========================================
+ Hits 115795 118378 +2583 - Misses 34572 35362 +790 - Partials 4750 4822 +72
FlagCoverage Δ
#Debug74.65% <74.36%> (ø)⬆️
#production70.11% <64.23%> (-0.11%)⬇️
#test89.84% <100%> (+0.28%)⬆️
Impacted FilesCoverage Δ
...tor/CodeGenerator/CSharp/TransformGeneratorBase.cs100% <100%> (ø)
...r.Tests/ApprovalTests/ConsoleCodeGeneratorTests.cs100% <100%> (ø)
....ML.CodeGenerator.Tests/TransformGeneratorTests.cs100% <100%> (ø)
...t/Microsoft.ML.CodeGenerator.Tests/CodeGenTests.cs100% <100%> (ø)
...ator/CodeGenerator/CSharp/CodeGeneratorSettings.cs100% <100%> (ø)
...t.ML.CodeGenerator/CodeGenerator/CSharp/Symbols.cs100% <100%> (ø)
...ft.ML.CodeGenerator.Tests/TrainerGeneratorTests.cs100% <100%> (ø)
src/Microsoft.ML.CodeGenerator/Utils.cs18.3% <18.3%> (ø)
...ML.CodeGenerator/Templates/Console/ModelProject.cs29.29% <29.29%> (ø)
...odeGenerator/Templates/Console/ModelOutputClass.cs35.19% <35.19%> (ø)
... and 33 more

Comment threadsrc/Microsoft.ML.CodeGenerator/Microsoft.ML.CodeGenerator.csproj Outdated
Comment threadsrc/Microsoft.ML.CodeGenerator/Microsoft.ML.CodeGenerator.csproj Outdated
Comment threadsrc/Microsoft.ML.CodeGenerator/Templates/Console/Annotation.cs Outdated
@maryamariyan

Copy link
Copy Markdown
ContributorAuthor

NOTE:
@eerhardt@LittleLittleCloud since you both had already approved the PR I just squash rebased it with master, to get the latest changes from #4377.

The diff should be the same as you both reviewed.

@justinormontjustinormont left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@codemzs
codemzs merged commit 4f6de52 into dotnet:masterOct 25, 2019
frank-dong-ms-zz pushed a commit to frank-dong-ms-zz/machinelearning that referenced this pull request Nov 4, 2019
)
All legs are passing except for one which appears to be a random failure and likely will not break the CI. This PR is blocking a few other important PRs that need to be in today hence I'm merging it. Spoke with @eerhardt who also agrees is ok to merge it with one leg still building and all greens.
@ghostghost locked as resolved and limited conversation to collaborators Mar 20, 2022
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants

@maryamariyan@eerhardt@sharwell@justinormont@LittleLittleCloud@codemzs
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

Adds CodeGen to master (from features/automl) - #4365

Merged
codemzs merged 1 commit into
dotnet:masterfrom
maryamariyan:codegen-to-master
Oct 25, 2019
Merged

Adds CodeGen to master (from features/automl)#4365
codemzs merged 1 commit into
dotnet:masterfrom
maryamariyan:codegen-to-master

Conversation

@maryamariyan

@maryamariyanmaryamariyan commented Oct 22, 2019

Copy link
Copy Markdown
Contributor
  • Add src project for CodeGen
  • Add test project for CodeGen
  • Add packaging for CodeGen

cc: @eerhardt@LittleLittleCloud

@maryamariyanmaryamariyan self-assigned this Oct 22, 2019
Comment threadsrc/Microsoft.ML.CodeGen/Microsoft.ML.CodeGenerator.csproj Outdated
@maryamariyanmaryamariyan changed the title [WIP] Adds CodeGen to master (from features/automl)Adds CodeGen to master (from features/automl)Oct 22, 2019
Comment threadsrc/mlnet/mlnet.Build.props Outdated
Comment threadsrc/Microsoft.ML.CodeGen/Utils.cs Outdated
Comment threadsrc/Microsoft.ML.CodeGen/Microsoft.ML.CodeGenerator.csproj Outdated
Comment threadsrc/Microsoft.ML.CodeGen/Templates/Console/Annotation.txt Outdated
Comment threadsrc/Microsoft.ML.AutoML/Assembly.cs
@maryamariyan
maryamariyan marked this pull request as ready for review October 22, 2019 23:55
@maryamariyan
maryamariyan requested review from a team as code ownersOctober 22, 2019 23:55
Comment threadsrc/Microsoft.ML.CodeGenerator/Microsoft.ML.CodeGenerator.csproj Outdated
Comment threadsrc/Microsoft.ML.CodeGenerator/Microsoft.ML.CodeGenerator.csproj Outdated
Comment threadsrc/Microsoft.ML.CodeGenerator/Microsoft.ML.CodeGenerator.csproj Outdated
Comment threadpkg/Microsoft.ML.CodeGen/Microsoft.ML.CodeGen.nupkgproj
Comment threadpkg/Microsoft.ML.CodeGen/Microsoft.ML.CodeGen.nupkgproj
Comment threadsrc/Microsoft.ML.CodeGenerator/CodeGenerator/CSharp/CodeGenerator.cs Outdated
@eerhardt

Copy link
Copy Markdown
Member

<#@ template language="C#" #>

We should put a visibility="internal" on these, so the class that is generated is internal instead of public.

<#@ template language="C#" visibility="internal" #>

Refers to: src/Microsoft.ML.CodeGenerator/Templates/Console/ConsumeModel.tt:1 in a98fa15. [](commit_id = a98fa15, deletion_comment = False)

@maryamariyan

maryamariyan commented Oct 23, 2019

Copy link
Copy Markdown
ContributorAuthor

I'm looking at this comment: #4365 (comment)

Update: addressed in 2f894f4

Comment threadsrc/Microsoft.ML.AutoML/Assembly.cs
Comment threadsrc/Microsoft.ML.AutoML/Assembly.cs
{#>
//*****************************************************************************************
//* *
//* This is an auto-generated file by Microsoft ML.NET CLI (Command-Line Interface) tool. *

@sharwellsharwellOct 23, 2019

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ This is missing the <auto-generated/> comment, which means code style analyzer diagnostics will be reported in the file even though it is generated.

@eerhardteerhardtOct 24, 2019

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My thinking is that these code files aren't true "auto generated" files. I expect users to update them. Especially considering that we are putting hard-coded paths in some of the files, and we are making public types, that I assume most users will want to make internal. Or, if they do want them to be public - they would want to put XML docs on them.

So what is the appropriate pattern here? I think it is similar to "scaffolding" in ASP.NET. Or even "Add -> new Item" in VS. Do they put any "this was generated" comments at the top there?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For this case, the ideal header would be more like a template, with links explaining what the file is and where to find documentation of expected changes users would need to make. You would not need the <auto-generated/> comment for templates (one-time code generation).

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@maryamariyan - can you log an issue to address this? I think we should make that change separately and we will need @JakeRadMSFT@LittleLittleCloud and @rustd input as well.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Created #4387

<#+
void MB_Annotation()
{#>
// This file was auto-generated by ML.NET Model Builder.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ This is missing the <auto-generated/> comment, which means diagnostics will be reported in the file even though it is generated.

Comment threadsrc/Microsoft.ML.CodeGenerator/Templates/Console/PredictProject.cs Outdated

@eerhardteerhardt left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this change is ready to be merged.

We can address @sharwell's concern in a follow-up change. But let's not let that block us from moving this code from features/automl into the master branch. It is an existing issue in the code, and there isn't a clear direction we should take on it right now.

@codecov

codecovBot commented Oct 24, 2019

Copy link
Copy Markdown

Codecov Report

Merging #4365 into master will increase coverage by <.01%.
The diff coverage is 74.36%.

@@ Coverage Diff @@## master #4365 +/- ##
==========================================
+ Coverage 74.65% 74.65% +<.01% 
==========================================
Files 883 903 +20 Lines 155117 158562 +3445 Branches 16931 17136 +205 ==========================================
+ Hits 115795 118378 +2583 - Misses 34572 35362 +790 - Partials 4750 4822 +72
FlagCoverage Δ
#Debug74.65% <74.36%> (ø)⬆️
#production70.11% <64.23%> (-0.11%)⬇️
#test89.84% <100%> (+0.28%)⬆️
Impacted FilesCoverage Δ
...tor/CodeGenerator/CSharp/TransformGeneratorBase.cs100% <100%> (ø)
...r.Tests/ApprovalTests/ConsoleCodeGeneratorTests.cs100% <100%> (ø)
....ML.CodeGenerator.Tests/TransformGeneratorTests.cs100% <100%> (ø)
...t/Microsoft.ML.CodeGenerator.Tests/CodeGenTests.cs100% <100%> (ø)
...ator/CodeGenerator/CSharp/CodeGeneratorSettings.cs100% <100%> (ø)
...t.ML.CodeGenerator/CodeGenerator/CSharp/Symbols.cs100% <100%> (ø)
...ft.ML.CodeGenerator.Tests/TrainerGeneratorTests.cs100% <100%> (ø)
src/Microsoft.ML.CodeGenerator/Utils.cs18.3% <18.3%> (ø)
...ML.CodeGenerator/Templates/Console/ModelProject.cs29.29% <29.29%> (ø)
...odeGenerator/Templates/Console/ModelOutputClass.cs35.19% <35.19%> (ø)
... and 33 more

Comment threadsrc/Microsoft.ML.CodeGenerator/Microsoft.ML.CodeGenerator.csproj Outdated
Comment threadsrc/Microsoft.ML.CodeGenerator/Microsoft.ML.CodeGenerator.csproj Outdated
Comment threadsrc/Microsoft.ML.CodeGenerator/Templates/Console/Annotation.cs Outdated
@maryamariyan

Copy link
Copy Markdown
ContributorAuthor

NOTE:
@eerhardt@LittleLittleCloud since you both had already approved the PR I just squash rebased it with master, to get the latest changes from #4377.

The diff should be the same as you both reviewed.

@justinormontjustinormont left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@codemzs
codemzs merged commit 4f6de52 into dotnet:masterOct 25, 2019
frank-dong-ms-zz pushed a commit to frank-dong-ms-zz/machinelearning that referenced this pull request Nov 4, 2019
)
All legs are passing except for one which appears to be a random failure and likely will not break the CI. This PR is blocking a few other important PRs that need to be in today hence I'm merging it. Spoke with @eerhardt who also agrees is ok to merge it with one leg still building and all greens.
@ghostghost locked as resolved and limited conversation to collaborators Mar 20, 2022
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants

@maryamariyan@eerhardt@sharwell@justinormont@LittleLittleCloud@codemzs