JIT: change profile slop assert to a jitdump note - #81377

Merged
AndyAyersMS merged 1 commit into
dotnet:mainfrom
AndyAyersMS:ChangeSlopAssertToDiagnosticDump
Feb 2, 2023
Merged

JIT: change profile slop assert to a jitdump note#81377
AndyAyersMS merged 1 commit into
dotnet:mainfrom
AndyAyersMS:ChangeSlopAssertToDiagnosticDump

Conversation

@AndyAyersMS

@AndyAyersMSAndyAyersMS commented Jan 30, 2023

Copy link
Copy Markdown
Member

Stop asserting if we see unusually large discrepancies in the outgoing profile flow from a block. Instead just make a note in the jit dump.

Fixes#77450.

@AndyAyersMS
JIT: change profile slop assert to a jitdump note
3b63865
Stop asserting if we see unusually large discrepancies in the outgoing profile
flow from a block. Instead just make a note in the jit dump.
Fixesdotnet#77450.
@ghostghost added the area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI label Jan 30, 2023
@ghost

ghost commented Jan 30, 2023

Copy link
Copy Markdown

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch, @kunalspathak
See info in area-owners.md if you want to be subscribed.

Issue Details

Stop asserting if we see unusually large discrepancies in the outgoing profile flow from a block. Instead just make a note in the jit dump.

Fixes #77450.

Author:AndyAyersMS
Assignees:-
Labels:

area-CodeGen-coreclr

Milestone:-

@AndyAyersMS

AndyAyersMS commented Jan 30, 2023

Copy link
Copy Markdown
MemberAuthor

@EgorBo PTAL
cc @dotnet/jit-contrib

Note edge profile weights are slated to be revised during .NET 8, so this is just an interim fix to stop this assert from firing.

EgorBo
EgorBo approved these changes Jan 30, 2023
View reviewed changes

@EgorBoEgorBo left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wonder if we should have a special format for sort of warnings in the DUMP

@AndyAyersMS

AndyAyersMS commented Jan 31, 2023•
edited
Loading

Copy link
Copy Markdown
MemberAuthor

Failure is a crash in ILC building crossgen2. Likely unrelated since this PR is only changing an assert into a dump.

 crossgen2 -> /__w/1/s/artifacts/bin/coreclr/linux.arm64.Debug/crossgen2/crossgen2.dll
Generating native code
Segmentation fault (core dumped)
/__w/1/s/artifacts/transport/coreclr/build/Microsoft.NETCore.Native.targets(281,5): error MSB3073: The command ""/__w/1/s/artifacts/transport/coreclr/x64/ilc/ilc" @"/__w/1/s/artifacts/obj/coreclr/crossgen2/arm64/Debug/native/crossgen2.ilc.rsp"" exited with code 139. [/__w/1/s/src/coreclr/tools/aot/crossgen2/crossgen2.csproj]
##[error]artifacts/transport/coreclr/build/Microsoft.NETCore.Native.targets(281,5): error MSB3073: (NETCORE_ENGINEERING_TELEMETRY=Build) The command ""/__w/1/s/artifacts/transport/coreclr/x64/ilc/ilc" @"/__w/1/s/artifacts/obj/coreclr/crossgen2/arm64/Debug/native/crossgen2.ilc.rsp"" exited with code 139.

@AndyAyersMS

AndyAyersMS commented Jan 31, 2023

Copy link
Copy Markdown
MemberAuthor

Hmm, the failure is persistent. I am going to see if I can get a local repro.

@MichalStrehovsky is there any way to get crash dumps from CI when ILC fails like this? Or a stack trace?

@MichalStrehovsky

MichalStrehovsky commented Feb 1, 2023

Copy link
Copy Markdown
Member

@MichalStrehovsky is there any way to get crash dumps from CI when ILC fails like this? Or a stack trace?

That would be a question for @dotnet/runtime-infrastructure - I don't think we are able to collect crash dumps in build legs. This is using live-built ILC with live-built JIT, hosted on top of whatever CoreCLR was used to build the repo. Such capability would have been useful in the past, and not just when using live-built tools.

I'll trigger more thorough NativeAOT testing run just in case we can hit this on a less obscure configuration.

@MichalStrehovsky

MichalStrehovsky commented Feb 1, 2023

Copy link
Copy Markdown
Member

/azp run runtime-extra-platforms

@azure-pipelines

azure-pipelinesBot commented Feb 1, 2023

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

@AndyAyersMS

AndyAyersMS commented Feb 1, 2023

Copy link
Copy Markdown
MemberAuthor

I'll trigger more thorough NativeAOT testing run just in case we can hit this on a less obscure configuration.

Seems like lots of issues?

This change only affects jit internal diagnostics and is changing an assert into something that won't assert. So hard for me to see how it could cause any sort of failures.

@MichalStrehovsky

MichalStrehovsky commented Feb 1, 2023

Copy link
Copy Markdown
Member

I'll trigger more thorough NativeAOT testing run just in case we can hit this on a less obscure configuration.

Seems like lots of issues?

Yes, we had a regression (that is now fixed), but this would show up as a build break, not test break and those are easy to see. The only build break I see is in the Build windows-x64 Release NativeAOT_Checked_Libs leg and that one is #81460.

This change only affects jit internal diagnostics and is changing an assert into something that won't assert. So hard for me to see how it could cause any sort of failures.

I'm a bit worried if you say this happened on repeated run. I'm not aware of seeing this in other PRs. I guess we could merge this and if this starts happening everywhere, we can start suspecting something bad like a C++ codegen bug.

@AndyAyersMS

AndyAyersMS commented Feb 1, 2023

Copy link
Copy Markdown
MemberAuthor

Let me bounce this and see if it still repros with merged-up bits.

@AndyAyersMSAndyAyersMS reopened this Feb 1, 2023
@AndyAyersMS

AndyAyersMS commented Feb 2, 2023

Copy link
Copy Markdown
MemberAuthor

arm64 linux CI machines see to be mis-provisioned or something?

CMake Error at /usr/share/cmake-3.25/Modules/CMakeTestCCompiler.cmake:70 (message):
The C compiler
"/usr/bin/clang-9"
is not able to compile a simple test program.

@AndyAyersMSAndyAyersMS mentioned this pull request Feb 2, 2023
Closed
5 tasks
@AndyAyersMSAndyAyersMS reopened this Feb 2, 2023
@AndyAyersMS

AndyAyersMS commented Feb 2, 2023

Copy link
Copy Markdown
MemberAuthor

retry didn't fix the build issues, so bounced the PR once more.

@AndyAyersMS

AndyAyersMS commented Feb 2, 2023

Copy link
Copy Markdown
MemberAuthor

ILC failure didn't repro. Extra platform failures are unrelated (and mostly fixed elsewhere).

@AndyAyersMS
AndyAyersMS merged commit 6c9190c into dotnet:mainFeb 2, 2023
@MichalStrehovsky

MichalStrehovsky commented Feb 2, 2023•
edited
Loading

Copy link
Copy Markdown
Member

#81476 now hit the crossgen2 musl arm64 crossbuild crash that we observed here. The run started after this PR merged to main so it's still inconclusive as to whether this change triggered it. I've not seen this failure before. Cc @ivanpovazan

@AndyAyersMS

AndyAyersMS commented Feb 3, 2023

Copy link
Copy Markdown
MemberAuthor

Hopefully we can figure this out; let me know if I can help somehow.

@ivanpovazan

ivanpovazan commented Feb 3, 2023

Copy link
Copy Markdown
Member

#81476 now hit the crossgen2 musl arm64 crossbuild crash that we observed here. The run started after this PR merged to main so it's still inconclusive as to whether this change triggered it. I've not seen this failure before. Cc @ivanpovazan

After a rerun, the failure was not observed.

@ghostghost locked as resolved and limited conversation to collaborators Mar 5, 2023
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

@EgorBoEgorBoEgorBo approved these changes

Assignees

@AndyAyersMSAndyAyersMS

Labels

area-CodeGen-coreclrCLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[pgo] Assertion failed '((-slop) <= diff) && (diff <= slop)' during 'Compute edge weights (2, false)'

4 participants

@AndyAyersMS@MichalStrehovsky@ivanpovazan@EgorBo
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all 
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '

Skip to content
JIT: change profile slop assert to a jitdump note - #81377
Merged
AndyAyersMS merged 1 commit into
dotnet:mainfrom 
AndyAyersMS:ChangeSlopAssertToDiagnosticDump
Feb 2, 2023
Merged
JIT: change profile slop assert to a jitdump note#81377
AndyAyersMS merged 1 commit into
dotnet:mainfrom 
AndyAyersMS:ChangeSlopAssertToDiagnosticDump
Conversation
@AndyAyersMS
@AndyAyersMSAndyAyersMS
commented
Jan 30, 2023

Copy link


Copy Markdown

Member
Stop asserting if we see unusually large discrepancies in the outgoing profile flow from a block. Instead just make a note in the jit dump.
Fixes#77450.
@AndyAyersMS
JIT: change profile slop assert to a jitdump note
3b63865
Stop asserting if we see unusually large discrepancies in the outgoing profile
flow from a block. Instead just make a note in the jit dump.
Fixesdotnet#77450.
@ghostghost
added
the 
area-CodeGen-coreclr
CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI
label
Jan 30, 2023
@ghost
ghost
commented
Jan 30, 2023

Copy link


Copy Markdown

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch, @kunalspathak

See info in area-owners.md if you want to be subscribed.
Issue Details
Stop asserting if we see unusually large discrepancies in the outgoing profile flow from a block. Instead just make a note in the jit dump.
Fixes #77450.
Author:AndyAyersMS
Assignees:-
Labels:
area-CodeGen-coreclr
Milestone:-
@AndyAyersMS
AndyAyersMS
commented
Jan 30, 2023

Copy link


Copy Markdown

MemberAuthor
@EgorBo PTAL

cc @dotnet/jit-contrib
Note edge profile weights are slated to be revised during .NET 8, so this is just an interim fix to stop this assert from firing.
EgorBo
EgorBo
approved these changes
Jan 30, 2023
View reviewed changes
@EgorBoEgorBo
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wonder if we should have a special format for sort of warnings in the DUMP
@AndyAyersMS
AndyAyersMS
commented
Jan 31, 2023•

edited

Loading

Copy link


Copy Markdown

MemberAuthor
Failure is a crash in ILC building crossgen2. Likely unrelated since this PR is only changing an assert into a dump.
 crossgen2 -> /__w/1/s/artifacts/bin/coreclr/linux.arm64.Debug/crossgen2/crossgen2.dll
Generating native code
Segmentation fault (core dumped)
/__w/1/s/artifacts/transport/coreclr/build/Microsoft.NETCore.Native.targets(281,5): error MSB3073: The command ""/__w/1/s/artifacts/transport/coreclr/x64/ilc/ilc" @"/__w/1/s/artifacts/obj/coreclr/crossgen2/arm64/Debug/native/crossgen2.ilc.rsp"" exited with code 139. [/__w/1/s/src/coreclr/tools/aot/crossgen2/crossgen2.csproj]
##[error]artifacts/transport/coreclr/build/Microsoft.NETCore.Native.targets(281,5): error MSB3073: (NETCORE_ENGINEERING_TELEMETRY=Build) The command ""/__w/1/s/artifacts/transport/coreclr/x64/ilc/ilc" @"/__w/1/s/artifacts/obj/coreclr/crossgen2/arm64/Debug/native/crossgen2.ilc.rsp"" exited with code 139.

@AndyAyersMS
AndyAyersMS
commented
Jan 31, 2023

Copy link


Copy Markdown

MemberAuthor
Hmm, the failure is persistent. I am going to see if I can get a local repro.
@MichalStrehovsky is there any way to get crash dumps from CI when ILC fails like this? Or a stack trace?
@MichalStrehovsky
MichalStrehovsky
commented
Feb 1, 2023

Copy link


Copy Markdown

Member
@MichalStrehovsky is there any way to get crash dumps from CI when ILC fails like this? Or a stack trace?
That would be a question for @dotnet/runtime-infrastructure - I don't think we are able to collect crash dumps in build legs. This is using live-built ILC with live-built JIT, hosted on top of whatever CoreCLR was used to build the repo. Such capability would have been useful in the past, and not just when using live-built tools.
I'll trigger more thorough NativeAOT testing run just in case we can hit this on a less obscure configuration.
@MichalStrehovsky
MichalStrehovsky
commented
Feb 1, 2023

Copy link


Copy Markdown

Member
/azp run runtime-extra-platforms
@azure-pipelines
azure-pipelinesBot
commented
Feb 1, 2023

Copy link


Copy Markdown


Azure Pipelines successfully started running 1 pipeline(s).
@AndyAyersMS
AndyAyersMS
commented
Feb 1, 2023

Copy link


Copy Markdown

MemberAuthor
I'll trigger more thorough NativeAOT testing run just in case we can hit this on a less obscure configuration.
Seems like lots of issues?
This change only affects jit internal diagnostics and is changing an assert into something that won't assert. So hard for me to see how it could cause any sort of failures.
@MichalStrehovsky
MichalStrehovsky
commented
Feb 1, 2023

Copy link


Copy Markdown

Member
I'll trigger more thorough NativeAOT testing run just in case we can hit this on a less obscure configuration.
Seems like lots of issues?
Yes, we had a regression (that is now fixed), but this would show up as a build break, not test break and those are easy to see. The only build break I see is in the Build windows-x64 Release NativeAOT_Checked_Libs leg and that one is #81460.
This change only affects jit internal diagnostics and is changing an assert into something that won't assert. So hard for me to see how it could cause any sort of failures.
I'm a bit worried if you say this happened on repeated run. I'm not aware of seeing this in other PRs. I guess we could merge this and if this starts happening everywhere, we can start suspecting something bad like a C++ codegen bug.
@AndyAyersMS
AndyAyersMS
commented
Feb 1, 2023

Copy link


Copy Markdown

MemberAuthor
Let me bounce this and see if it still repros with merged-up bits.
@AndyAyersMSAndyAyersMS
reopened this
Feb 1, 2023
@AndyAyersMS
AndyAyersMS
commented
Feb 2, 2023

Copy link


Copy Markdown

MemberAuthor
arm64 linux CI machines see to be mis-provisioned or something?
CMake Error at /usr/share/cmake-3.25/Modules/CMakeTestCCompiler.cmake:70 (message):
The C compiler
"/usr/bin/clang-9"
is not able to compile a simple test program.

@AndyAyersMSAndyAyersMS
mentioned this pull request
Feb 2, 2023

Closed

5 tasks
@AndyAyersMSAndyAyersMS
reopened this
Feb 2, 2023
@AndyAyersMS
AndyAyersMS
commented
Feb 2, 2023

Copy link


Copy Markdown

MemberAuthor
retry didn't fix the build issues, so bounced the PR once more.
@AndyAyersMS
AndyAyersMS
commented
Feb 2, 2023

Copy link


Copy Markdown

MemberAuthor
ILC failure didn't repro. Extra platform failures are unrelated (and mostly fixed elsewhere).
@AndyAyersMS
AndyAyersMS
merged commit 6c9190c
into
dotnet:mainFeb 2, 2023
@MichalStrehovsky
MichalStrehovsky
commented
Feb 2, 2023•

edited

Loading

Copy link


Copy Markdown

Member
#81476 now hit the crossgen2 musl arm64 crossbuild crash that we observed here. The run started after this PR merged to main so it's still inconclusive as to whether this change triggered it. I've not seen this failure before. Cc @ivanpovazan
@AndyAyersMS
AndyAyersMS
commented
Feb 3, 2023

Copy link


Copy Markdown

MemberAuthor
Hopefully we can figure this out; let me know if I can help somehow.
@ivanpovazan
ivanpovazan
commented
Feb 3, 2023

Copy link


Copy Markdown

Member
#81476 now hit the crossgen2 musl arm64 crossbuild crash that we observed here. The run started after this PR merged to main so it's still inconclusive as to whether this change triggered it. I've not seen this failure before. Cc @ivanpovazan
After a rerun, the failure was not observed.
@ghostghost
locked as resolved and limited conversation to collaborators
Mar 5, 2023
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@EgorBoEgorBoEgorBo approved these changes

Assignees

@AndyAyersMSAndyAyersMS

Labels

area-CodeGen-coreclrCLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

[pgo] Assertion failed '((-slop) <= diff) && (diff <= slop)' during 'Compute edge weights (2, false)'


4 participants

@AndyAyersMS@MichalStrehovsky@ivanpovazan@EgorBo

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix");
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
JIT: change profile slop assert to a jitdump note - #81377
Merged
AndyAyersMS merged 1 commit into
dotnet:mainfrom 
AndyAyersMS:ChangeSlopAssertToDiagnosticDump
Feb 2, 2023
Merged
JIT: change profile slop assert to a jitdump note#81377
AndyAyersMS merged 1 commit into
dotnet:mainfrom 
AndyAyersMS:ChangeSlopAssertToDiagnosticDump
Conversation
@AndyAyersMS
@AndyAyersMSAndyAyersMS
commented
Jan 30, 2023

Copy link


Copy Markdown

Member
Stop asserting if we see unusually large discrepancies in the outgoing profile flow from a block. Instead just make a note in the jit dump.
Fixes#77450.
@AndyAyersMS
JIT: change profile slop assert to a jitdump note
3b63865
Stop asserting if we see unusually large discrepancies in the outgoing profile
flow from a block. Instead just make a note in the jit dump.
Fixesdotnet#77450.
@ghostghost
added
the 
area-CodeGen-coreclr
CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI
label
Jan 30, 2023
@ghost
ghost
commented
Jan 30, 2023

Copy link


Copy Markdown

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch, @kunalspathak

See info in area-owners.md if you want to be subscribed.
Issue Details
Stop asserting if we see unusually large discrepancies in the outgoing profile flow from a block. Instead just make a note in the jit dump.
Fixes #77450.
Author:AndyAyersMS
Assignees:-
Labels:
area-CodeGen-coreclr
Milestone:-
@AndyAyersMS
AndyAyersMS
commented
Jan 30, 2023

Copy link


Copy Markdown

MemberAuthor
@EgorBo PTAL

cc @dotnet/jit-contrib
Note edge profile weights are slated to be revised during .NET 8, so this is just an interim fix to stop this assert from firing.
EgorBo
EgorBo
approved these changes
Jan 30, 2023
View reviewed changes
@EgorBoEgorBo
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wonder if we should have a special format for sort of warnings in the DUMP
@AndyAyersMS
AndyAyersMS
commented
Jan 31, 2023•

edited

Loading

Copy link


Copy Markdown

MemberAuthor
Failure is a crash in ILC building crossgen2. Likely unrelated since this PR is only changing an assert into a dump.
 crossgen2 -> /__w/1/s/artifacts/bin/coreclr/linux.arm64.Debug/crossgen2/crossgen2.dll
Generating native code
Segmentation fault (core dumped)
/__w/1/s/artifacts/transport/coreclr/build/Microsoft.NETCore.Native.targets(281,5): error MSB3073: The command ""/__w/1/s/artifacts/transport/coreclr/x64/ilc/ilc" @"/__w/1/s/artifacts/obj/coreclr/crossgen2/arm64/Debug/native/crossgen2.ilc.rsp"" exited with code 139. [/__w/1/s/src/coreclr/tools/aot/crossgen2/crossgen2.csproj]
##[error]artifacts/transport/coreclr/build/Microsoft.NETCore.Native.targets(281,5): error MSB3073: (NETCORE_ENGINEERING_TELEMETRY=Build) The command ""/__w/1/s/artifacts/transport/coreclr/x64/ilc/ilc" @"/__w/1/s/artifacts/obj/coreclr/crossgen2/arm64/Debug/native/crossgen2.ilc.rsp"" exited with code 139.

@AndyAyersMS
AndyAyersMS
commented
Jan 31, 2023

Copy link


Copy Markdown

MemberAuthor
Hmm, the failure is persistent. I am going to see if I can get a local repro.
@MichalStrehovsky is there any way to get crash dumps from CI when ILC fails like this? Or a stack trace?
@MichalStrehovsky
MichalStrehovsky
commented
Feb 1, 2023

Copy link


Copy Markdown

Member
@MichalStrehovsky is there any way to get crash dumps from CI when ILC fails like this? Or a stack trace?
That would be a question for @dotnet/runtime-infrastructure - I don't think we are able to collect crash dumps in build legs. This is using live-built ILC with live-built JIT, hosted on top of whatever CoreCLR was used to build the repo. Such capability would have been useful in the past, and not just when using live-built tools.
I'll trigger more thorough NativeAOT testing run just in case we can hit this on a less obscure configuration.
@MichalStrehovsky
MichalStrehovsky
commented
Feb 1, 2023

Copy link


Copy Markdown

Member
/azp run runtime-extra-platforms
@azure-pipelines
azure-pipelinesBot
commented
Feb 1, 2023

Copy link


Copy Markdown


Azure Pipelines successfully started running 1 pipeline(s).
@AndyAyersMS
AndyAyersMS
commented
Feb 1, 2023

Copy link


Copy Markdown

MemberAuthor
I'll trigger more thorough NativeAOT testing run just in case we can hit this on a less obscure configuration.
Seems like lots of issues?
This change only affects jit internal diagnostics and is changing an assert into something that won't assert. So hard for me to see how it could cause any sort of failures.
@MichalStrehovsky
MichalStrehovsky
commented
Feb 1, 2023

Copy link


Copy Markdown

Member
I'll trigger more thorough NativeAOT testing run just in case we can hit this on a less obscure configuration.
Seems like lots of issues?
Yes, we had a regression (that is now fixed), but this would show up as a build break, not test break and those are easy to see. The only build break I see is in the Build windows-x64 Release NativeAOT_Checked_Libs leg and that one is #81460.
This change only affects jit internal diagnostics and is changing an assert into something that won't assert. So hard for me to see how it could cause any sort of failures.
I'm a bit worried if you say this happened on repeated run. I'm not aware of seeing this in other PRs. I guess we could merge this and if this starts happening everywhere, we can start suspecting something bad like a C++ codegen bug.
@AndyAyersMS
AndyAyersMS
commented
Feb 1, 2023

Copy link


Copy Markdown

MemberAuthor
Let me bounce this and see if it still repros with merged-up bits.
@AndyAyersMSAndyAyersMS
reopened this
Feb 1, 2023
@AndyAyersMS
AndyAyersMS
commented
Feb 2, 2023

Copy link


Copy Markdown

MemberAuthor
arm64 linux CI machines see to be mis-provisioned or something?
CMake Error at /usr/share/cmake-3.25/Modules/CMakeTestCCompiler.cmake:70 (message):
The C compiler
"/usr/bin/clang-9"
is not able to compile a simple test program.

@AndyAyersMSAndyAyersMS
mentioned this pull request
Feb 2, 2023

Closed

5 tasks
@AndyAyersMSAndyAyersMS
reopened this
Feb 2, 2023
@AndyAyersMS
AndyAyersMS
commented
Feb 2, 2023

Copy link


Copy Markdown

MemberAuthor
retry didn't fix the build issues, so bounced the PR once more.
@AndyAyersMS
AndyAyersMS
commented
Feb 2, 2023

Copy link


Copy Markdown

MemberAuthor
ILC failure didn't repro. Extra platform failures are unrelated (and mostly fixed elsewhere).
@AndyAyersMS
AndyAyersMS
merged commit 6c9190c
into
dotnet:mainFeb 2, 2023
@MichalStrehovsky
MichalStrehovsky
commented
Feb 2, 2023•

edited

Loading

Copy link


Copy Markdown

Member
#81476 now hit the crossgen2 musl arm64 crossbuild crash that we observed here. The run started after this PR merged to main so it's still inconclusive as to whether this change triggered it. I've not seen this failure before. Cc @ivanpovazan
@AndyAyersMS
AndyAyersMS
commented
Feb 3, 2023

Copy link


Copy Markdown

MemberAuthor
Hopefully we can figure this out; let me know if I can help somehow.
@ivanpovazan
ivanpovazan
commented
Feb 3, 2023

Copy link


Copy Markdown

Member
#81476 now hit the crossgen2 musl arm64 crossbuild crash that we observed here. The run started after this PR merged to main so it's still inconclusive as to whether this change triggered it. I've not seen this failure before. Cc @ivanpovazan
After a rerun, the failure was not observed.
@ghostghost
locked as resolved and limited conversation to collaborators
Mar 5, 2023
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@EgorBoEgorBoEgorBo approved these changes

Assignees

@AndyAyersMSAndyAyersMS

Labels

area-CodeGen-coreclrCLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

[pgo] Assertion failed '((-slop) <= diff) && (diff <= slop)' during 'Compute edge weights (2, false)'


4 participants

@AndyAyersMS@MichalStrehovsky@ivanpovazan@EgorBo

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms");
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
JIT: change profile slop assert to a jitdump note - #81377
Merged
AndyAyersMS merged 1 commit into
dotnet:mainfrom 
AndyAyersMS:ChangeSlopAssertToDiagnosticDump
Feb 2, 2023
Merged
JIT: change profile slop assert to a jitdump note#81377
AndyAyersMS merged 1 commit into
dotnet:mainfrom 
AndyAyersMS:ChangeSlopAssertToDiagnosticDump
Conversation
@AndyAyersMS
@AndyAyersMSAndyAyersMS
commented
Jan 30, 2023

Copy link


Copy Markdown

Member
Stop asserting if we see unusually large discrepancies in the outgoing profile flow from a block. Instead just make a note in the jit dump.
Fixes#77450.
@AndyAyersMS
JIT: change profile slop assert to a jitdump note
3b63865
Stop asserting if we see unusually large discrepancies in the outgoing profile
flow from a block. Instead just make a note in the jit dump.
Fixesdotnet#77450.
@ghostghost
added
the 
area-CodeGen-coreclr
CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI
label
Jan 30, 2023
@ghost
ghost
commented
Jan 30, 2023

Copy link


Copy Markdown

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch, @kunalspathak

See info in area-owners.md if you want to be subscribed.
Issue Details
Stop asserting if we see unusually large discrepancies in the outgoing profile flow from a block. Instead just make a note in the jit dump.
Fixes #77450.
Author:AndyAyersMS
Assignees:-
Labels:
area-CodeGen-coreclr
Milestone:-
@AndyAyersMS
AndyAyersMS
commented
Jan 30, 2023

Copy link


Copy Markdown

MemberAuthor
@EgorBo PTAL

cc @dotnet/jit-contrib
Note edge profile weights are slated to be revised during .NET 8, so this is just an interim fix to stop this assert from firing.
EgorBo
EgorBo
approved these changes
Jan 30, 2023
View reviewed changes
@EgorBoEgorBo
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wonder if we should have a special format for sort of warnings in the DUMP
@AndyAyersMS
AndyAyersMS
commented
Jan 31, 2023•

edited

Loading

Copy link


Copy Markdown

MemberAuthor
Failure is a crash in ILC building crossgen2. Likely unrelated since this PR is only changing an assert into a dump.
 crossgen2 -> /__w/1/s/artifacts/bin/coreclr/linux.arm64.Debug/crossgen2/crossgen2.dll
Generating native code
Segmentation fault (core dumped)
/__w/1/s/artifacts/transport/coreclr/build/Microsoft.NETCore.Native.targets(281,5): error MSB3073: The command ""/__w/1/s/artifacts/transport/coreclr/x64/ilc/ilc" @"/__w/1/s/artifacts/obj/coreclr/crossgen2/arm64/Debug/native/crossgen2.ilc.rsp"" exited with code 139. [/__w/1/s/src/coreclr/tools/aot/crossgen2/crossgen2.csproj]
##[error]artifacts/transport/coreclr/build/Microsoft.NETCore.Native.targets(281,5): error MSB3073: (NETCORE_ENGINEERING_TELEMETRY=Build) The command ""/__w/1/s/artifacts/transport/coreclr/x64/ilc/ilc" @"/__w/1/s/artifacts/obj/coreclr/crossgen2/arm64/Debug/native/crossgen2.ilc.rsp"" exited with code 139.

@AndyAyersMS
AndyAyersMS
commented
Jan 31, 2023

Copy link


Copy Markdown

MemberAuthor
Hmm, the failure is persistent. I am going to see if I can get a local repro.
@MichalStrehovsky is there any way to get crash dumps from CI when ILC fails like this? Or a stack trace?
@MichalStrehovsky
MichalStrehovsky
commented
Feb 1, 2023

Copy link


Copy Markdown

Member
@MichalStrehovsky is there any way to get crash dumps from CI when ILC fails like this? Or a stack trace?
That would be a question for @dotnet/runtime-infrastructure - I don't think we are able to collect crash dumps in build legs. This is using live-built ILC with live-built JIT, hosted on top of whatever CoreCLR was used to build the repo. Such capability would have been useful in the past, and not just when using live-built tools.
I'll trigger more thorough NativeAOT testing run just in case we can hit this on a less obscure configuration.
@MichalStrehovsky
MichalStrehovsky
commented
Feb 1, 2023

Copy link


Copy Markdown

Member
/azp run runtime-extra-platforms
@azure-pipelines
azure-pipelinesBot
commented
Feb 1, 2023

Copy link


Copy Markdown


Azure Pipelines successfully started running 1 pipeline(s).
@AndyAyersMS
AndyAyersMS
commented
Feb 1, 2023

Copy link


Copy Markdown

MemberAuthor
I'll trigger more thorough NativeAOT testing run just in case we can hit this on a less obscure configuration.
Seems like lots of issues?
This change only affects jit internal diagnostics and is changing an assert into something that won't assert. So hard for me to see how it could cause any sort of failures.
@MichalStrehovsky
MichalStrehovsky
commented
Feb 1, 2023

Copy link


Copy Markdown

Member
I'll trigger more thorough NativeAOT testing run just in case we can hit this on a less obscure configuration.
Seems like lots of issues?
Yes, we had a regression (that is now fixed), but this would show up as a build break, not test break and those are easy to see. The only build break I see is in the Build windows-x64 Release NativeAOT_Checked_Libs leg and that one is #81460.
This change only affects jit internal diagnostics and is changing an assert into something that won't assert. So hard for me to see how it could cause any sort of failures.
I'm a bit worried if you say this happened on repeated run. I'm not aware of seeing this in other PRs. I guess we could merge this and if this starts happening everywhere, we can start suspecting something bad like a C++ codegen bug.
@AndyAyersMS
AndyAyersMS
commented
Feb 1, 2023

Copy link


Copy Markdown

MemberAuthor
Let me bounce this and see if it still repros with merged-up bits.
@AndyAyersMSAndyAyersMS
reopened this
Feb 1, 2023
@AndyAyersMS
AndyAyersMS
commented
Feb 2, 2023

Copy link


Copy Markdown

MemberAuthor
arm64 linux CI machines see to be mis-provisioned or something?
CMake Error at /usr/share/cmake-3.25/Modules/CMakeTestCCompiler.cmake:70 (message):
The C compiler
"/usr/bin/clang-9"
is not able to compile a simple test program.

@AndyAyersMSAndyAyersMS
mentioned this pull request
Feb 2, 2023

Closed

5 tasks
@AndyAyersMSAndyAyersMS
reopened this
Feb 2, 2023
@AndyAyersMS
AndyAyersMS
commented
Feb 2, 2023

Copy link


Copy Markdown

MemberAuthor
retry didn't fix the build issues, so bounced the PR once more.
@AndyAyersMS
AndyAyersMS
commented
Feb 2, 2023

Copy link


Copy Markdown

MemberAuthor
ILC failure didn't repro. Extra platform failures are unrelated (and mostly fixed elsewhere).
@AndyAyersMS
AndyAyersMS
merged commit 6c9190c
into
dotnet:mainFeb 2, 2023
@MichalStrehovsky
MichalStrehovsky
commented
Feb 2, 2023•

edited

Loading

Copy link


Copy Markdown

Member
#81476 now hit the crossgen2 musl arm64 crossbuild crash that we observed here. The run started after this PR merged to main so it's still inconclusive as to whether this change triggered it. I've not seen this failure before. Cc @ivanpovazan
@AndyAyersMS
AndyAyersMS
commented
Feb 3, 2023

Copy link


Copy Markdown

MemberAuthor
Hopefully we can figure this out; let me know if I can help somehow.
@ivanpovazan
ivanpovazan
commented
Feb 3, 2023

Copy link


Copy Markdown

Member
#81476 now hit the crossgen2 musl arm64 crossbuild crash that we observed here. The run started after this PR merged to main so it's still inconclusive as to whether this change triggered it. I've not seen this failure before. Cc @ivanpovazan
After a rerun, the failure was not observed.
@ghostghost
locked as resolved and limited conversation to collaborators
Mar 5, 2023
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@EgorBoEgorBoEgorBo approved these changes

Assignees

@AndyAyersMSAndyAyersMS

Labels

area-CodeGen-coreclrCLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

[pgo] Assertion failed '((-slop) <= diff) && (diff <= slop)' during 'Compute edge weights (2, false)'


4 participants

@AndyAyersMS@MichalStrehovsky@ivanpovazan@EgorBo

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links");
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '

Skip to content
JIT: change profile slop assert to a jitdump note - #81377
Merged
AndyAyersMS merged 1 commit into
dotnet:mainfrom 
AndyAyersMS:ChangeSlopAssertToDiagnosticDump
Feb 2, 2023
Merged
JIT: change profile slop assert to a jitdump note#81377
AndyAyersMS merged 1 commit into
dotnet:mainfrom 
AndyAyersMS:ChangeSlopAssertToDiagnosticDump
Conversation
@AndyAyersMS
@AndyAyersMSAndyAyersMS
commented
Jan 30, 2023

Copy link


Copy Markdown

Member
Stop asserting if we see unusually large discrepancies in the outgoing profile flow from a block. Instead just make a note in the jit dump.
Fixes#77450.
@AndyAyersMS
JIT: change profile slop assert to a jitdump note
3b63865
Stop asserting if we see unusually large discrepancies in the outgoing profile
flow from a block. Instead just make a note in the jit dump.
Fixesdotnet#77450.
@ghostghost
added
the 
area-CodeGen-coreclr
CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI
label
Jan 30, 2023
@ghost
ghost
commented
Jan 30, 2023

Copy link


Copy Markdown

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch, @kunalspathak

See info in area-owners.md if you want to be subscribed.
Issue Details
Stop asserting if we see unusually large discrepancies in the outgoing profile flow from a block. Instead just make a note in the jit dump.
Fixes #77450.
Author:AndyAyersMS
Assignees:-
Labels:
area-CodeGen-coreclr
Milestone:-
@AndyAyersMS
AndyAyersMS
commented
Jan 30, 2023

Copy link


Copy Markdown

MemberAuthor
@EgorBo PTAL

cc @dotnet/jit-contrib
Note edge profile weights are slated to be revised during .NET 8, so this is just an interim fix to stop this assert from firing.
EgorBo
EgorBo
approved these changes
Jan 30, 2023
View reviewed changes
@EgorBoEgorBo
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wonder if we should have a special format for sort of warnings in the DUMP
@AndyAyersMS
AndyAyersMS
commented
Jan 31, 2023•

edited

Loading

Copy link


Copy Markdown

MemberAuthor
Failure is a crash in ILC building crossgen2. Likely unrelated since this PR is only changing an assert into a dump.
 crossgen2 -> /__w/1/s/artifacts/bin/coreclr/linux.arm64.Debug/crossgen2/crossgen2.dll
Generating native code
Segmentation fault (core dumped)
/__w/1/s/artifacts/transport/coreclr/build/Microsoft.NETCore.Native.targets(281,5): error MSB3073: The command ""/__w/1/s/artifacts/transport/coreclr/x64/ilc/ilc" @"/__w/1/s/artifacts/obj/coreclr/crossgen2/arm64/Debug/native/crossgen2.ilc.rsp"" exited with code 139. [/__w/1/s/src/coreclr/tools/aot/crossgen2/crossgen2.csproj]
##[error]artifacts/transport/coreclr/build/Microsoft.NETCore.Native.targets(281,5): error MSB3073: (NETCORE_ENGINEERING_TELEMETRY=Build) The command ""/__w/1/s/artifacts/transport/coreclr/x64/ilc/ilc" @"/__w/1/s/artifacts/obj/coreclr/crossgen2/arm64/Debug/native/crossgen2.ilc.rsp"" exited with code 139.

@AndyAyersMS
AndyAyersMS
commented
Jan 31, 2023

Copy link


Copy Markdown

MemberAuthor
Hmm, the failure is persistent. I am going to see if I can get a local repro.
@MichalStrehovsky is there any way to get crash dumps from CI when ILC fails like this? Or a stack trace?
@MichalStrehovsky
MichalStrehovsky
commented
Feb 1, 2023

Copy link


Copy Markdown

Member
@MichalStrehovsky is there any way to get crash dumps from CI when ILC fails like this? Or a stack trace?
That would be a question for @dotnet/runtime-infrastructure - I don't think we are able to collect crash dumps in build legs. This is using live-built ILC with live-built JIT, hosted on top of whatever CoreCLR was used to build the repo. Such capability would have been useful in the past, and not just when using live-built tools.
I'll trigger more thorough NativeAOT testing run just in case we can hit this on a less obscure configuration.
@MichalStrehovsky
MichalStrehovsky
commented
Feb 1, 2023

Copy link


Copy Markdown

Member
/azp run runtime-extra-platforms
@azure-pipelines
azure-pipelinesBot
commented
Feb 1, 2023

Copy link


Copy Markdown


Azure Pipelines successfully started running 1 pipeline(s).
@AndyAyersMS
AndyAyersMS
commented
Feb 1, 2023

Copy link


Copy Markdown

MemberAuthor
I'll trigger more thorough NativeAOT testing run just in case we can hit this on a less obscure configuration.
Seems like lots of issues?
This change only affects jit internal diagnostics and is changing an assert into something that won't assert. So hard for me to see how it could cause any sort of failures.
@MichalStrehovsky
MichalStrehovsky
commented
Feb 1, 2023

Copy link


Copy Markdown

Member
I'll trigger more thorough NativeAOT testing run just in case we can hit this on a less obscure configuration.
Seems like lots of issues?
Yes, we had a regression (that is now fixed), but this would show up as a build break, not test break and those are easy to see. The only build break I see is in the Build windows-x64 Release NativeAOT_Checked_Libs leg and that one is #81460.
This change only affects jit internal diagnostics and is changing an assert into something that won't assert. So hard for me to see how it could cause any sort of failures.
I'm a bit worried if you say this happened on repeated run. I'm not aware of seeing this in other PRs. I guess we could merge this and if this starts happening everywhere, we can start suspecting something bad like a C++ codegen bug.
@AndyAyersMS
AndyAyersMS
commented
Feb 1, 2023

Copy link


Copy Markdown

MemberAuthor
Let me bounce this and see if it still repros with merged-up bits.
@AndyAyersMSAndyAyersMS
reopened this
Feb 1, 2023
@AndyAyersMS
AndyAyersMS
commented
Feb 2, 2023

Copy link


Copy Markdown

MemberAuthor
arm64 linux CI machines see to be mis-provisioned or something?
CMake Error at /usr/share/cmake-3.25/Modules/CMakeTestCCompiler.cmake:70 (message):
The C compiler
"/usr/bin/clang-9"
is not able to compile a simple test program.

@AndyAyersMSAndyAyersMS
mentioned this pull request
Feb 2, 2023

Closed

5 tasks
@AndyAyersMSAndyAyersMS
reopened this
Feb 2, 2023
@AndyAyersMS
AndyAyersMS
commented
Feb 2, 2023

Copy link


Copy Markdown

MemberAuthor
retry didn't fix the build issues, so bounced the PR once more.
@AndyAyersMS
AndyAyersMS
commented
Feb 2, 2023

Copy link


Copy Markdown

MemberAuthor
ILC failure didn't repro. Extra platform failures are unrelated (and mostly fixed elsewhere).
@AndyAyersMS
AndyAyersMS
merged commit 6c9190c
into
dotnet:mainFeb 2, 2023
@MichalStrehovsky
MichalStrehovsky
commented
Feb 2, 2023•

edited

Loading

Copy link


Copy Markdown

Member
#81476 now hit the crossgen2 musl arm64 crossbuild crash that we observed here. The run started after this PR merged to main so it's still inconclusive as to whether this change triggered it. I've not seen this failure before. Cc @ivanpovazan
@AndyAyersMS
AndyAyersMS
commented
Feb 3, 2023

Copy link


Copy Markdown

MemberAuthor
Hopefully we can figure this out; let me know if I can help somehow.
@ivanpovazan
ivanpovazan
commented
Feb 3, 2023

Copy link


Copy Markdown

Member
#81476 now hit the crossgen2 musl arm64 crossbuild crash that we observed here. The run started after this PR merged to main so it's still inconclusive as to whether this change triggered it. I've not seen this failure before. Cc @ivanpovazan
After a rerun, the failure was not observed.
@ghostghost
locked as resolved and limited conversation to collaborators
Mar 5, 2023
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@EgorBoEgorBoEgorBo approved these changes

Assignees

@AndyAyersMSAndyAyersMS

Labels

area-CodeGen-coreclrCLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

[pgo] Assertion failed '((-slop) <= diff) && (diff <= slop)' during 'Compute edge weights (2, false)'


4 participants

@AndyAyersMS@MichalStrehovsky@ivanpovazan@EgorBo

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default");
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
JIT: change profile slop assert to a jitdump note - #81377
Merged
AndyAyersMS merged 1 commit into
dotnet:mainfrom 
AndyAyersMS:ChangeSlopAssertToDiagnosticDump
Feb 2, 2023
Merged
JIT: change profile slop assert to a jitdump note#81377
AndyAyersMS merged 1 commit into
dotnet:mainfrom 
AndyAyersMS:ChangeSlopAssertToDiagnosticDump
Conversation
@AndyAyersMS
@AndyAyersMSAndyAyersMS
commented
Jan 30, 2023

Copy link


Copy Markdown

Member
Stop asserting if we see unusually large discrepancies in the outgoing profile flow from a block. Instead just make a note in the jit dump.
Fixes#77450.
@AndyAyersMS
JIT: change profile slop assert to a jitdump note
3b63865
Stop asserting if we see unusually large discrepancies in the outgoing profile
flow from a block. Instead just make a note in the jit dump.
Fixesdotnet#77450.
@ghostghost
added
the 
area-CodeGen-coreclr
CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI
label
Jan 30, 2023
@ghost
ghost
commented
Jan 30, 2023

Copy link


Copy Markdown

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch, @kunalspathak

See info in area-owners.md if you want to be subscribed.
Issue Details
Stop asserting if we see unusually large discrepancies in the outgoing profile flow from a block. Instead just make a note in the jit dump.
Fixes #77450.
Author:AndyAyersMS
Assignees:-
Labels:
area-CodeGen-coreclr
Milestone:-
@AndyAyersMS
AndyAyersMS
commented
Jan 30, 2023

Copy link


Copy Markdown

MemberAuthor
@EgorBo PTAL

cc @dotnet/jit-contrib
Note edge profile weights are slated to be revised during .NET 8, so this is just an interim fix to stop this assert from firing.
EgorBo
EgorBo
approved these changes
Jan 30, 2023
View reviewed changes
@EgorBoEgorBo
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wonder if we should have a special format for sort of warnings in the DUMP
@AndyAyersMS
AndyAyersMS
commented
Jan 31, 2023•

edited

Loading

Copy link


Copy Markdown

MemberAuthor
Failure is a crash in ILC building crossgen2. Likely unrelated since this PR is only changing an assert into a dump.
 crossgen2 -> /__w/1/s/artifacts/bin/coreclr/linux.arm64.Debug/crossgen2/crossgen2.dll
Generating native code
Segmentation fault (core dumped)
/__w/1/s/artifacts/transport/coreclr/build/Microsoft.NETCore.Native.targets(281,5): error MSB3073: The command ""/__w/1/s/artifacts/transport/coreclr/x64/ilc/ilc" @"/__w/1/s/artifacts/obj/coreclr/crossgen2/arm64/Debug/native/crossgen2.ilc.rsp"" exited with code 139. [/__w/1/s/src/coreclr/tools/aot/crossgen2/crossgen2.csproj]
##[error]artifacts/transport/coreclr/build/Microsoft.NETCore.Native.targets(281,5): error MSB3073: (NETCORE_ENGINEERING_TELEMETRY=Build) The command ""/__w/1/s/artifacts/transport/coreclr/x64/ilc/ilc" @"/__w/1/s/artifacts/obj/coreclr/crossgen2/arm64/Debug/native/crossgen2.ilc.rsp"" exited with code 139.

@AndyAyersMS
AndyAyersMS
commented
Jan 31, 2023

Copy link


Copy Markdown

MemberAuthor
Hmm, the failure is persistent. I am going to see if I can get a local repro.
@MichalStrehovsky is there any way to get crash dumps from CI when ILC fails like this? Or a stack trace?
@MichalStrehovsky
MichalStrehovsky
commented
Feb 1, 2023

Copy link


Copy Markdown

Member
@MichalStrehovsky is there any way to get crash dumps from CI when ILC fails like this? Or a stack trace?
That would be a question for @dotnet/runtime-infrastructure - I don't think we are able to collect crash dumps in build legs. This is using live-built ILC with live-built JIT, hosted on top of whatever CoreCLR was used to build the repo. Such capability would have been useful in the past, and not just when using live-built tools.
I'll trigger more thorough NativeAOT testing run just in case we can hit this on a less obscure configuration.
@MichalStrehovsky
MichalStrehovsky
commented
Feb 1, 2023

Copy link


Copy Markdown

Member
/azp run runtime-extra-platforms
@azure-pipelines
azure-pipelinesBot
commented
Feb 1, 2023

Copy link


Copy Markdown


Azure Pipelines successfully started running 1 pipeline(s).
@AndyAyersMS
AndyAyersMS
commented
Feb 1, 2023

Copy link


Copy Markdown

MemberAuthor
I'll trigger more thorough NativeAOT testing run just in case we can hit this on a less obscure configuration.
Seems like lots of issues?
This change only affects jit internal diagnostics and is changing an assert into something that won't assert. So hard for me to see how it could cause any sort of failures.
@MichalStrehovsky
MichalStrehovsky
commented
Feb 1, 2023

Copy link


Copy Markdown

Member
I'll trigger more thorough NativeAOT testing run just in case we can hit this on a less obscure configuration.
Seems like lots of issues?
Yes, we had a regression (that is now fixed), but this would show up as a build break, not test break and those are easy to see. The only build break I see is in the Build windows-x64 Release NativeAOT_Checked_Libs leg and that one is #81460.
This change only affects jit internal diagnostics and is changing an assert into something that won't assert. So hard for me to see how it could cause any sort of failures.
I'm a bit worried if you say this happened on repeated run. I'm not aware of seeing this in other PRs. I guess we could merge this and if this starts happening everywhere, we can start suspecting something bad like a C++ codegen bug.
@AndyAyersMS
AndyAyersMS
commented
Feb 1, 2023

Copy link


Copy Markdown

MemberAuthor
Let me bounce this and see if it still repros with merged-up bits.
@AndyAyersMSAndyAyersMS
reopened this
Feb 1, 2023
@AndyAyersMS
AndyAyersMS
commented
Feb 2, 2023

Copy link


Copy Markdown

MemberAuthor
arm64 linux CI machines see to be mis-provisioned or something?
CMake Error at /usr/share/cmake-3.25/Modules/CMakeTestCCompiler.cmake:70 (message):
The C compiler
"/usr/bin/clang-9"
is not able to compile a simple test program.

@AndyAyersMSAndyAyersMS
mentioned this pull request
Feb 2, 2023

Closed

5 tasks
@AndyAyersMSAndyAyersMS
reopened this
Feb 2, 2023
@AndyAyersMS
AndyAyersMS
commented
Feb 2, 2023

Copy link


Copy Markdown

MemberAuthor
retry didn't fix the build issues, so bounced the PR once more.
@AndyAyersMS
AndyAyersMS
commented
Feb 2, 2023

Copy link


Copy Markdown

MemberAuthor
ILC failure didn't repro. Extra platform failures are unrelated (and mostly fixed elsewhere).
@AndyAyersMS
AndyAyersMS
merged commit 6c9190c
into
dotnet:mainFeb 2, 2023
@MichalStrehovsky
MichalStrehovsky
commented
Feb 2, 2023•

edited

Loading

Copy link


Copy Markdown

Member
#81476 now hit the crossgen2 musl arm64 crossbuild crash that we observed here. The run started after this PR merged to main so it's still inconclusive as to whether this change triggered it. I've not seen this failure before. Cc @ivanpovazan
@AndyAyersMS
AndyAyersMS
commented
Feb 3, 2023

Copy link


Copy Markdown

MemberAuthor
Hopefully we can figure this out; let me know if I can help somehow.
@ivanpovazan
ivanpovazan
commented
Feb 3, 2023

Copy link


Copy Markdown

Member
#81476 now hit the crossgen2 musl arm64 crossbuild crash that we observed here. The run started after this PR merged to main so it's still inconclusive as to whether this change triggered it. I've not seen this failure before. Cc @ivanpovazan
After a rerun, the failure was not observed.
@ghostghost
locked as resolved and limited conversation to collaborators
Mar 5, 2023
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@EgorBoEgorBoEgorBo approved these changes

Assignees

@AndyAyersMSAndyAyersMS

Labels

area-CodeGen-coreclrCLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

[pgo] Assertion failed '((-slop) <= diff) && (diff <= slop)' during 'Compute edge weights (2, false)'


4 participants

@AndyAyersMS@MichalStrehovsky@ivanpovazan@EgorBo

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers");
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
JIT: change profile slop assert to a jitdump note - #81377
Merged
AndyAyersMS merged 1 commit into
dotnet:mainfrom 
AndyAyersMS:ChangeSlopAssertToDiagnosticDump
Feb 2, 2023
Merged
JIT: change profile slop assert to a jitdump note#81377
AndyAyersMS merged 1 commit into
dotnet:mainfrom 
AndyAyersMS:ChangeSlopAssertToDiagnosticDump
Conversation
@AndyAyersMS
@AndyAyersMSAndyAyersMS
commented
Jan 30, 2023

Copy link


Copy Markdown

Member
Stop asserting if we see unusually large discrepancies in the outgoing profile flow from a block. Instead just make a note in the jit dump.
Fixes#77450.
@AndyAyersMS
JIT: change profile slop assert to a jitdump note
3b63865
Stop asserting if we see unusually large discrepancies in the outgoing profile
flow from a block. Instead just make a note in the jit dump.
Fixesdotnet#77450.
@ghostghost
added
the 
area-CodeGen-coreclr
CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI
label
Jan 30, 2023
@ghost
ghost
commented
Jan 30, 2023

Copy link


Copy Markdown

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch, @kunalspathak

See info in area-owners.md if you want to be subscribed.
Issue Details
Stop asserting if we see unusually large discrepancies in the outgoing profile flow from a block. Instead just make a note in the jit dump.
Fixes #77450.
Author:AndyAyersMS
Assignees:-
Labels:
area-CodeGen-coreclr
Milestone:-
@AndyAyersMS
AndyAyersMS
commented
Jan 30, 2023

Copy link


Copy Markdown

MemberAuthor
@EgorBo PTAL

cc @dotnet/jit-contrib
Note edge profile weights are slated to be revised during .NET 8, so this is just an interim fix to stop this assert from firing.
EgorBo
EgorBo
approved these changes
Jan 30, 2023
View reviewed changes
@EgorBoEgorBo
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wonder if we should have a special format for sort of warnings in the DUMP
@AndyAyersMS
AndyAyersMS
commented
Jan 31, 2023•

edited

Loading

Copy link


Copy Markdown

MemberAuthor
Failure is a crash in ILC building crossgen2. Likely unrelated since this PR is only changing an assert into a dump.
 crossgen2 -> /__w/1/s/artifacts/bin/coreclr/linux.arm64.Debug/crossgen2/crossgen2.dll
Generating native code
Segmentation fault (core dumped)
/__w/1/s/artifacts/transport/coreclr/build/Microsoft.NETCore.Native.targets(281,5): error MSB3073: The command ""/__w/1/s/artifacts/transport/coreclr/x64/ilc/ilc" @"/__w/1/s/artifacts/obj/coreclr/crossgen2/arm64/Debug/native/crossgen2.ilc.rsp"" exited with code 139. [/__w/1/s/src/coreclr/tools/aot/crossgen2/crossgen2.csproj]
##[error]artifacts/transport/coreclr/build/Microsoft.NETCore.Native.targets(281,5): error MSB3073: (NETCORE_ENGINEERING_TELEMETRY=Build) The command ""/__w/1/s/artifacts/transport/coreclr/x64/ilc/ilc" @"/__w/1/s/artifacts/obj/coreclr/crossgen2/arm64/Debug/native/crossgen2.ilc.rsp"" exited with code 139.

@AndyAyersMS
AndyAyersMS
commented
Jan 31, 2023

Copy link


Copy Markdown

MemberAuthor
Hmm, the failure is persistent. I am going to see if I can get a local repro.
@MichalStrehovsky is there any way to get crash dumps from CI when ILC fails like this? Or a stack trace?
@MichalStrehovsky
MichalStrehovsky
commented
Feb 1, 2023

Copy link


Copy Markdown

Member
@MichalStrehovsky is there any way to get crash dumps from CI when ILC fails like this? Or a stack trace?
That would be a question for @dotnet/runtime-infrastructure - I don't think we are able to collect crash dumps in build legs. This is using live-built ILC with live-built JIT, hosted on top of whatever CoreCLR was used to build the repo. Such capability would have been useful in the past, and not just when using live-built tools.
I'll trigger more thorough NativeAOT testing run just in case we can hit this on a less obscure configuration.
@MichalStrehovsky
MichalStrehovsky
commented
Feb 1, 2023

Copy link


Copy Markdown

Member
/azp run runtime-extra-platforms
@azure-pipelines
azure-pipelinesBot
commented
Feb 1, 2023

Copy link


Copy Markdown


Azure Pipelines successfully started running 1 pipeline(s).
@AndyAyersMS
AndyAyersMS
commented
Feb 1, 2023

Copy link


Copy Markdown

MemberAuthor
I'll trigger more thorough NativeAOT testing run just in case we can hit this on a less obscure configuration.
Seems like lots of issues?
This change only affects jit internal diagnostics and is changing an assert into something that won't assert. So hard for me to see how it could cause any sort of failures.
@MichalStrehovsky
MichalStrehovsky
commented
Feb 1, 2023

Copy link


Copy Markdown

Member
I'll trigger more thorough NativeAOT testing run just in case we can hit this on a less obscure configuration.
Seems like lots of issues?
Yes, we had a regression (that is now fixed), but this would show up as a build break, not test break and those are easy to see. The only build break I see is in the Build windows-x64 Release NativeAOT_Checked_Libs leg and that one is #81460.
This change only affects jit internal diagnostics and is changing an assert into something that won't assert. So hard for me to see how it could cause any sort of failures.
I'm a bit worried if you say this happened on repeated run. I'm not aware of seeing this in other PRs. I guess we could merge this and if this starts happening everywhere, we can start suspecting something bad like a C++ codegen bug.
@AndyAyersMS
AndyAyersMS
commented
Feb 1, 2023

Copy link


Copy Markdown

MemberAuthor
Let me bounce this and see if it still repros with merged-up bits.
@AndyAyersMSAndyAyersMS
reopened this
Feb 1, 2023
@AndyAyersMS
AndyAyersMS
commented
Feb 2, 2023

Copy link


Copy Markdown

MemberAuthor
arm64 linux CI machines see to be mis-provisioned or something?
CMake Error at /usr/share/cmake-3.25/Modules/CMakeTestCCompiler.cmake:70 (message):
The C compiler
"/usr/bin/clang-9"
is not able to compile a simple test program.

@AndyAyersMSAndyAyersMS
mentioned this pull request
Feb 2, 2023

Closed

5 tasks
@AndyAyersMSAndyAyersMS
reopened this
Feb 2, 2023
@AndyAyersMS
AndyAyersMS
commented
Feb 2, 2023

Copy link


Copy Markdown

MemberAuthor
retry didn't fix the build issues, so bounced the PR once more.
@AndyAyersMS
AndyAyersMS
commented
Feb 2, 2023

Copy link


Copy Markdown

MemberAuthor
ILC failure didn't repro. Extra platform failures are unrelated (and mostly fixed elsewhere).
@AndyAyersMS
AndyAyersMS
merged commit 6c9190c
into
dotnet:mainFeb 2, 2023
@MichalStrehovsky
MichalStrehovsky
commented
Feb 2, 2023•

edited

Loading

Copy link


Copy Markdown

Member
#81476 now hit the crossgen2 musl arm64 crossbuild crash that we observed here. The run started after this PR merged to main so it's still inconclusive as to whether this change triggered it. I've not seen this failure before. Cc @ivanpovazan
@AndyAyersMS
AndyAyersMS
commented
Feb 3, 2023

Copy link


Copy Markdown

MemberAuthor
Hopefully we can figure this out; let me know if I can help somehow.
@ivanpovazan
ivanpovazan
commented
Feb 3, 2023

Copy link


Copy Markdown

Member
#81476 now hit the crossgen2 musl arm64 crossbuild crash that we observed here. The run started after this PR merged to main so it's still inconclusive as to whether this change triggered it. I've not seen this failure before. Cc @ivanpovazan
After a rerun, the failure was not observed.
@ghostghost
locked as resolved and limited conversation to collaborators
Mar 5, 2023
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@EgorBoEgorBoEgorBo approved these changes

Assignees

@AndyAyersMSAndyAyersMS

Labels

area-CodeGen-coreclrCLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

[pgo] Assertion failed '((-slop) <= diff) && (diff <= slop)' during 'Compute edge weights (2, false)'


4 participants

@AndyAyersMS@MichalStrehovsky@ivanpovazan@EgorBo

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode");
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();

Skip to content
JIT: change profile slop assert to a jitdump note - #81377
Merged
AndyAyersMS merged 1 commit into
dotnet:mainfrom 
AndyAyersMS:ChangeSlopAssertToDiagnosticDump
Feb 2, 2023
Merged
JIT: change profile slop assert to a jitdump note#81377
AndyAyersMS merged 1 commit into
dotnet:mainfrom 
AndyAyersMS:ChangeSlopAssertToDiagnosticDump
Conversation
@AndyAyersMS
@AndyAyersMSAndyAyersMS
commented
Jan 30, 2023

Copy link


Copy Markdown

Member
Stop asserting if we see unusually large discrepancies in the outgoing profile flow from a block. Instead just make a note in the jit dump.
Fixes#77450.
@AndyAyersMS
JIT: change profile slop assert to a jitdump note
3b63865
Stop asserting if we see unusually large discrepancies in the outgoing profile
flow from a block. Instead just make a note in the jit dump.
Fixesdotnet#77450.
@ghostghost
added
the 
area-CodeGen-coreclr
CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI
label
Jan 30, 2023
@ghost
ghost
commented
Jan 30, 2023

Copy link


Copy Markdown

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch, @kunalspathak

See info in area-owners.md if you want to be subscribed.
Issue Details
Stop asserting if we see unusually large discrepancies in the outgoing profile flow from a block. Instead just make a note in the jit dump.
Fixes #77450.
Author:AndyAyersMS
Assignees:-
Labels:
area-CodeGen-coreclr
Milestone:-
@AndyAyersMS
AndyAyersMS
commented
Jan 30, 2023

Copy link


Copy Markdown

MemberAuthor
@EgorBo PTAL

cc @dotnet/jit-contrib
Note edge profile weights are slated to be revised during .NET 8, so this is just an interim fix to stop this assert from firing.
EgorBo
EgorBo
approved these changes
Jan 30, 2023
View reviewed changes
@EgorBoEgorBo
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wonder if we should have a special format for sort of warnings in the DUMP
@AndyAyersMS
AndyAyersMS
commented
Jan 31, 2023•

edited

Loading

Copy link


Copy Markdown

MemberAuthor
Failure is a crash in ILC building crossgen2. Likely unrelated since this PR is only changing an assert into a dump.
 crossgen2 -> /__w/1/s/artifacts/bin/coreclr/linux.arm64.Debug/crossgen2/crossgen2.dll
Generating native code
Segmentation fault (core dumped)
/__w/1/s/artifacts/transport/coreclr/build/Microsoft.NETCore.Native.targets(281,5): error MSB3073: The command ""/__w/1/s/artifacts/transport/coreclr/x64/ilc/ilc" @"/__w/1/s/artifacts/obj/coreclr/crossgen2/arm64/Debug/native/crossgen2.ilc.rsp"" exited with code 139. [/__w/1/s/src/coreclr/tools/aot/crossgen2/crossgen2.csproj]
##[error]artifacts/transport/coreclr/build/Microsoft.NETCore.Native.targets(281,5): error MSB3073: (NETCORE_ENGINEERING_TELEMETRY=Build) The command ""/__w/1/s/artifacts/transport/coreclr/x64/ilc/ilc" @"/__w/1/s/artifacts/obj/coreclr/crossgen2/arm64/Debug/native/crossgen2.ilc.rsp"" exited with code 139.

@AndyAyersMS
AndyAyersMS
commented
Jan 31, 2023

Copy link


Copy Markdown

MemberAuthor
Hmm, the failure is persistent. I am going to see if I can get a local repro.
@MichalStrehovsky is there any way to get crash dumps from CI when ILC fails like this? Or a stack trace?
@MichalStrehovsky
MichalStrehovsky
commented
Feb 1, 2023

Copy link


Copy Markdown

Member
@MichalStrehovsky is there any way to get crash dumps from CI when ILC fails like this? Or a stack trace?
That would be a question for @dotnet/runtime-infrastructure - I don't think we are able to collect crash dumps in build legs. This is using live-built ILC with live-built JIT, hosted on top of whatever CoreCLR was used to build the repo. Such capability would have been useful in the past, and not just when using live-built tools.
I'll trigger more thorough NativeAOT testing run just in case we can hit this on a less obscure configuration.
@MichalStrehovsky
MichalStrehovsky
commented
Feb 1, 2023

Copy link


Copy Markdown

Member
/azp run runtime-extra-platforms
@azure-pipelines
azure-pipelinesBot
commented
Feb 1, 2023

Copy link


Copy Markdown


Azure Pipelines successfully started running 1 pipeline(s).
@AndyAyersMS
AndyAyersMS
commented
Feb 1, 2023

Copy link


Copy Markdown

MemberAuthor
I'll trigger more thorough NativeAOT testing run just in case we can hit this on a less obscure configuration.
Seems like lots of issues?
This change only affects jit internal diagnostics and is changing an assert into something that won't assert. So hard for me to see how it could cause any sort of failures.
@MichalStrehovsky
MichalStrehovsky
commented
Feb 1, 2023

Copy link


Copy Markdown

Member
I'll trigger more thorough NativeAOT testing run just in case we can hit this on a less obscure configuration.
Seems like lots of issues?
Yes, we had a regression (that is now fixed), but this would show up as a build break, not test break and those are easy to see. The only build break I see is in the Build windows-x64 Release NativeAOT_Checked_Libs leg and that one is #81460.
This change only affects jit internal diagnostics and is changing an assert into something that won't assert. So hard for me to see how it could cause any sort of failures.
I'm a bit worried if you say this happened on repeated run. I'm not aware of seeing this in other PRs. I guess we could merge this and if this starts happening everywhere, we can start suspecting something bad like a C++ codegen bug.
@AndyAyersMS
AndyAyersMS
commented
Feb 1, 2023

Copy link


Copy Markdown

MemberAuthor
Let me bounce this and see if it still repros with merged-up bits.
@AndyAyersMSAndyAyersMS
reopened this
Feb 1, 2023
@AndyAyersMS
AndyAyersMS
commented
Feb 2, 2023

Copy link


Copy Markdown

MemberAuthor
arm64 linux CI machines see to be mis-provisioned or something?
CMake Error at /usr/share/cmake-3.25/Modules/CMakeTestCCompiler.cmake:70 (message):
The C compiler
"/usr/bin/clang-9"
is not able to compile a simple test program.

@AndyAyersMSAndyAyersMS
mentioned this pull request
Feb 2, 2023

Closed

5 tasks
@AndyAyersMSAndyAyersMS
reopened this
Feb 2, 2023
@AndyAyersMS
AndyAyersMS
commented
Feb 2, 2023

Copy link


Copy Markdown

MemberAuthor
retry didn't fix the build issues, so bounced the PR once more.
@AndyAyersMS
AndyAyersMS
commented
Feb 2, 2023

Copy link


Copy Markdown

MemberAuthor
ILC failure didn't repro. Extra platform failures are unrelated (and mostly fixed elsewhere).
@AndyAyersMS
AndyAyersMS
merged commit 6c9190c
into
dotnet:mainFeb 2, 2023
@MichalStrehovsky
MichalStrehovsky
commented
Feb 2, 2023•

edited

Loading

Copy link


Copy Markdown

Member
#81476 now hit the crossgen2 musl arm64 crossbuild crash that we observed here. The run started after this PR merged to main so it's still inconclusive as to whether this change triggered it. I've not seen this failure before. Cc @ivanpovazan
@AndyAyersMS
AndyAyersMS
commented
Feb 3, 2023

Copy link


Copy Markdown

MemberAuthor
Hopefully we can figure this out; let me know if I can help somehow.
@ivanpovazan
ivanpovazan
commented
Feb 3, 2023

Copy link


Copy Markdown

Member
#81476 now hit the crossgen2 musl arm64 crossbuild crash that we observed here. The run started after this PR merged to main so it's still inconclusive as to whether this change triggered it. I've not seen this failure before. Cc @ivanpovazan
After a rerun, the failure was not observed.
@ghostghost
locked as resolved and limited conversation to collaborators
Mar 5, 2023
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@EgorBoEgorBoEgorBo approved these changes

Assignees

@AndyAyersMSAndyAyersMS

Labels

area-CodeGen-coreclrCLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

[pgo] Assertion failed '((-slop) <= diff) && (diff <= slop)' during 'Compute edge weights (2, false)'


4 participants

@AndyAyersMS@MichalStrehovsky@ivanpovazan@EgorBo