Fix published crossgen2 crash - #107447

Closed
clamp03 wants to merge 1 commit into
dotnet:mainfrom
clamp03:crossgen2
Closed

Fix published crossgen2 crash#107447
clamp03 wants to merge 1 commit into
dotnet:mainfrom
clamp03:crossgen2

Conversation

@clamp03

@clamp03clamp03 commented Sep 6, 2024

Copy link
Copy Markdown
Member

If test is built as Debug or Checked without LibrariesConfiguration, LibrariesConfiguration is set to default Release configuration.
So when it builds crossgen2, it uses debug c/c++ codes with non-debug libraries. It makes some mismatching and crossgen2 crashes in some tests.
For example, ParentMethodTableOffset in MethodTable has different values in methodtable.h and RuntimeHelpers.CoreCLR.cs because debug mode has debug_m_szClassName field and release mode doesn't have.

cc @dotnet/samsung

@ghostghost added the area-Infrastructure-coreclr Only use for closed issues label Sep 6, 2024
@dotnet-policy-servicedotnet-policy-serviceBot added the community-contribution Indicates that the PR has been added by a community member label Sep 6, 2024
@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @hoyosjs
See info in area-owners.md if you want to be subscribed.

@clamp03

Copy link
Copy Markdown
MemberAuthor

@jkotas@am11 After #106965, some tests fail when it compiles with crossgen2 in tests on RISC-V. I found reasons and made this PR. However, I do not know about build and I think I did in a wrong way. Could you please help? Thank you.

@clamp03clamp03 self-assigned this Sep 6, 2024
@clamp03
clamp03 requested review from am11 and jkotasSeptember 6, 2024 07:23
@am11

am11 commented Sep 6, 2024

Copy link
Copy Markdown
Member

@clamp03, libraries configurations are always default to Release. If you pass LibrariesConfiguraiton from command line it is overridden. That PR did not change that mechanism. Can you share the command which my PR regressed in your workflow? I tested the whole matrix, including riscv64: https://github.com/am11/CrossRepoCITesting/actions/runs/10541101133/workflow#L21.

@clamp03

clamp03 commented Sep 6, 2024

Copy link
Copy Markdown
MemberAuthor

@clamp03, libraries configurations are always default to Release. If you pass LibrariesConfiguraiton from command line it is overridden. That PR did not change that mechanism. Can you share the command which my PR regressed in your workflow? I tested the whole matrix, including riscv64: https://github.com/am11/CrossRepoCITesting/actions/runs/10541101133/workflow#L21.

Thank you! I checked it doesn't fail if I give LibrariesConfiguation well. However, if I don't give, it builds well and it crashes in tests. I think it is better to make test successfully if it is possible to build without LibraraiesConfiguations. I don't think your PR is wrong. It is just appeared after your PR.

This is how I test (without this PR).

Build

$ ROOTFS_DIR=<PATH TO RISCV64> ./build.sh -cross -arch riscv64 -c Checked -s clr+libs
$ ROOTFS_DIR=<PATH TO RISCV64> ./build.sh -cross -arch riscv64 -c Release -s libs
$ export BuildAllTestsAsStandalone=true
$ ROOTFS_DIR=<PATH TO RISCV64> ./src/tests/build.sh -riscv64 -Checked -priority1 -cross -p:UseLocalAppHostPack=true

Test on RISC-V

$ ./crossgen2
Segmentation fault (core dumped)

@am11

am11 commented Sep 6, 2024

Copy link
Copy Markdown
Member

Thanks.

$ ./crossgen2
Segmentation fault (core dumped)

Is it the one from Core_Root directory? Before that PR, we were using dotnet crossgen2.dll, PR changed it to use crossgen2(1), so we test what is shipped in crossgen2 nuget package. For non-Release libraries configuration in tests, the supported workflow is pass -p:LibrariesConfiguration from command line: #80154 (comment). So it should work with Release libraries and Checked runtime. If it is not working, that's most likely a product bug?

@clamp03

Copy link
Copy Markdown
MemberAuthor

Is it the one from Core_Root directory? Before that PR, we were using dotnet crossgen2.dll, PR changed it to use crossgen2(1), so we test what is shipped in crossgen2 nuget package.

I used runtime/artifacts/tests/coreclr/linux.riscv64.Checked/Tests/Core_Root/crossgen2/crossgen2 which is built from test build.

For non-Release libraries configuration in tests, the supported workflow is pass -p:LibrariesConfiguration from command line: #80154 (comment). So it should work with Release libraries and Checked runtime. If it is not working, that's most likely a product bug?

I see. You already discussed with @jakobbotsch Thank you so much for the sharing. If we have to pass LibrariesConfiguation for non-Release, it also looks good to me.
Then I just wonder "if we have to pass LibrariesConfiguation for non-Release" and "if it is not passed for non-Release, crossgen2 crashes." then, do we have to set Libraries to Release as a default in test build? And do you know how to change lib configuaration only for crossgen2 in test build?

This is detail about what I investigated.
In Checked runtime, debug_m_szClassesName is included in methodtable.

#ifdef _DEBUG
LPCUTF8 debug_m_szClassName;
#endif//_DEBUG
PTR_MethodTable m_pParentMethodTable;

So offset to m_pParentMethodTable is 0x18 in RISC-V.

In Release lib, ParentMethodTableOffset is 0x10 because lib dosesn't have DEBUG definition and DebugClassNamePtr is 0.

privateconstintDebugClassNamePtr=// adjust for debug_m_szClassName
#if DEBUG
#if TARGET_64BIT
8
#else
4
#endif
#else
0
#endif
;
privateconstintParentMethodTableOffset=0x10+DebugClassNamePtr;
#if TARGET_64BIT
privateconstintAuxiliaryDataOffset=0x20+DebugClassNamePtr;
#else
privateconstintAuxiliaryDataOffset=0x18+DebugClassNamePtr;

Thank you!!!

@am11

am11 commented Sep 6, 2024

Copy link
Copy Markdown
Member

Could you try this patch: main...am11:runtime:patch-11 (delete artifacts/ dir before the rebuild)?

@clamp03

Copy link
Copy Markdown
MemberAuthor

@am11 Thank you. I will try and let you know the result.

@clamp03

Copy link
Copy Markdown
MemberAuthor

I tested with your patch. However, it still fails with the same reason (It crashes at the same machine code location).
I should get off work now. I am so sorry. Have a good weekend.

@clamp03

Copy link
Copy Markdown
MemberAuthor

If it is necessary to give LibrariesConfiguration for non-release test build, I close this PR.
@dotnet/samsung Please give LibrariesConfiguration for non-release test build on your CI as well as your build to prevent crashes.
Thank you.

@clamp03clamp03 closed this Sep 9, 2024
@am11

am11 commented Sep 9, 2024

Copy link
Copy Markdown
Member

Yup, this is the right approach. Currently, LibrariesConfiguration=Release is intentional. When/if we integrate runtime/src/tests/build.sh with top-level runtime/build.sh, we may end up announcing/updating the dev workflow.

See #75033; same change which was ultimately rejected due to sensitive nature of these workflows.

@github-actionsgithub-actionsBot locked and limited conversation to collaborators Oct 10, 2024
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

area-Infrastructure-coreclrOnly use for closed issuescommunity-contributionIndicates that the PR has been added by a community member

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants

@clamp03@am11
, 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all
 blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

Fix published crossgen2 crash - #107447

Closed
clamp03 wants to merge 1 commit into
dotnet:mainfrom
clamp03:crossgen2
Closed

Fix published crossgen2 crash#107447
clamp03 wants to merge 1 commit into
dotnet:mainfrom
clamp03:crossgen2

Conversation

@clamp03

@clamp03clamp03 commented Sep 6, 2024

Copy link
Copy Markdown
Member

If test is built as Debug or Checked without LibrariesConfiguration, LibrariesConfiguration is set to default Release configuration.
So when it builds crossgen2, it uses debug c/c++ codes with non-debug libraries. It makes some mismatching and crossgen2 crashes in some tests.
For example, ParentMethodTableOffset in MethodTable has different values in methodtable.h and RuntimeHelpers.CoreCLR.cs because debug mode has debug_m_szClassName field and release mode doesn't have.

cc @dotnet/samsung

@ghostghost added the area-Infrastructure-coreclr Only use for closed issues label Sep 6, 2024
@dotnet-policy-servicedotnet-policy-serviceBot added the community-contribution Indicates that the PR has been added by a community member label Sep 6, 2024
@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @hoyosjs
See info in area-owners.md if you want to be subscribed.

@clamp03

Copy link
Copy Markdown
MemberAuthor

@jkotas@am11 After #106965, some tests fail when it compiles with crossgen2 in tests on RISC-V. I found reasons and made this PR. However, I do not know about build and I think I did in a wrong way. Could you please help? Thank you.

@clamp03clamp03 self-assigned this Sep 6, 2024
@clamp03
clamp03 requested review from am11 and jkotasSeptember 6, 2024 07:23
@am11

am11 commented Sep 6, 2024

Copy link
Copy Markdown
Member

@clamp03, libraries configurations are always default to Release. If you pass LibrariesConfiguraiton from command line it is overridden. That PR did not change that mechanism. Can you share the command which my PR regressed in your workflow? I tested the whole matrix, including riscv64: https://github.com/am11/CrossRepoCITesting/actions/runs/10541101133/workflow#L21.

@clamp03

clamp03 commented Sep 6, 2024

Copy link
Copy Markdown
MemberAuthor

@clamp03, libraries configurations are always default to Release. If you pass LibrariesConfiguraiton from command line it is overridden. That PR did not change that mechanism. Can you share the command which my PR regressed in your workflow? I tested the whole matrix, including riscv64: https://github.com/am11/CrossRepoCITesting/actions/runs/10541101133/workflow#L21.

Thank you! I checked it doesn't fail if I give LibrariesConfiguation well. However, if I don't give, it builds well and it crashes in tests. I think it is better to make test successfully if it is possible to build without LibraraiesConfiguations. I don't think your PR is wrong. It is just appeared after your PR.

This is how I test (without this PR).

Build

$ ROOTFS_DIR=<PATH TO RISCV64> ./build.sh -cross -arch riscv64 -c Checked -s clr+libs
$ ROOTFS_DIR=<PATH TO RISCV64> ./build.sh -cross -arch riscv64 -c Release -s libs
$ export BuildAllTestsAsStandalone=true
$ ROOTFS_DIR=<PATH TO RISCV64> ./src/tests/build.sh -riscv64 -Checked -priority1 -cross -p:UseLocalAppHostPack=true

Test on RISC-V

$ ./crossgen2
Segmentation fault (core dumped)

@am11

am11 commented Sep 6, 2024

Copy link
Copy Markdown
Member

Thanks.

$ ./crossgen2
Segmentation fault (core dumped)

Is it the one from Core_Root directory? Before that PR, we were using dotnet crossgen2.dll, PR changed it to use crossgen2(1), so we test what is shipped in crossgen2 nuget package. For non-Release libraries configuration in tests, the supported workflow is pass -p:LibrariesConfiguration from command line: #80154 (comment). So it should work with Release libraries and Checked runtime. If it is not working, that's most likely a product bug?

@clamp03

Copy link
Copy Markdown
MemberAuthor

Is it the one from Core_Root directory? Before that PR, we were using dotnet crossgen2.dll, PR changed it to use crossgen2(1), so we test what is shipped in crossgen2 nuget package.

I used runtime/artifacts/tests/coreclr/linux.riscv64.Checked/Tests/Core_Root/crossgen2/crossgen2 which is built from test build.

For non-Release libraries configuration in tests, the supported workflow is pass -p:LibrariesConfiguration from command line: #80154 (comment). So it should work with Release libraries and Checked runtime. If it is not working, that's most likely a product bug?

I see. You already discussed with @jakobbotsch Thank you so much for the sharing. If we have to pass LibrariesConfiguation for non-Release, it also looks good to me.
Then I just wonder "if we have to pass LibrariesConfiguation for non-Release" and "if it is not passed for non-Release, crossgen2 crashes." then, do we have to set Libraries to Release as a default in test build? And do you know how to change lib configuaration only for crossgen2 in test build?

This is detail about what I investigated.
In Checked runtime, debug_m_szClassesName is included in methodtable.

#ifdef _DEBUG
LPCUTF8 debug_m_szClassName;
#endif//_DEBUG
PTR_MethodTable m_pParentMethodTable;

So offset to m_pParentMethodTable is 0x18 in RISC-V.

In Release lib, ParentMethodTableOffset is 0x10 because lib dosesn't have DEBUG definition and DebugClassNamePtr is 0.

privateconstintDebugClassNamePtr=// adjust for debug_m_szClassName
#if DEBUG
#if TARGET_64BIT
8
#else
4
#endif
#else
0
#endif
;
privateconstintParentMethodTableOffset=0x10+DebugClassNamePtr;
#if TARGET_64BIT
privateconstintAuxiliaryDataOffset=0x20+DebugClassNamePtr;
#else
privateconstintAuxiliaryDataOffset=0x18+DebugClassNamePtr;

Thank you!!!

@am11

am11 commented Sep 6, 2024

Copy link
Copy Markdown
Member

Could you try this patch: main...am11:runtime:patch-11 (delete artifacts/ dir before the rebuild)?

@clamp03

Copy link
Copy Markdown
MemberAuthor

@am11 Thank you. I will try and let you know the result.

@clamp03

Copy link
Copy Markdown
MemberAuthor

I tested with your patch. However, it still fails with the same reason (It crashes at the same machine code location).
I should get off work now. I am so sorry. Have a good weekend.

@clamp03

Copy link
Copy Markdown
MemberAuthor

If it is necessary to give LibrariesConfiguration for non-release test build, I close this PR.
@dotnet/samsung Please give LibrariesConfiguration for non-release test build on your CI as well as your build to prevent crashes.
Thank you.

@clamp03clamp03 closed this Sep 9, 2024
@am11

am11 commented Sep 9, 2024

Copy link
Copy Markdown
Member

Yup, this is the right approach. Currently, LibrariesConfiguration=Release is intentional. When/if we integrate runtime/src/tests/build.sh with top-level runtime/build.sh, we may end up announcing/updating the dev workflow.

See #75033; same change which was ultimately rejected due to sensitive nature of these workflows.

@github-actionsgithub-actionsBot locked and limited conversation to collaborators Oct 10, 2024
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

area-Infrastructure-coreclrOnly use for closed issuescommunity-contributionIndicates that the PR has been added by a community member

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants

@clamp03@am11
, 'i'); if (__m === '*' || __re.test(location.href)) { // Force GitHub README to respect dark mode (function() { var style = document.createElement('style'); style.textContent = ' .markdown-body { color-scheme: dark light; } .markdown-body pre { background: #161b22 !important; } .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; } .markdown-body table th, .markdown-body table td { border-color: #30363d !important; } .markdown-body img { background: #0d1117; } .markdown-body blockquote { border-left-color: #8b949e; } .markdown-body hr { border-color: #30363d; } '; document.head.appendChild(style); })(); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Fix published crossgen2 crash - #107447

Closed
clamp03 wants to merge 1 commit into
dotnet:mainfrom
clamp03:crossgen2
Closed

Fix published crossgen2 crash#107447
clamp03 wants to merge 1 commit into
dotnet:mainfrom
clamp03:crossgen2

Conversation

@clamp03

@clamp03clamp03 commented Sep 6, 2024

Copy link
Copy Markdown
Member

If test is built as Debug or Checked without LibrariesConfiguration, LibrariesConfiguration is set to default Release configuration.
So when it builds crossgen2, it uses debug c/c++ codes with non-debug libraries. It makes some mismatching and crossgen2 crashes in some tests.
For example, ParentMethodTableOffset in MethodTable has different values in methodtable.h and RuntimeHelpers.CoreCLR.cs because debug mode has debug_m_szClassName field and release mode doesn't have.

cc @dotnet/samsung

@ghostghost added the area-Infrastructure-coreclr Only use for closed issues label Sep 6, 2024
@dotnet-policy-servicedotnet-policy-serviceBot added the community-contribution Indicates that the PR has been added by a community member label Sep 6, 2024
@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @hoyosjs
See info in area-owners.md if you want to be subscribed.

@clamp03

Copy link
Copy Markdown
MemberAuthor

@jkotas@am11 After #106965, some tests fail when it compiles with crossgen2 in tests on RISC-V. I found reasons and made this PR. However, I do not know about build and I think I did in a wrong way. Could you please help? Thank you.

@clamp03clamp03 self-assigned this Sep 6, 2024
@clamp03
clamp03 requested review from am11 and jkotasSeptember 6, 2024 07:23
@am11

am11 commented Sep 6, 2024

Copy link
Copy Markdown
Member

@clamp03, libraries configurations are always default to Release. If you pass LibrariesConfiguraiton from command line it is overridden. That PR did not change that mechanism. Can you share the command which my PR regressed in your workflow? I tested the whole matrix, including riscv64: https://github.com/am11/CrossRepoCITesting/actions/runs/10541101133/workflow#L21.

@clamp03

clamp03 commented Sep 6, 2024

Copy link
Copy Markdown
MemberAuthor

@clamp03, libraries configurations are always default to Release. If you pass LibrariesConfiguraiton from command line it is overridden. That PR did not change that mechanism. Can you share the command which my PR regressed in your workflow? I tested the whole matrix, including riscv64: https://github.com/am11/CrossRepoCITesting/actions/runs/10541101133/workflow#L21.

Thank you! I checked it doesn't fail if I give LibrariesConfiguation well. However, if I don't give, it builds well and it crashes in tests. I think it is better to make test successfully if it is possible to build without LibraraiesConfiguations. I don't think your PR is wrong. It is just appeared after your PR.

This is how I test (without this PR).

Build

$ ROOTFS_DIR=<PATH TO RISCV64> ./build.sh -cross -arch riscv64 -c Checked -s clr+libs
$ ROOTFS_DIR=<PATH TO RISCV64> ./build.sh -cross -arch riscv64 -c Release -s libs
$ export BuildAllTestsAsStandalone=true
$ ROOTFS_DIR=<PATH TO RISCV64> ./src/tests/build.sh -riscv64 -Checked -priority1 -cross -p:UseLocalAppHostPack=true

Test on RISC-V

$ ./crossgen2
Segmentation fault (core dumped)

@am11

am11 commented Sep 6, 2024

Copy link
Copy Markdown
Member

Thanks.

$ ./crossgen2
Segmentation fault (core dumped)

Is it the one from Core_Root directory? Before that PR, we were using dotnet crossgen2.dll, PR changed it to use crossgen2(1), so we test what is shipped in crossgen2 nuget package. For non-Release libraries configuration in tests, the supported workflow is pass -p:LibrariesConfiguration from command line: #80154 (comment). So it should work with Release libraries and Checked runtime. If it is not working, that's most likely a product bug?

@clamp03

Copy link
Copy Markdown
MemberAuthor

Is it the one from Core_Root directory? Before that PR, we were using dotnet crossgen2.dll, PR changed it to use crossgen2(1), so we test what is shipped in crossgen2 nuget package.

I used runtime/artifacts/tests/coreclr/linux.riscv64.Checked/Tests/Core_Root/crossgen2/crossgen2 which is built from test build.

For non-Release libraries configuration in tests, the supported workflow is pass -p:LibrariesConfiguration from command line: #80154 (comment). So it should work with Release libraries and Checked runtime. If it is not working, that's most likely a product bug?

I see. You already discussed with @jakobbotsch Thank you so much for the sharing. If we have to pass LibrariesConfiguation for non-Release, it also looks good to me.
Then I just wonder "if we have to pass LibrariesConfiguation for non-Release" and "if it is not passed for non-Release, crossgen2 crashes." then, do we have to set Libraries to Release as a default in test build? And do you know how to change lib configuaration only for crossgen2 in test build?

This is detail about what I investigated.
In Checked runtime, debug_m_szClassesName is included in methodtable.

#ifdef _DEBUG
LPCUTF8 debug_m_szClassName;
#endif//_DEBUG
PTR_MethodTable m_pParentMethodTable;

So offset to m_pParentMethodTable is 0x18 in RISC-V.

In Release lib, ParentMethodTableOffset is 0x10 because lib dosesn't have DEBUG definition and DebugClassNamePtr is 0.

privateconstintDebugClassNamePtr=// adjust for debug_m_szClassName
#if DEBUG
#if TARGET_64BIT
8
#else
4
#endif
#else
0
#endif
;
privateconstintParentMethodTableOffset=0x10+DebugClassNamePtr;
#if TARGET_64BIT
privateconstintAuxiliaryDataOffset=0x20+DebugClassNamePtr;
#else
privateconstintAuxiliaryDataOffset=0x18+DebugClassNamePtr;

Thank you!!!

@am11

am11 commented Sep 6, 2024

Copy link
Copy Markdown
Member

Could you try this patch: main...am11:runtime:patch-11 (delete artifacts/ dir before the rebuild)?

@clamp03

Copy link
Copy Markdown
MemberAuthor

@am11 Thank you. I will try and let you know the result.

@clamp03

Copy link
Copy Markdown
MemberAuthor

I tested with your patch. However, it still fails with the same reason (It crashes at the same machine code location).
I should get off work now. I am so sorry. Have a good weekend.

@clamp03

Copy link
Copy Markdown
MemberAuthor

If it is necessary to give LibrariesConfiguration for non-release test build, I close this PR.
@dotnet/samsung Please give LibrariesConfiguration for non-release test build on your CI as well as your build to prevent crashes.
Thank you.

@clamp03clamp03 closed this Sep 9, 2024
@am11

am11 commented Sep 9, 2024

Copy link
Copy Markdown
Member

Yup, this is the right approach. Currently, LibrariesConfiguration=Release is intentional. When/if we integrate runtime/src/tests/build.sh with top-level runtime/build.sh, we may end up announcing/updating the dev workflow.

See #75033; same change which was ultimately rejected due to sensitive nature of these workflows.

@github-actionsgithub-actionsBot locked and limited conversation to collaborators Oct 10, 2024
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

area-Infrastructure-coreclrOnly use for closed issuescommunity-contributionIndicates that the PR has been added by a community member

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants

@clamp03@am11
, 'i'); if (__m === '*' || __re.test(location.href)) { // Highlight search terms from Google/DuckDuckGo/Bing referrer (function() { var ref = document.referrer; var terms = []; if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) { var url = new URL(ref); var q = url.searchParams.get('q') || url.searchParams.get('p'); if (q) { terms = q.split(/\s+/).filter(function(t) { return t.length > 2; }); } } if (terms.length === 0) return; var style = document.createElement('style'); style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }'; document.head.appendChild(style); function highlight(node) { if (node.nodeType === 3) { // text node var text = node.textContent; var found = false; terms.forEach(function(term) { var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi'); if (regex.test(text)) { found = true; var frag = document.createDocumentFragment(); var parts = text.split(regex); parts.forEach(function(part, i) { if (i % 2 === 0) { frag.appendChild(document.createTextNode(part)); } else { var span = document.createElement('span'); span.className = 'userscript-highlight'; span.textContent = part; frag.appendChild(span); } }); node.parentNode.replaceChild(frag, node); } }); } else if (node.nodeType === 1 && node.childNodes) { // element var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT']; if (!skipTags.includes(node.tagName)) { Array.from(node.childNodes).forEach(highlight); } } } highlight(document.body); // Re-highlight on dynamic content var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1 || node.nodeType === 3) highlight(node); }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Fix published crossgen2 crash - #107447

Closed
clamp03 wants to merge 1 commit into
dotnet:mainfrom
clamp03:crossgen2
Closed

Fix published crossgen2 crash#107447
clamp03 wants to merge 1 commit into
dotnet:mainfrom
clamp03:crossgen2

Conversation

@clamp03

@clamp03clamp03 commented Sep 6, 2024

Copy link
Copy Markdown
Member

If test is built as Debug or Checked without LibrariesConfiguration, LibrariesConfiguration is set to default Release configuration.
So when it builds crossgen2, it uses debug c/c++ codes with non-debug libraries. It makes some mismatching and crossgen2 crashes in some tests.
For example, ParentMethodTableOffset in MethodTable has different values in methodtable.h and RuntimeHelpers.CoreCLR.cs because debug mode has debug_m_szClassName field and release mode doesn't have.

cc @dotnet/samsung

@ghostghost added the area-Infrastructure-coreclr Only use for closed issues label Sep 6, 2024
@dotnet-policy-servicedotnet-policy-serviceBot added the community-contribution Indicates that the PR has been added by a community member label Sep 6, 2024
@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @hoyosjs
See info in area-owners.md if you want to be subscribed.

@clamp03

Copy link
Copy Markdown
MemberAuthor

@jkotas@am11 After #106965, some tests fail when it compiles with crossgen2 in tests on RISC-V. I found reasons and made this PR. However, I do not know about build and I think I did in a wrong way. Could you please help? Thank you.

@clamp03clamp03 self-assigned this Sep 6, 2024
@clamp03
clamp03 requested review from am11 and jkotasSeptember 6, 2024 07:23
@am11

am11 commented Sep 6, 2024

Copy link
Copy Markdown
Member

@clamp03, libraries configurations are always default to Release. If you pass LibrariesConfiguraiton from command line it is overridden. That PR did not change that mechanism. Can you share the command which my PR regressed in your workflow? I tested the whole matrix, including riscv64: https://github.com/am11/CrossRepoCITesting/actions/runs/10541101133/workflow#L21.

@clamp03

clamp03 commented Sep 6, 2024

Copy link
Copy Markdown
MemberAuthor

@clamp03, libraries configurations are always default to Release. If you pass LibrariesConfiguraiton from command line it is overridden. That PR did not change that mechanism. Can you share the command which my PR regressed in your workflow? I tested the whole matrix, including riscv64: https://github.com/am11/CrossRepoCITesting/actions/runs/10541101133/workflow#L21.

Thank you! I checked it doesn't fail if I give LibrariesConfiguation well. However, if I don't give, it builds well and it crashes in tests. I think it is better to make test successfully if it is possible to build without LibraraiesConfiguations. I don't think your PR is wrong. It is just appeared after your PR.

This is how I test (without this PR).

Build

$ ROOTFS_DIR=<PATH TO RISCV64> ./build.sh -cross -arch riscv64 -c Checked -s clr+libs
$ ROOTFS_DIR=<PATH TO RISCV64> ./build.sh -cross -arch riscv64 -c Release -s libs
$ export BuildAllTestsAsStandalone=true
$ ROOTFS_DIR=<PATH TO RISCV64> ./src/tests/build.sh -riscv64 -Checked -priority1 -cross -p:UseLocalAppHostPack=true

Test on RISC-V

$ ./crossgen2
Segmentation fault (core dumped)

@am11

am11 commented Sep 6, 2024

Copy link
Copy Markdown
Member

Thanks.

$ ./crossgen2
Segmentation fault (core dumped)

Is it the one from Core_Root directory? Before that PR, we were using dotnet crossgen2.dll, PR changed it to use crossgen2(1), so we test what is shipped in crossgen2 nuget package. For non-Release libraries configuration in tests, the supported workflow is pass -p:LibrariesConfiguration from command line: #80154 (comment). So it should work with Release libraries and Checked runtime. If it is not working, that's most likely a product bug?

@clamp03

Copy link
Copy Markdown
MemberAuthor

Is it the one from Core_Root directory? Before that PR, we were using dotnet crossgen2.dll, PR changed it to use crossgen2(1), so we test what is shipped in crossgen2 nuget package.

I used runtime/artifacts/tests/coreclr/linux.riscv64.Checked/Tests/Core_Root/crossgen2/crossgen2 which is built from test build.

For non-Release libraries configuration in tests, the supported workflow is pass -p:LibrariesConfiguration from command line: #80154 (comment). So it should work with Release libraries and Checked runtime. If it is not working, that's most likely a product bug?

I see. You already discussed with @jakobbotsch Thank you so much for the sharing. If we have to pass LibrariesConfiguation for non-Release, it also looks good to me.
Then I just wonder "if we have to pass LibrariesConfiguation for non-Release" and "if it is not passed for non-Release, crossgen2 crashes." then, do we have to set Libraries to Release as a default in test build? And do you know how to change lib configuaration only for crossgen2 in test build?

This is detail about what I investigated.
In Checked runtime, debug_m_szClassesName is included in methodtable.

#ifdef _DEBUG
LPCUTF8 debug_m_szClassName;
#endif//_DEBUG
PTR_MethodTable m_pParentMethodTable;

So offset to m_pParentMethodTable is 0x18 in RISC-V.

In Release lib, ParentMethodTableOffset is 0x10 because lib dosesn't have DEBUG definition and DebugClassNamePtr is 0.

privateconstintDebugClassNamePtr=// adjust for debug_m_szClassName
#if DEBUG
#if TARGET_64BIT
8
#else
4
#endif
#else
0
#endif
;
privateconstintParentMethodTableOffset=0x10+DebugClassNamePtr;
#if TARGET_64BIT
privateconstintAuxiliaryDataOffset=0x20+DebugClassNamePtr;
#else
privateconstintAuxiliaryDataOffset=0x18+DebugClassNamePtr;

Thank you!!!

@am11

am11 commented Sep 6, 2024

Copy link
Copy Markdown
Member

Could you try this patch: main...am11:runtime:patch-11 (delete artifacts/ dir before the rebuild)?

@clamp03

Copy link
Copy Markdown
MemberAuthor

@am11 Thank you. I will try and let you know the result.

@clamp03

Copy link
Copy Markdown
MemberAuthor

I tested with your patch. However, it still fails with the same reason (It crashes at the same machine code location).
I should get off work now. I am so sorry. Have a good weekend.

@clamp03

Copy link
Copy Markdown
MemberAuthor

If it is necessary to give LibrariesConfiguration for non-release test build, I close this PR.
@dotnet/samsung Please give LibrariesConfiguration for non-release test build on your CI as well as your build to prevent crashes.
Thank you.

@clamp03clamp03 closed this Sep 9, 2024
@am11

am11 commented Sep 9, 2024

Copy link
Copy Markdown
Member

Yup, this is the right approach. Currently, LibrariesConfiguration=Release is intentional. When/if we integrate runtime/src/tests/build.sh with top-level runtime/build.sh, we may end up announcing/updating the dev workflow.

See #75033; same change which was ultimately rejected due to sensitive nature of these workflows.

@github-actionsgithub-actionsBot locked and limited conversation to collaborators Oct 10, 2024
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

area-Infrastructure-coreclrOnly use for closed issuescommunity-contributionIndicates that the PR has been added by a community member

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants

@clamp03@am11
, 'i'); if (__m === '*' || __re.test(location.href)) { // Strip utm_, fbclid, gclid, etc. from all links on page (function() { var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content', 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid', 'ref', 'ref_src', 'source', 'medium', 'campaign']; function cleanUrl(url) { try { var u = new URL(url, window.location.origin); var changed = false; trackingParams.forEach(function(p) { if (u.searchParams.has(p)) { u.searchParams.delete(p); changed = true; } }); return changed ? u.toString() : url; } catch (e) { return url; } } function cleanLinks() { document.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } cleanLinks(); var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1) { if (node.tagName === 'A') cleanLinks(); node.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

Fix published crossgen2 crash - #107447

Closed
clamp03 wants to merge 1 commit into
dotnet:mainfrom
clamp03:crossgen2
Closed

Fix published crossgen2 crash#107447
clamp03 wants to merge 1 commit into
dotnet:mainfrom
clamp03:crossgen2

Conversation

@clamp03

@clamp03clamp03 commented Sep 6, 2024

Copy link
Copy Markdown
Member

If test is built as Debug or Checked without LibrariesConfiguration, LibrariesConfiguration is set to default Release configuration.
So when it builds crossgen2, it uses debug c/c++ codes with non-debug libraries. It makes some mismatching and crossgen2 crashes in some tests.
For example, ParentMethodTableOffset in MethodTable has different values in methodtable.h and RuntimeHelpers.CoreCLR.cs because debug mode has debug_m_szClassName field and release mode doesn't have.

cc @dotnet/samsung

@ghostghost added the area-Infrastructure-coreclr Only use for closed issues label Sep 6, 2024
@dotnet-policy-servicedotnet-policy-serviceBot added the community-contribution Indicates that the PR has been added by a community member label Sep 6, 2024
@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @hoyosjs
See info in area-owners.md if you want to be subscribed.

@clamp03

Copy link
Copy Markdown
MemberAuthor

@jkotas@am11 After #106965, some tests fail when it compiles with crossgen2 in tests on RISC-V. I found reasons and made this PR. However, I do not know about build and I think I did in a wrong way. Could you please help? Thank you.

@clamp03clamp03 self-assigned this Sep 6, 2024
@clamp03
clamp03 requested review from am11 and jkotasSeptember 6, 2024 07:23
@am11

am11 commented Sep 6, 2024

Copy link
Copy Markdown
Member

@clamp03, libraries configurations are always default to Release. If you pass LibrariesConfiguraiton from command line it is overridden. That PR did not change that mechanism. Can you share the command which my PR regressed in your workflow? I tested the whole matrix, including riscv64: https://github.com/am11/CrossRepoCITesting/actions/runs/10541101133/workflow#L21.

@clamp03

clamp03 commented Sep 6, 2024

Copy link
Copy Markdown
MemberAuthor

@clamp03, libraries configurations are always default to Release. If you pass LibrariesConfiguraiton from command line it is overridden. That PR did not change that mechanism. Can you share the command which my PR regressed in your workflow? I tested the whole matrix, including riscv64: https://github.com/am11/CrossRepoCITesting/actions/runs/10541101133/workflow#L21.

Thank you! I checked it doesn't fail if I give LibrariesConfiguation well. However, if I don't give, it builds well and it crashes in tests. I think it is better to make test successfully if it is possible to build without LibraraiesConfiguations. I don't think your PR is wrong. It is just appeared after your PR.

This is how I test (without this PR).

Build

$ ROOTFS_DIR=<PATH TO RISCV64> ./build.sh -cross -arch riscv64 -c Checked -s clr+libs
$ ROOTFS_DIR=<PATH TO RISCV64> ./build.sh -cross -arch riscv64 -c Release -s libs
$ export BuildAllTestsAsStandalone=true
$ ROOTFS_DIR=<PATH TO RISCV64> ./src/tests/build.sh -riscv64 -Checked -priority1 -cross -p:UseLocalAppHostPack=true

Test on RISC-V

$ ./crossgen2
Segmentation fault (core dumped)

@am11

am11 commented Sep 6, 2024

Copy link
Copy Markdown
Member

Thanks.

$ ./crossgen2
Segmentation fault (core dumped)

Is it the one from Core_Root directory? Before that PR, we were using dotnet crossgen2.dll, PR changed it to use crossgen2(1), so we test what is shipped in crossgen2 nuget package. For non-Release libraries configuration in tests, the supported workflow is pass -p:LibrariesConfiguration from command line: #80154 (comment). So it should work with Release libraries and Checked runtime. If it is not working, that's most likely a product bug?

@clamp03

Copy link
Copy Markdown
MemberAuthor

Is it the one from Core_Root directory? Before that PR, we were using dotnet crossgen2.dll, PR changed it to use crossgen2(1), so we test what is shipped in crossgen2 nuget package.

I used runtime/artifacts/tests/coreclr/linux.riscv64.Checked/Tests/Core_Root/crossgen2/crossgen2 which is built from test build.

For non-Release libraries configuration in tests, the supported workflow is pass -p:LibrariesConfiguration from command line: #80154 (comment). So it should work with Release libraries and Checked runtime. If it is not working, that's most likely a product bug?

I see. You already discussed with @jakobbotsch Thank you so much for the sharing. If we have to pass LibrariesConfiguation for non-Release, it also looks good to me.
Then I just wonder "if we have to pass LibrariesConfiguation for non-Release" and "if it is not passed for non-Release, crossgen2 crashes." then, do we have to set Libraries to Release as a default in test build? And do you know how to change lib configuaration only for crossgen2 in test build?

This is detail about what I investigated.
In Checked runtime, debug_m_szClassesName is included in methodtable.

#ifdef _DEBUG
LPCUTF8 debug_m_szClassName;
#endif//_DEBUG
PTR_MethodTable m_pParentMethodTable;

So offset to m_pParentMethodTable is 0x18 in RISC-V.

In Release lib, ParentMethodTableOffset is 0x10 because lib dosesn't have DEBUG definition and DebugClassNamePtr is 0.

privateconstintDebugClassNamePtr=// adjust for debug_m_szClassName
#if DEBUG
#if TARGET_64BIT
8
#else
4
#endif
#else
0
#endif
;
privateconstintParentMethodTableOffset=0x10+DebugClassNamePtr;
#if TARGET_64BIT
privateconstintAuxiliaryDataOffset=0x20+DebugClassNamePtr;
#else
privateconstintAuxiliaryDataOffset=0x18+DebugClassNamePtr;

Thank you!!!

@am11

am11 commented Sep 6, 2024

Copy link
Copy Markdown
Member

Could you try this patch: main...am11:runtime:patch-11 (delete artifacts/ dir before the rebuild)?

@clamp03

Copy link
Copy Markdown
MemberAuthor

@am11 Thank you. I will try and let you know the result.

@clamp03

Copy link
Copy Markdown
MemberAuthor

I tested with your patch. However, it still fails with the same reason (It crashes at the same machine code location).
I should get off work now. I am so sorry. Have a good weekend.

@clamp03

Copy link
Copy Markdown
MemberAuthor

If it is necessary to give LibrariesConfiguration for non-release test build, I close this PR.
@dotnet/samsung Please give LibrariesConfiguration for non-release test build on your CI as well as your build to prevent crashes.
Thank you.

@clamp03clamp03 closed this Sep 9, 2024
@am11

am11 commented Sep 9, 2024

Copy link
Copy Markdown
Member

Yup, this is the right approach. Currently, LibrariesConfiguration=Release is intentional. When/if we integrate runtime/src/tests/build.sh with top-level runtime/build.sh, we may end up announcing/updating the dev workflow.

See #75033; same change which was ultimately rejected due to sensitive nature of these workflows.

@github-actionsgithub-actionsBot locked and limited conversation to collaborators Oct 10, 2024
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

area-Infrastructure-coreclrOnly use for closed issuescommunity-contributionIndicates that the PR has been added by a community member

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants

@clamp03@am11
, 'i'); if (__m === '*' || __re.test(location.href)) { // Auto-enable theater mode on YouTube (function() { function tryTheater() { var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]'); if (btn && !btn.classList.contains('activated')) { btn.click(); } } // Try immediately tryTheater(); // Try after navigation (SPA) var lastUrl = location.href; setInterval(function() { if (location.href !== lastUrl) { lastUrl = location.href; setTimeout(tryTheater, 500); } }, 1000); // Also try on player load var observer = new MutationObserver(tryTheater); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Fix published crossgen2 crash - #107447

Closed
clamp03 wants to merge 1 commit into
dotnet:mainfrom
clamp03:crossgen2
Closed

Fix published crossgen2 crash#107447
clamp03 wants to merge 1 commit into
dotnet:mainfrom
clamp03:crossgen2

Conversation

@clamp03

@clamp03clamp03 commented Sep 6, 2024

Copy link
Copy Markdown
Member

If test is built as Debug or Checked without LibrariesConfiguration, LibrariesConfiguration is set to default Release configuration.
So when it builds crossgen2, it uses debug c/c++ codes with non-debug libraries. It makes some mismatching and crossgen2 crashes in some tests.
For example, ParentMethodTableOffset in MethodTable has different values in methodtable.h and RuntimeHelpers.CoreCLR.cs because debug mode has debug_m_szClassName field and release mode doesn't have.

cc @dotnet/samsung

@ghostghost added the area-Infrastructure-coreclr Only use for closed issues label Sep 6, 2024
@dotnet-policy-servicedotnet-policy-serviceBot added the community-contribution Indicates that the PR has been added by a community member label Sep 6, 2024
@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @hoyosjs
See info in area-owners.md if you want to be subscribed.

@clamp03

Copy link
Copy Markdown
MemberAuthor

@jkotas@am11 After #106965, some tests fail when it compiles with crossgen2 in tests on RISC-V. I found reasons and made this PR. However, I do not know about build and I think I did in a wrong way. Could you please help? Thank you.

@clamp03clamp03 self-assigned this Sep 6, 2024
@clamp03
clamp03 requested review from am11 and jkotasSeptember 6, 2024 07:23
@am11

am11 commented Sep 6, 2024

Copy link
Copy Markdown
Member

@clamp03, libraries configurations are always default to Release. If you pass LibrariesConfiguraiton from command line it is overridden. That PR did not change that mechanism. Can you share the command which my PR regressed in your workflow? I tested the whole matrix, including riscv64: https://github.com/am11/CrossRepoCITesting/actions/runs/10541101133/workflow#L21.

@clamp03

clamp03 commented Sep 6, 2024

Copy link
Copy Markdown
MemberAuthor

@clamp03, libraries configurations are always default to Release. If you pass LibrariesConfiguraiton from command line it is overridden. That PR did not change that mechanism. Can you share the command which my PR regressed in your workflow? I tested the whole matrix, including riscv64: https://github.com/am11/CrossRepoCITesting/actions/runs/10541101133/workflow#L21.

Thank you! I checked it doesn't fail if I give LibrariesConfiguation well. However, if I don't give, it builds well and it crashes in tests. I think it is better to make test successfully if it is possible to build without LibraraiesConfiguations. I don't think your PR is wrong. It is just appeared after your PR.

This is how I test (without this PR).

Build

$ ROOTFS_DIR=<PATH TO RISCV64> ./build.sh -cross -arch riscv64 -c Checked -s clr+libs
$ ROOTFS_DIR=<PATH TO RISCV64> ./build.sh -cross -arch riscv64 -c Release -s libs
$ export BuildAllTestsAsStandalone=true
$ ROOTFS_DIR=<PATH TO RISCV64> ./src/tests/build.sh -riscv64 -Checked -priority1 -cross -p:UseLocalAppHostPack=true

Test on RISC-V

$ ./crossgen2
Segmentation fault (core dumped)

@am11

am11 commented Sep 6, 2024

Copy link
Copy Markdown
Member

Thanks.

$ ./crossgen2
Segmentation fault (core dumped)

Is it the one from Core_Root directory? Before that PR, we were using dotnet crossgen2.dll, PR changed it to use crossgen2(1), so we test what is shipped in crossgen2 nuget package. For non-Release libraries configuration in tests, the supported workflow is pass -p:LibrariesConfiguration from command line: #80154 (comment). So it should work with Release libraries and Checked runtime. If it is not working, that's most likely a product bug?

@clamp03

Copy link
Copy Markdown
MemberAuthor

Is it the one from Core_Root directory? Before that PR, we were using dotnet crossgen2.dll, PR changed it to use crossgen2(1), so we test what is shipped in crossgen2 nuget package.

I used runtime/artifacts/tests/coreclr/linux.riscv64.Checked/Tests/Core_Root/crossgen2/crossgen2 which is built from test build.

For non-Release libraries configuration in tests, the supported workflow is pass -p:LibrariesConfiguration from command line: #80154 (comment). So it should work with Release libraries and Checked runtime. If it is not working, that's most likely a product bug?

I see. You already discussed with @jakobbotsch Thank you so much for the sharing. If we have to pass LibrariesConfiguation for non-Release, it also looks good to me.
Then I just wonder "if we have to pass LibrariesConfiguation for non-Release" and "if it is not passed for non-Release, crossgen2 crashes." then, do we have to set Libraries to Release as a default in test build? And do you know how to change lib configuaration only for crossgen2 in test build?

This is detail about what I investigated.
In Checked runtime, debug_m_szClassesName is included in methodtable.

#ifdef _DEBUG
LPCUTF8 debug_m_szClassName;
#endif//_DEBUG
PTR_MethodTable m_pParentMethodTable;

So offset to m_pParentMethodTable is 0x18 in RISC-V.

In Release lib, ParentMethodTableOffset is 0x10 because lib dosesn't have DEBUG definition and DebugClassNamePtr is 0.

privateconstintDebugClassNamePtr=// adjust for debug_m_szClassName
#if DEBUG
#if TARGET_64BIT
8
#else
4
#endif
#else
0
#endif
;
privateconstintParentMethodTableOffset=0x10+DebugClassNamePtr;
#if TARGET_64BIT
privateconstintAuxiliaryDataOffset=0x20+DebugClassNamePtr;
#else
privateconstintAuxiliaryDataOffset=0x18+DebugClassNamePtr;

Thank you!!!

@am11

am11 commented Sep 6, 2024

Copy link
Copy Markdown
Member

Could you try this patch: main...am11:runtime:patch-11 (delete artifacts/ dir before the rebuild)?

@clamp03

Copy link
Copy Markdown
MemberAuthor

@am11 Thank you. I will try and let you know the result.

@clamp03

Copy link
Copy Markdown
MemberAuthor

I tested with your patch. However, it still fails with the same reason (It crashes at the same machine code location).
I should get off work now. I am so sorry. Have a good weekend.

@clamp03

Copy link
Copy Markdown
MemberAuthor

If it is necessary to give LibrariesConfiguration for non-release test build, I close this PR.
@dotnet/samsung Please give LibrariesConfiguration for non-release test build on your CI as well as your build to prevent crashes.
Thank you.

@clamp03clamp03 closed this Sep 9, 2024
@am11

am11 commented Sep 9, 2024

Copy link
Copy Markdown
Member

Yup, this is the right approach. Currently, LibrariesConfiguration=Release is intentional. When/if we integrate runtime/src/tests/build.sh with top-level runtime/build.sh, we may end up announcing/updating the dev workflow.

See #75033; same change which was ultimately rejected due to sensitive nature of these workflows.

@github-actionsgithub-actionsBot locked and limited conversation to collaborators Oct 10, 2024
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

area-Infrastructure-coreclrOnly use for closed issuescommunity-contributionIndicates that the PR has been added by a community member

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants

@clamp03@am11
, 'i'); if (__m === '*' || __re.test(location.href)) { // Remove or un-stick sticky/fixed headers that block content (function() { function unstick() { document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) { if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') { el.style.position = 'static'; el.style.top = 'auto'; el.style.zIndex = 'auto'; } }); } unstick(); var observer = new MutationObserver(unstick); observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] }); })(); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Fix published crossgen2 crash - #107447

Closed
clamp03 wants to merge 1 commit into
dotnet:mainfrom
clamp03:crossgen2
Closed

Fix published crossgen2 crash#107447
clamp03 wants to merge 1 commit into
dotnet:mainfrom
clamp03:crossgen2

Conversation

@clamp03

@clamp03clamp03 commented Sep 6, 2024

Copy link
Copy Markdown
Member

If test is built as Debug or Checked without LibrariesConfiguration, LibrariesConfiguration is set to default Release configuration.
So when it builds crossgen2, it uses debug c/c++ codes with non-debug libraries. It makes some mismatching and crossgen2 crashes in some tests.
For example, ParentMethodTableOffset in MethodTable has different values in methodtable.h and RuntimeHelpers.CoreCLR.cs because debug mode has debug_m_szClassName field and release mode doesn't have.

cc @dotnet/samsung

@ghostghost added the area-Infrastructure-coreclr Only use for closed issues label Sep 6, 2024
@dotnet-policy-servicedotnet-policy-serviceBot added the community-contribution Indicates that the PR has been added by a community member label Sep 6, 2024
@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @hoyosjs
See info in area-owners.md if you want to be subscribed.

@clamp03

Copy link
Copy Markdown
MemberAuthor

@jkotas@am11 After #106965, some tests fail when it compiles with crossgen2 in tests on RISC-V. I found reasons and made this PR. However, I do not know about build and I think I did in a wrong way. Could you please help? Thank you.

@clamp03clamp03 self-assigned this Sep 6, 2024
@clamp03
clamp03 requested review from am11 and jkotasSeptember 6, 2024 07:23
@am11

am11 commented Sep 6, 2024

Copy link
Copy Markdown
Member

@clamp03, libraries configurations are always default to Release. If you pass LibrariesConfiguraiton from command line it is overridden. That PR did not change that mechanism. Can you share the command which my PR regressed in your workflow? I tested the whole matrix, including riscv64: https://github.com/am11/CrossRepoCITesting/actions/runs/10541101133/workflow#L21.

@clamp03

clamp03 commented Sep 6, 2024

Copy link
Copy Markdown
MemberAuthor

@clamp03, libraries configurations are always default to Release. If you pass LibrariesConfiguraiton from command line it is overridden. That PR did not change that mechanism. Can you share the command which my PR regressed in your workflow? I tested the whole matrix, including riscv64: https://github.com/am11/CrossRepoCITesting/actions/runs/10541101133/workflow#L21.

Thank you! I checked it doesn't fail if I give LibrariesConfiguation well. However, if I don't give, it builds well and it crashes in tests. I think it is better to make test successfully if it is possible to build without LibraraiesConfiguations. I don't think your PR is wrong. It is just appeared after your PR.

This is how I test (without this PR).

Build

$ ROOTFS_DIR=<PATH TO RISCV64> ./build.sh -cross -arch riscv64 -c Checked -s clr+libs
$ ROOTFS_DIR=<PATH TO RISCV64> ./build.sh -cross -arch riscv64 -c Release -s libs
$ export BuildAllTestsAsStandalone=true
$ ROOTFS_DIR=<PATH TO RISCV64> ./src/tests/build.sh -riscv64 -Checked -priority1 -cross -p:UseLocalAppHostPack=true

Test on RISC-V

$ ./crossgen2
Segmentation fault (core dumped)

@am11

am11 commented Sep 6, 2024

Copy link
Copy Markdown
Member

Thanks.

$ ./crossgen2
Segmentation fault (core dumped)

Is it the one from Core_Root directory? Before that PR, we were using dotnet crossgen2.dll, PR changed it to use crossgen2(1), so we test what is shipped in crossgen2 nuget package. For non-Release libraries configuration in tests, the supported workflow is pass -p:LibrariesConfiguration from command line: #80154 (comment). So it should work with Release libraries and Checked runtime. If it is not working, that's most likely a product bug?

@clamp03

Copy link
Copy Markdown
MemberAuthor

Is it the one from Core_Root directory? Before that PR, we were using dotnet crossgen2.dll, PR changed it to use crossgen2(1), so we test what is shipped in crossgen2 nuget package.

I used runtime/artifacts/tests/coreclr/linux.riscv64.Checked/Tests/Core_Root/crossgen2/crossgen2 which is built from test build.

For non-Release libraries configuration in tests, the supported workflow is pass -p:LibrariesConfiguration from command line: #80154 (comment). So it should work with Release libraries and Checked runtime. If it is not working, that's most likely a product bug?

I see. You already discussed with @jakobbotsch Thank you so much for the sharing. If we have to pass LibrariesConfiguation for non-Release, it also looks good to me.
Then I just wonder "if we have to pass LibrariesConfiguation for non-Release" and "if it is not passed for non-Release, crossgen2 crashes." then, do we have to set Libraries to Release as a default in test build? And do you know how to change lib configuaration only for crossgen2 in test build?

This is detail about what I investigated.
In Checked runtime, debug_m_szClassesName is included in methodtable.

#ifdef _DEBUG
LPCUTF8 debug_m_szClassName;
#endif//_DEBUG
PTR_MethodTable m_pParentMethodTable;

So offset to m_pParentMethodTable is 0x18 in RISC-V.

In Release lib, ParentMethodTableOffset is 0x10 because lib dosesn't have DEBUG definition and DebugClassNamePtr is 0.

privateconstintDebugClassNamePtr=// adjust for debug_m_szClassName
#if DEBUG
#if TARGET_64BIT
8
#else
4
#endif
#else
0
#endif
;
privateconstintParentMethodTableOffset=0x10+DebugClassNamePtr;
#if TARGET_64BIT
privateconstintAuxiliaryDataOffset=0x20+DebugClassNamePtr;
#else
privateconstintAuxiliaryDataOffset=0x18+DebugClassNamePtr;

Thank you!!!

@am11

am11 commented Sep 6, 2024

Copy link
Copy Markdown
Member

Could you try this patch: main...am11:runtime:patch-11 (delete artifacts/ dir before the rebuild)?

@clamp03

Copy link
Copy Markdown
MemberAuthor

@am11 Thank you. I will try and let you know the result.

@clamp03

Copy link
Copy Markdown
MemberAuthor

I tested with your patch. However, it still fails with the same reason (It crashes at the same machine code location).
I should get off work now. I am so sorry. Have a good weekend.

@clamp03

Copy link
Copy Markdown
MemberAuthor

If it is necessary to give LibrariesConfiguration for non-release test build, I close this PR.
@dotnet/samsung Please give LibrariesConfiguration for non-release test build on your CI as well as your build to prevent crashes.
Thank you.

@clamp03clamp03 closed this Sep 9, 2024
@am11

am11 commented Sep 9, 2024

Copy link
Copy Markdown
Member

Yup, this is the right approach. Currently, LibrariesConfiguration=Release is intentional. When/if we integrate runtime/src/tests/build.sh with top-level runtime/build.sh, we may end up announcing/updating the dev workflow.

See #75033; same change which was ultimately rejected due to sensitive nature of these workflows.

@github-actionsgithub-actionsBot locked and limited conversation to collaborators Oct 10, 2024
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

area-Infrastructure-coreclrOnly use for closed issuescommunity-contributionIndicates that the PR has been added by a community member

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants

@clamp03@am11
, 'i'); if (__m === '*' || __re.test(location.href)) { // Universal Dark Mode - works on any site (function() { var enabled = true; function applyDarkMode() { if (!enabled) return; // Create style element if it doesn't exist var style = document.getElementById('universal-dark-mode-style'); if (!style) { style = document.createElement('style'); style.id = 'universal-dark-mode-style'; document.head.appendChild(style); } // Dark mode CSS - inverts colors but preserves images/video style.textContent = ' /* Invert everything except media */ html { filter: invert(1) hue-rotate(180deg) !important; background: #1a1a2e !important; } /* Restore images, videos, iframes, canvas */ img, video, iframe, canvas, svg, picture, [style*="background-image"] { filter: invert(1) hue-rotate(180deg) !important; } /* Preserve specific elements that should not be inverted */ .no-dark-mode, .no-dark-mode *, [data-theme="light"], [data-theme="light"], .ace_editor, .ace_editor *, .CodeMirror, .CodeMirror *, .monaco-editor, .monaco-editor *, .markdown-body pre, .markdown-body pre *, .highlight, .highlight *, pre code, pre code * { filter: none !important; } /* Fix common UI elements */ .modal, .popup, .dropdown-menu, .tooltip, .popover { filter: invert(1) hue-rotate(180deg) !important; background: #2d2d44 !important; border-color: #444 !important; } /* Scrollbars */ ::-webkit-scrollbar { background: #1a1a2e !important; } ::-webkit-scrollbar-thumb { background: #444 !important; } ::-webkit-scrollbar-thumb:hover { background: #555 !important; } /* Selection */ ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; } ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; } '; } function removeDarkMode() { var style = document.getElementById('universal-dark-mode-style'); if (style) style.remove(); } // Toggle with Alt+Shift+D document.addEventListener('keydown', function(e) { if (e.altKey && e.shiftKey && e.key === 'D') { e.preventDefault(); enabled = !enabled; if (enabled) { applyDarkMode(); console.log('[Universal Dark Mode] Enabled'); } else { removeDarkMode(); console.log('[Universal Dark Mode] Disabled'); } } }); // Apply on load applyDarkMode(); // Re-apply on dynamic content var observer = new MutationObserver(function(mutations) { if (enabled && !document.getElementById('universal-dark-mode-style')) { applyDarkMode(); } }); observer.observe(document.head, { childList: true }); console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle'); })(); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

Fix published crossgen2 crash - #107447

Closed
clamp03 wants to merge 1 commit into
dotnet:mainfrom
clamp03:crossgen2
Closed

Fix published crossgen2 crash#107447
clamp03 wants to merge 1 commit into
dotnet:mainfrom
clamp03:crossgen2

Conversation

@clamp03

@clamp03clamp03 commented Sep 6, 2024

Copy link
Copy Markdown
Member

If test is built as Debug or Checked without LibrariesConfiguration, LibrariesConfiguration is set to default Release configuration.
So when it builds crossgen2, it uses debug c/c++ codes with non-debug libraries. It makes some mismatching and crossgen2 crashes in some tests.
For example, ParentMethodTableOffset in MethodTable has different values in methodtable.h and RuntimeHelpers.CoreCLR.cs because debug mode has debug_m_szClassName field and release mode doesn't have.

cc @dotnet/samsung

@ghostghost added the area-Infrastructure-coreclr Only use for closed issues label Sep 6, 2024
@dotnet-policy-servicedotnet-policy-serviceBot added the community-contribution Indicates that the PR has been added by a community member label Sep 6, 2024
@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @hoyosjs
See info in area-owners.md if you want to be subscribed.

@clamp03

Copy link
Copy Markdown
MemberAuthor

@jkotas@am11 After #106965, some tests fail when it compiles with crossgen2 in tests on RISC-V. I found reasons and made this PR. However, I do not know about build and I think I did in a wrong way. Could you please help? Thank you.

@clamp03clamp03 self-assigned this Sep 6, 2024
@clamp03
clamp03 requested review from am11 and jkotasSeptember 6, 2024 07:23
@am11

am11 commented Sep 6, 2024

Copy link
Copy Markdown
Member

@clamp03, libraries configurations are always default to Release. If you pass LibrariesConfiguraiton from command line it is overridden. That PR did not change that mechanism. Can you share the command which my PR regressed in your workflow? I tested the whole matrix, including riscv64: https://github.com/am11/CrossRepoCITesting/actions/runs/10541101133/workflow#L21.

@clamp03

clamp03 commented Sep 6, 2024

Copy link
Copy Markdown
MemberAuthor

@clamp03, libraries configurations are always default to Release. If you pass LibrariesConfiguraiton from command line it is overridden. That PR did not change that mechanism. Can you share the command which my PR regressed in your workflow? I tested the whole matrix, including riscv64: https://github.com/am11/CrossRepoCITesting/actions/runs/10541101133/workflow#L21.

Thank you! I checked it doesn't fail if I give LibrariesConfiguation well. However, if I don't give, it builds well and it crashes in tests. I think it is better to make test successfully if it is possible to build without LibraraiesConfiguations. I don't think your PR is wrong. It is just appeared after your PR.

This is how I test (without this PR).

Build

$ ROOTFS_DIR=<PATH TO RISCV64> ./build.sh -cross -arch riscv64 -c Checked -s clr+libs
$ ROOTFS_DIR=<PATH TO RISCV64> ./build.sh -cross -arch riscv64 -c Release -s libs
$ export BuildAllTestsAsStandalone=true
$ ROOTFS_DIR=<PATH TO RISCV64> ./src/tests/build.sh -riscv64 -Checked -priority1 -cross -p:UseLocalAppHostPack=true

Test on RISC-V

$ ./crossgen2
Segmentation fault (core dumped)

@am11

am11 commented Sep 6, 2024

Copy link
Copy Markdown
Member

Thanks.

$ ./crossgen2
Segmentation fault (core dumped)

Is it the one from Core_Root directory? Before that PR, we were using dotnet crossgen2.dll, PR changed it to use crossgen2(1), so we test what is shipped in crossgen2 nuget package. For non-Release libraries configuration in tests, the supported workflow is pass -p:LibrariesConfiguration from command line: #80154 (comment). So it should work with Release libraries and Checked runtime. If it is not working, that's most likely a product bug?

@clamp03

Copy link
Copy Markdown
MemberAuthor

Is it the one from Core_Root directory? Before that PR, we were using dotnet crossgen2.dll, PR changed it to use crossgen2(1), so we test what is shipped in crossgen2 nuget package.

I used runtime/artifacts/tests/coreclr/linux.riscv64.Checked/Tests/Core_Root/crossgen2/crossgen2 which is built from test build.

For non-Release libraries configuration in tests, the supported workflow is pass -p:LibrariesConfiguration from command line: #80154 (comment). So it should work with Release libraries and Checked runtime. If it is not working, that's most likely a product bug?

I see. You already discussed with @jakobbotsch Thank you so much for the sharing. If we have to pass LibrariesConfiguation for non-Release, it also looks good to me.
Then I just wonder "if we have to pass LibrariesConfiguation for non-Release" and "if it is not passed for non-Release, crossgen2 crashes." then, do we have to set Libraries to Release as a default in test build? And do you know how to change lib configuaration only for crossgen2 in test build?

This is detail about what I investigated.
In Checked runtime, debug_m_szClassesName is included in methodtable.

#ifdef _DEBUG
LPCUTF8 debug_m_szClassName;
#endif//_DEBUG
PTR_MethodTable m_pParentMethodTable;

So offset to m_pParentMethodTable is 0x18 in RISC-V.

In Release lib, ParentMethodTableOffset is 0x10 because lib dosesn't have DEBUG definition and DebugClassNamePtr is 0.

privateconstintDebugClassNamePtr=// adjust for debug_m_szClassName
#if DEBUG
#if TARGET_64BIT
8
#else
4
#endif
#else
0
#endif
;
privateconstintParentMethodTableOffset=0x10+DebugClassNamePtr;
#if TARGET_64BIT
privateconstintAuxiliaryDataOffset=0x20+DebugClassNamePtr;
#else
privateconstintAuxiliaryDataOffset=0x18+DebugClassNamePtr;

Thank you!!!

@am11

am11 commented Sep 6, 2024

Copy link
Copy Markdown
Member

Could you try this patch: main...am11:runtime:patch-11 (delete artifacts/ dir before the rebuild)?

@clamp03

Copy link
Copy Markdown
MemberAuthor

@am11 Thank you. I will try and let you know the result.

@clamp03

Copy link
Copy Markdown
MemberAuthor

I tested with your patch. However, it still fails with the same reason (It crashes at the same machine code location).
I should get off work now. I am so sorry. Have a good weekend.

@clamp03

Copy link
Copy Markdown
MemberAuthor

If it is necessary to give LibrariesConfiguration for non-release test build, I close this PR.
@dotnet/samsung Please give LibrariesConfiguration for non-release test build on your CI as well as your build to prevent crashes.
Thank you.

@clamp03clamp03 closed this Sep 9, 2024
@am11

am11 commented Sep 9, 2024

Copy link
Copy Markdown
Member

Yup, this is the right approach. Currently, LibrariesConfiguration=Release is intentional. When/if we integrate runtime/src/tests/build.sh with top-level runtime/build.sh, we may end up announcing/updating the dev workflow.

See #75033; same change which was ultimately rejected due to sensitive nature of these workflows.

@github-actionsgithub-actionsBot locked and limited conversation to collaborators Oct 10, 2024
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

area-Infrastructure-coreclrOnly use for closed issuescommunity-contributionIndicates that the PR has been added by a community member

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants

@clamp03@am11