Skip to content

[mono][infra] Enable runtime tests in fullAOT LLVM mode on linux-x64 - #92057

Merged
kotlarmilos merged 110 commits into
dotnet:mainfrom
kotlarmilos:bugfix/linux-x64-fullaot-job
Apr 5, 2024
Merged

[mono][infra] Enable runtime tests in fullAOT LLVM mode on linux-x64#92057
kotlarmilos merged 110 commits into
dotnet:mainfrom
kotlarmilos:bugfix/linux-x64-fullaot-job

Conversation

@kotlarmilos

@kotlarmiloskotlarmilos commented Sep 14, 2023•
edited
Loading

Copy link
Copy Markdown
Member

Description

This PR enables jobs to run runtime tests in fullAOT LLVM mode on linux-x64. The jobs utilize the CBL-Mariner docker image, which contains the clang toolchain instead of binutils. Due to OOM and timeout issues, the tests are split into two jobs: intrinsics and other runtime tests.

Failing tests have been disabled and will be triaged in subsequent PRs. The arm64 support will be introduced in subsequent PRs. A list of disabled tests is included in #90427.

@kotlarmiloskotlarmilos added NO-REVIEW Experimental/testing PR, do NOT review it area-Infrastructure-mono labels Sep 14, 2023
@kotlarmiloskotlarmilos added this to the 9.0.0 milestone Sep 14, 2023
@ghost

ghost commented Sep 14, 2023

Copy link
Copy Markdown

Tagging subscribers to this area: @directhex
See info in area-owners.md if you want to be subscribed.

Issue Details

Work in progress.

This PR tests the linux-x64 fullAOT CI job.

Author:kotlarmilos
Assignees:-
Labels:

NO-REVIEW, area-Infrastructure-mono

Milestone:-

@kotlarmiloskotlarmilos added the NO-MERGE The PR is not ready for merge yet (see discussion for detailed reasons) label Sep 14, 2023
@kotlarmilos

kotlarmilos commented Sep 14, 2023

Copy link
Copy Markdown
MemberAuthor

/azp run runtime-extra-platforms

@azure-pipelines

azure-pipelinesBot commented Sep 14, 2023

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

This was referenced Sep 14, 2023
Open
Closed
Closed
Closed
Closed
Closed
Closed
Open
Open
@kotlarmilos

kotlarmilos commented Sep 15, 2023

Copy link
Copy Markdown
MemberAuthor

/azp run runtime-extra-platforms

@azure-pipelines

azure-pipelinesBot commented Sep 15, 2023

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

@build-analysisbuild-analysisBot mentioned this pull request Sep 15, 2023
Closed
@kotlarmilos

kotlarmilos commented Sep 15, 2023

Copy link
Copy Markdown
MemberAuthor

/azp run runtime-extra-platforms

@azure-pipelines

azure-pipelinesBot commented Sep 15, 2023

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

@kotlarmilos

kotlarmilos commented Sep 15, 2023

Copy link
Copy Markdown
MemberAuthor

/azp run runtime-extra-platforms

@azure-pipelines

azure-pipelinesBot commented Sep 15, 2023

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

@kotlarmilos

kotlarmilos commented Sep 16, 2023

Copy link
Copy Markdown
MemberAuthor

/azp run runtime-extra-platforms

@kotlarmilos

kotlarmilos commented Apr 2, 2024

Copy link
Copy Markdown
MemberAuthor

/azp run runtime-extra-platforms

@kotlarmilos

kotlarmilos commented Apr 3, 2024

Copy link
Copy Markdown
MemberAuthor

/azp run runtime-extra-platforms

@kotlarmilos

kotlarmilos commented Apr 3, 2024

Copy link
Copy Markdown
MemberAuthor

/azp run runtime-extra-platforms

@kotlarmilos

kotlarmilos commented Apr 3, 2024

Copy link
Copy Markdown
MemberAuthor

/azp run runtime-extra-platforms

@kotlarmilos

kotlarmilos commented Apr 3, 2024

Copy link
Copy Markdown
MemberAuthor

/azp run runtime-extra-platforms

@kotlarmiloskotlarmilos changed the title [mono][infra] Enable linux-x64 Mono fullAOT llvm job on extra platforms[mono][infra] Enable runtime tests in fullAOT LLVM mode on linux-x64Apr 3, 2024
@kotlarmilos
kotlarmilos marked this pull request as ready for review April 3, 2024 18:17
@kotlarmiloskotlarmilos removed the blocked Issue/PR is blocked on something - see comments label Apr 3, 2024
@kotlarmilos

kotlarmilos commented Apr 4, 2024

Copy link
Copy Markdown
MemberAuthor

This PR is ready for review. Failures shouldn't be related.

Runtime tests: https://dev.azure.com/dnceng-public/public/_build/results?buildId=629337&view=logs&jobId=d29f5ac9-2776-59d5-99ad-be6d4bcd416c&j=d29f5ac9-2776-59d5-99ad-be6d4bcd416c
Runtime intrinsics tests: https://dev.azure.com/dnceng-public/public/_build/results?buildId=629337&view=logs&jobId=d29f5ac9-2776-59d5-99ad-be6d4bcd416c&j=8c8e10f8-cac0-5393-e608-4ea6cc126022

matouskozak
matouskozak approved these changes Apr 4, 2024
View reviewed changes

@matouskozakmatouskozak left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! good job

Comment threadeng/pipelines/extra-platforms/runtime-extra-platforms-other.yml Outdated
testGroup: innerloop
nameSuffix: AllSubsets_Mono_LLVMFullAot_RuntimeTests_Tests
runtimeVariant: llvmfullaot
buildArgs: -s mono+libs+clr.hosts+clr.iltools -c Release /p:MonoEnableLLVM=true /p:MonoBundleLLVMOptimizer=true

@matouskozakmatouskozakApr 4, 2024

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if it is needed in this PR but we should consolidate which flags are used for fullAOT/AOT jobs across our builds. For example, for perf jobs we use /p:MonoAOTEnableLLVM=true /p:MonoEnableLLVM=true /p:BuildMonoAOTCrossCompiler=truehttps://github.com/matouskozak/runtime/blob/12f546416d4aeb23b980a92e407fbf693823236d/eng/pipelines/coreclr/perf-non-wasm-jobs.yml#L39

@kotlarmiloskotlarmilosApr 4, 2024

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. This command only builds the runtime.

The AOT compiler is built in an optional step which contains required parameters:

runtime/eng/pipelines/common/templates/runtimes/build-runtime-tests-and-send-to-helix.yml

Lines 42 to 63 in e610649

- ${{ if and(eq(parameters.runtimeFlavor, 'mono'), in(parameters.runtimeVariant, 'llvmaot', 'llvmfullaot', 'minifullaot')) }}:
- ${{ if eq(parameters.archType, 'arm64') }}:
- ${{ if eq(parameters.runtimeVariant, 'minifullaot') }}:
- script: ./build.sh
-subset mono
-c ${{ parameters.buildConfig }}
-arch ${{ parameters.archType }}
/p:BuildMonoAotCrossCompiler=true
/p:BuildMonoAotCrossCompilerOnly=true
/p:CrossBuild=true
displayName: "Build Mono Mini AOT cross compiler"
- ${{ else }}:
- script: ./build.sh
-subset mono
-c ${{ parameters.buildConfig }}
-arch ${{ parameters.archType }}
/p:BuildMonoAotCrossCompiler=true
/p:BuildMonoAotCrossCompilerOnly=true
/p:MonoLibClang="/usr/local/lib/libclang.so.16"
/p:MonoAOTEnableLLVM=true
/p:CrossBuild=true
displayName: "Build Mono LLVM AOT cross compiler"

@matouskozakmatouskozakApr 4, 2024•
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we align on the use of MonoBundleLLVMOptimizer vs MonoAOTEnableLLVM then? Is the MonoBundleLLVMOptimizer=true necessary then? Looks like MonoEnableLLVM=true should be enough

runtime/src/mono/mono.proj

Line 46 in c7ada75

<MonoBundleLLVMOptimizerCondition="'$(MonoEnableLLVM)' == 'true'">true</MonoBundleLLVMOptimizer>

Comment threadeng/pipelines/extra-platforms/runtime-extra-platforms-other.yml Outdated
testGroup: innerloop
nameSuffix: AllSubsets_Mono_LLVMFullAot_RuntimeIntrinsicsTests
runtimeVariant: llvmfullaot
buildArgs: -s mono+libs+clr.hosts+clr.iltools -c Release /p:MonoEnableLLVM=true /p:MonoBundleLLVMOptimizer=true

@matouskozakmatouskozakApr 4, 2024•
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if it is possible with Helix, but could we re-use the build from AllSubsets_Mono_LLVMFullAot_RuntimeTests_Tests job? The runtime build should be the same, the difference is only which tests get build...

@kotlarmiloskotlarmilosApr 4, 2024•
edited
Loading

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct, perhaps we can create a template. @fanyang-mono Should this be moved to runtime-llvm.yml?

@kotlarmiloskotlarmilosApr 5, 2024

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved to the runtime-llvm.yml and fixed llvmaot job.

@kotlarmilos

kotlarmilos commented Apr 4, 2024•
edited
Loading

Copy link
Copy Markdown
MemberAuthor

These jobs have been moved to runtime-llvm.yml. Test run: https://dev.azure.com/dnceng-public/public/_build/results?buildId=630560&view=results

@kotlarmilos

kotlarmilos commented Apr 5, 2024

Copy link
Copy Markdown
MemberAuthor

The failures shouldn't be related.

@kotlarmilos
kotlarmilos merged commit ddae9bd into dotnet:mainApr 5, 2024
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

@matouskozakmatouskozakmatouskozak approved these changes
@lambdageeklambdageekAwaiting requested review from lambdageek
@SamMonoRTSamMonoRTAwaiting requested review from SamMonoRT
@directhexdirecthexAwaiting requested review from directhex
@steveisoksteveisokAwaiting requested review from steveisok
@fanyang-monofanyang-monoAwaiting requested review from fanyang-mono
+1 more reviewer
@vargazvargazvargaz approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

@kotlarmiloskotlarmilos

Labels

area-Infrastructure-mono

Projects

None yet

Milestone

9.0.0

Development

Successfully merging this pull request may close these issues.

5 participants

@kotlarmilos@vargaz@lambdageek@fanyang-mono@matouskozak
, 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all 
 blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
[mono][infra] Enable runtime tests in fullAOT LLVM mode on linux-x64 by kotlarmilos · Pull Request #92057 · dotnet/runtime · GitHub
Skip to content
[mono][infra] Enable runtime tests in fullAOT LLVM mode on linux-x64 - #92057
Merged
kotlarmilos merged 110 commits into
dotnet:mainfrom 
kotlarmilos:bugfix/linux-x64-fullaot-job
Apr 5, 2024
Merged
[mono][infra] Enable runtime tests in fullAOT LLVM mode on linux-x64#92057
kotlarmilos merged 110 commits into
dotnet:mainfrom 
kotlarmilos:bugfix/linux-x64-fullaot-job
Conversation
@kotlarmilos
@kotlarmiloskotlarmilos
commented
Sep 14, 2023•

edited

Loading

Copy link


Copy Markdown

Member
Description
This PR enables jobs to run runtime tests in fullAOT LLVM mode on linux-x64. The jobs utilize the CBL-Mariner docker image, which contains the clang toolchain instead of binutils. Due to OOM and timeout issues, the tests are split into two jobs: intrinsics and other runtime tests.
Failing tests have been disabled and will be triaged in subsequent PRs. The arm64 support will be introduced in subsequent PRs. A list of disabled tests is included in #90427.
@kotlarmiloskotlarmilos
added

NO-REVIEW
Experimental/testing PR, do NOT review it
area-Infrastructure-mono

labels
Sep 14, 2023
@kotlarmiloskotlarmilos
added this to the 9.0.0 milestone
Sep 14, 2023
@ghost
ghost
commented
Sep 14, 2023

Copy link


Copy Markdown

Tagging subscribers to this area: @directhex

See info in area-owners.md if you want to be subscribed.
Issue Details
Work in progress.
This PR tests the linux-x64 fullAOT CI job.
Author:kotlarmilos
Assignees:-
Labels:
NO-REVIEW, area-Infrastructure-mono
Milestone:-
@kotlarmiloskotlarmilos
added
the 
NO-MERGE
The PR is not ready for merge yet (see discussion for detailed reasons)
label
Sep 14, 2023
@kotlarmilos
kotlarmilos
commented
Sep 14, 2023

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-extra-platforms
@azure-pipelines
azure-pipelinesBot
commented
Sep 14, 2023

Copy link


Copy Markdown


Azure Pipelines successfully started running 1 pipeline(s).

This was referenced Sep 14, 2023

Open


Closed


Closed


Closed


Closed


Closed


Closed


Open


Open

@kotlarmilos
kotlarmilos
commented
Sep 15, 2023

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-extra-platforms
@azure-pipelines
azure-pipelinesBot
commented
Sep 15, 2023

Copy link


Copy Markdown


Azure Pipelines successfully started running 1 pipeline(s).
@build-analysisbuild-analysisBot
mentioned this pull request
Sep 15, 2023

Closed

@kotlarmilos
kotlarmilos
commented
Sep 15, 2023

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-extra-platforms
@azure-pipelines
azure-pipelinesBot
commented
Sep 15, 2023

Copy link


Copy Markdown


Azure Pipelines successfully started running 1 pipeline(s).
@kotlarmilos
kotlarmilos
commented
Sep 15, 2023

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-extra-platforms
@azure-pipelines
azure-pipelinesBot
commented
Sep 15, 2023

Copy link


Copy Markdown


Azure Pipelines successfully started running 1 pipeline(s).
@kotlarmilos
kotlarmilos
commented
Sep 16, 2023

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-extra-platforms
@kotlarmilos
kotlarmilos
commented
Apr 2, 2024

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-extra-platforms
@kotlarmilos
kotlarmilos
commented
Apr 3, 2024

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-extra-platforms
@kotlarmilos
kotlarmilos
commented
Apr 3, 2024

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-extra-platforms
@kotlarmilos
kotlarmilos
commented
Apr 3, 2024

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-extra-platforms
@kotlarmilos
kotlarmilos
commented
Apr 3, 2024

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-extra-platforms
@kotlarmiloskotlarmilos
changed the title
[mono][infra] Enable linux-x64 Mono fullAOT llvm job on extra platforms[mono][infra] Enable runtime tests in fullAOT LLVM mode on linux-x64Apr 3, 2024
@kotlarmilos
kotlarmilos
marked this pull request as ready for review
April 3, 2024 18:17
@kotlarmiloskotlarmilos
removed
the 
blocked
Issue/PR is blocked on something - see comments
label
Apr 3, 2024
@kotlarmilos
kotlarmilos
commented
Apr 4, 2024

Copy link


Copy Markdown

MemberAuthor
This PR is ready for review. Failures shouldn't be related.
Runtime tests: https://dev.azure.com/dnceng-public/public/_build/results?buildId=629337&view=logs&jobId=d29f5ac9-2776-59d5-99ad-be6d4bcd416c&j=d29f5ac9-2776-59d5-99ad-be6d4bcd416c

Runtime intrinsics tests: https://dev.azure.com/dnceng-public/public/_build/results?buildId=629337&view=logs&jobId=d29f5ac9-2776-59d5-99ad-be6d4bcd416c&j=8c8e10f8-cac0-5393-e608-4ea6cc126022
matouskozak
matouskozak
approved these changes
Apr 4, 2024
View reviewed changes
@matouskozakmatouskozak
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! good job
Comment threadeng/pipelines/extra-platforms/runtime-extra-platforms-other.yml
Outdated

 testGroup: innerloop
 nameSuffix: AllSubsets_Mono_LLVMFullAot_RuntimeTests_Tests
 runtimeVariant: llvmfullaot
 buildArgs: -s mono+libs+clr.hosts+clr.iltools -c Release /p:MonoEnableLLVM=true /p:MonoBundleLLVMOptimizer=true
@matouskozakmatouskozakApr 4, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if it is needed in this PR but we should consolidate which flags are used for fullAOT/AOT jobs across our builds. For example, for perf jobs we use /p:MonoAOTEnableLLVM=true /p:MonoEnableLLVM=true /p:BuildMonoAOTCrossCompiler=truehttps://github.com/matouskozak/runtime/blob/12f546416d4aeb23b980a92e407fbf693823236d/eng/pipelines/coreclr/perf-non-wasm-jobs.yml#L39
@kotlarmiloskotlarmilosApr 4, 2024

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. This command only builds the runtime.
The AOT compiler is built in an optional step which contains required parameters:
runtime/eng/pipelines/common/templates/runtimes/build-runtime-tests-and-send-to-helix.yml

Lines 42 to 63
in
e610649
 - ${{ if and(eq(parameters.runtimeFlavor, 'mono'), in(parameters.runtimeVariant, 'llvmaot', 'llvmfullaot', 'minifullaot')) }}: 
 - ${{ if eq(parameters.archType, 'arm64') }}: 
 - ${{ if eq(parameters.runtimeVariant, 'minifullaot') }}: 
 - script: ./build.sh
-subset mono
-c ${{ parameters.buildConfig }}
-arch ${{ parameters.archType }}
/p:BuildMonoAotCrossCompiler=true
/p:BuildMonoAotCrossCompilerOnly=true
/p:CrossBuild=true
displayName: "Build Mono Mini AOT cross compiler"
 - ${{ else }}: 
 - script: ./build.sh
-subset mono
-c ${{ parameters.buildConfig }}
-arch ${{ parameters.archType }}
/p:BuildMonoAotCrossCompiler=true
/p:BuildMonoAotCrossCompilerOnly=true
/p:MonoLibClang="/usr/local/lib/libclang.so.16"
/p:MonoAOTEnableLLVM=true
/p:CrossBuild=true
displayName: "Build Mono LLVM AOT cross compiler"
@matouskozakmatouskozakApr 4, 2024•

edited

Loading

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we align on the use of MonoBundleLLVMOptimizer vs MonoAOTEnableLLVM then? Is the MonoBundleLLVMOptimizer=true necessary then? Looks like MonoEnableLLVM=true should be enough 
runtime/src/mono/mono.proj

Line 46
in
c7ada75
 <MonoBundleLLVMOptimizerCondition="'$(MonoEnableLLVM)' == 'true'">true</MonoBundleLLVMOptimizer> 
Comment threadeng/pipelines/extra-platforms/runtime-extra-platforms-other.yml
Outdated

 testGroup: innerloop
 nameSuffix: AllSubsets_Mono_LLVMFullAot_RuntimeIntrinsicsTests
 runtimeVariant: llvmfullaot
 buildArgs: -s mono+libs+clr.hosts+clr.iltools -c Release /p:MonoEnableLLVM=true /p:MonoBundleLLVMOptimizer=true
@matouskozakmatouskozakApr 4, 2024•

edited

Loading

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if it is possible with Helix, but could we re-use the build from AllSubsets_Mono_LLVMFullAot_RuntimeTests_Tests job? The runtime build should be the same, the difference is only which tests get build...
@kotlarmiloskotlarmilosApr 4, 2024•

edited

Loading

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct, perhaps we can create a template. @fanyang-mono Should this be moved to runtime-llvm.yml?
@kotlarmiloskotlarmilosApr 5, 2024

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved to the runtime-llvm.yml and fixed llvmaot job.
@kotlarmilos
kotlarmilos
commented
Apr 4, 2024•

edited

Loading

Copy link


Copy Markdown

MemberAuthor
These jobs have been moved to runtime-llvm.yml. Test run: https://dev.azure.com/dnceng-public/public/_build/results?buildId=630560&view=results
@kotlarmilos
kotlarmilos
commented
Apr 5, 2024

Copy link


Copy Markdown

MemberAuthor
The failures shouldn't be related.
@kotlarmilos
kotlarmilos
merged commit ddae9bd
into
dotnet:mainApr 5, 2024
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@matouskozakmatouskozakmatouskozak approved these changes
@lambdageeklambdageekAwaiting requested review from lambdageek
@SamMonoRTSamMonoRTAwaiting requested review from SamMonoRT
@directhexdirecthexAwaiting requested review from directhex
@steveisoksteveisokAwaiting requested review from steveisok
@fanyang-monofanyang-monoAwaiting requested review from fanyang-mono

+1 more reviewer

@vargazvargazvargaz approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

@kotlarmiloskotlarmilos

Labels

area-Infrastructure-mono

Projects


None yet


Milestone

9.0.0

Development

Successfully merging this pull request may close these issues.

5 participants

@kotlarmilos@vargaz@lambdageek@fanyang-mono@matouskozak

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Force GitHub README to respect dark mode
(function() {
var style = document.createElement('style');
style.textContent = '
.markdown-body {
color-scheme: dark light;
}
.markdown-body pre { background: #161b22 !important; }
.markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }
.markdown-body table th, .markdown-body table td { border-color: #30363d !important; }
.markdown-body img { background: #0d1117; }
.markdown-body blockquote { border-left-color: #8b949e; }
.markdown-body hr { border-color: #30363d; }
';
document.head.appendChild(style);
})();
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
[mono][infra] Enable runtime tests in fullAOT LLVM mode on linux-x64 by kotlarmilos · Pull Request #92057 · dotnet/runtime · GitHub
Skip to content
[mono][infra] Enable runtime tests in fullAOT LLVM mode on linux-x64 - #92057
Merged
kotlarmilos merged 110 commits into
dotnet:mainfrom 
kotlarmilos:bugfix/linux-x64-fullaot-job
Apr 5, 2024
Merged
[mono][infra] Enable runtime tests in fullAOT LLVM mode on linux-x64#92057
kotlarmilos merged 110 commits into
dotnet:mainfrom 
kotlarmilos:bugfix/linux-x64-fullaot-job
Conversation
@kotlarmilos
@kotlarmiloskotlarmilos
commented
Sep 14, 2023•

edited

Loading

Copy link


Copy Markdown

Member
Description
This PR enables jobs to run runtime tests in fullAOT LLVM mode on linux-x64. The jobs utilize the CBL-Mariner docker image, which contains the clang toolchain instead of binutils. Due to OOM and timeout issues, the tests are split into two jobs: intrinsics and other runtime tests.
Failing tests have been disabled and will be triaged in subsequent PRs. The arm64 support will be introduced in subsequent PRs. A list of disabled tests is included in #90427.
@kotlarmiloskotlarmilos
added

NO-REVIEW
Experimental/testing PR, do NOT review it
area-Infrastructure-mono

labels
Sep 14, 2023
@kotlarmiloskotlarmilos
added this to the 9.0.0 milestone
Sep 14, 2023
@ghost
ghost
commented
Sep 14, 2023

Copy link


Copy Markdown

Tagging subscribers to this area: @directhex

See info in area-owners.md if you want to be subscribed.
Issue Details
Work in progress.
This PR tests the linux-x64 fullAOT CI job.
Author:kotlarmilos
Assignees:-
Labels:
NO-REVIEW, area-Infrastructure-mono
Milestone:-
@kotlarmiloskotlarmilos
added
the 
NO-MERGE
The PR is not ready for merge yet (see discussion for detailed reasons)
label
Sep 14, 2023
@kotlarmilos
kotlarmilos
commented
Sep 14, 2023

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-extra-platforms
@azure-pipelines
azure-pipelinesBot
commented
Sep 14, 2023

Copy link


Copy Markdown


Azure Pipelines successfully started running 1 pipeline(s).

This was referenced Sep 14, 2023

Open


Closed


Closed


Closed


Closed


Closed


Closed


Open


Open

@kotlarmilos
kotlarmilos
commented
Sep 15, 2023

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-extra-platforms
@azure-pipelines
azure-pipelinesBot
commented
Sep 15, 2023

Copy link


Copy Markdown


Azure Pipelines successfully started running 1 pipeline(s).
@build-analysisbuild-analysisBot
mentioned this pull request
Sep 15, 2023

Closed

@kotlarmilos
kotlarmilos
commented
Sep 15, 2023

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-extra-platforms
@azure-pipelines
azure-pipelinesBot
commented
Sep 15, 2023

Copy link


Copy Markdown


Azure Pipelines successfully started running 1 pipeline(s).
@kotlarmilos
kotlarmilos
commented
Sep 15, 2023

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-extra-platforms
@azure-pipelines
azure-pipelinesBot
commented
Sep 15, 2023

Copy link


Copy Markdown


Azure Pipelines successfully started running 1 pipeline(s).
@kotlarmilos
kotlarmilos
commented
Sep 16, 2023

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-extra-platforms
@kotlarmilos
kotlarmilos
commented
Apr 2, 2024

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-extra-platforms
@kotlarmilos
kotlarmilos
commented
Apr 3, 2024

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-extra-platforms
@kotlarmilos
kotlarmilos
commented
Apr 3, 2024

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-extra-platforms
@kotlarmilos
kotlarmilos
commented
Apr 3, 2024

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-extra-platforms
@kotlarmilos
kotlarmilos
commented
Apr 3, 2024

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-extra-platforms
@kotlarmiloskotlarmilos
changed the title
[mono][infra] Enable linux-x64 Mono fullAOT llvm job on extra platforms[mono][infra] Enable runtime tests in fullAOT LLVM mode on linux-x64Apr 3, 2024
@kotlarmilos
kotlarmilos
marked this pull request as ready for review
April 3, 2024 18:17
@kotlarmiloskotlarmilos
removed
the 
blocked
Issue/PR is blocked on something - see comments
label
Apr 3, 2024
@kotlarmilos
kotlarmilos
commented
Apr 4, 2024

Copy link


Copy Markdown

MemberAuthor
This PR is ready for review. Failures shouldn't be related.
Runtime tests: https://dev.azure.com/dnceng-public/public/_build/results?buildId=629337&view=logs&jobId=d29f5ac9-2776-59d5-99ad-be6d4bcd416c&j=d29f5ac9-2776-59d5-99ad-be6d4bcd416c

Runtime intrinsics tests: https://dev.azure.com/dnceng-public/public/_build/results?buildId=629337&view=logs&jobId=d29f5ac9-2776-59d5-99ad-be6d4bcd416c&j=8c8e10f8-cac0-5393-e608-4ea6cc126022
matouskozak
matouskozak
approved these changes
Apr 4, 2024
View reviewed changes
@matouskozakmatouskozak
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! good job
Comment threadeng/pipelines/extra-platforms/runtime-extra-platforms-other.yml
Outdated

 testGroup: innerloop
 nameSuffix: AllSubsets_Mono_LLVMFullAot_RuntimeTests_Tests
 runtimeVariant: llvmfullaot
 buildArgs: -s mono+libs+clr.hosts+clr.iltools -c Release /p:MonoEnableLLVM=true /p:MonoBundleLLVMOptimizer=true
@matouskozakmatouskozakApr 4, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if it is needed in this PR but we should consolidate which flags are used for fullAOT/AOT jobs across our builds. For example, for perf jobs we use /p:MonoAOTEnableLLVM=true /p:MonoEnableLLVM=true /p:BuildMonoAOTCrossCompiler=truehttps://github.com/matouskozak/runtime/blob/12f546416d4aeb23b980a92e407fbf693823236d/eng/pipelines/coreclr/perf-non-wasm-jobs.yml#L39
@kotlarmiloskotlarmilosApr 4, 2024

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. This command only builds the runtime.
The AOT compiler is built in an optional step which contains required parameters:
runtime/eng/pipelines/common/templates/runtimes/build-runtime-tests-and-send-to-helix.yml

Lines 42 to 63
in
e610649
 - ${{ if and(eq(parameters.runtimeFlavor, 'mono'), in(parameters.runtimeVariant, 'llvmaot', 'llvmfullaot', 'minifullaot')) }}: 
 - ${{ if eq(parameters.archType, 'arm64') }}: 
 - ${{ if eq(parameters.runtimeVariant, 'minifullaot') }}: 
 - script: ./build.sh
-subset mono
-c ${{ parameters.buildConfig }}
-arch ${{ parameters.archType }}
/p:BuildMonoAotCrossCompiler=true
/p:BuildMonoAotCrossCompilerOnly=true
/p:CrossBuild=true
displayName: "Build Mono Mini AOT cross compiler"
 - ${{ else }}: 
 - script: ./build.sh
-subset mono
-c ${{ parameters.buildConfig }}
-arch ${{ parameters.archType }}
/p:BuildMonoAotCrossCompiler=true
/p:BuildMonoAotCrossCompilerOnly=true
/p:MonoLibClang="/usr/local/lib/libclang.so.16"
/p:MonoAOTEnableLLVM=true
/p:CrossBuild=true
displayName: "Build Mono LLVM AOT cross compiler"
@matouskozakmatouskozakApr 4, 2024•

edited

Loading

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we align on the use of MonoBundleLLVMOptimizer vs MonoAOTEnableLLVM then? Is the MonoBundleLLVMOptimizer=true necessary then? Looks like MonoEnableLLVM=true should be enough 
runtime/src/mono/mono.proj

Line 46
in
c7ada75
 <MonoBundleLLVMOptimizerCondition="'$(MonoEnableLLVM)' == 'true'">true</MonoBundleLLVMOptimizer> 
Comment threadeng/pipelines/extra-platforms/runtime-extra-platforms-other.yml
Outdated

 testGroup: innerloop
 nameSuffix: AllSubsets_Mono_LLVMFullAot_RuntimeIntrinsicsTests
 runtimeVariant: llvmfullaot
 buildArgs: -s mono+libs+clr.hosts+clr.iltools -c Release /p:MonoEnableLLVM=true /p:MonoBundleLLVMOptimizer=true
@matouskozakmatouskozakApr 4, 2024•

edited

Loading

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if it is possible with Helix, but could we re-use the build from AllSubsets_Mono_LLVMFullAot_RuntimeTests_Tests job? The runtime build should be the same, the difference is only which tests get build...
@kotlarmiloskotlarmilosApr 4, 2024•

edited

Loading

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct, perhaps we can create a template. @fanyang-mono Should this be moved to runtime-llvm.yml?
@kotlarmiloskotlarmilosApr 5, 2024

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved to the runtime-llvm.yml and fixed llvmaot job.
@kotlarmilos
kotlarmilos
commented
Apr 4, 2024•

edited

Loading

Copy link


Copy Markdown

MemberAuthor
These jobs have been moved to runtime-llvm.yml. Test run: https://dev.azure.com/dnceng-public/public/_build/results?buildId=630560&view=results
@kotlarmilos
kotlarmilos
commented
Apr 5, 2024

Copy link


Copy Markdown

MemberAuthor
The failures shouldn't be related.
@kotlarmilos
kotlarmilos
merged commit ddae9bd
into
dotnet:mainApr 5, 2024
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@matouskozakmatouskozakmatouskozak approved these changes
@lambdageeklambdageekAwaiting requested review from lambdageek
@SamMonoRTSamMonoRTAwaiting requested review from SamMonoRT
@directhexdirecthexAwaiting requested review from directhex
@steveisoksteveisokAwaiting requested review from steveisok
@fanyang-monofanyang-monoAwaiting requested review from fanyang-mono

+1 more reviewer

@vargazvargazvargaz approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

@kotlarmiloskotlarmilos

Labels

area-Infrastructure-mono

Projects


None yet


Milestone

9.0.0

Development

Successfully merging this pull request may close these issues.

5 participants

@kotlarmilos@vargaz@lambdageek@fanyang-mono@matouskozak

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Highlight search terms from Google/DuckDuckGo/Bing referrer
(function() {
var ref = document.referrer;
var terms = [];
if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {
var url = new URL(ref);
var q = url.searchParams.get('q') || url.searchParams.get('p');
if (q) {
terms = q.split(/\s+/).filter(function(t) { return t.length > 2; });
}
}
if (terms.length === 0) return;
var style = document.createElement('style');
style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';
document.head.appendChild(style);
function highlight(node) {
if (node.nodeType === 3) { // text node
var text = node.textContent;
var found = false;
terms.forEach(function(term) {
var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi');
if (regex.test(text)) {
found = true;
var frag = document.createDocumentFragment();
var parts = text.split(regex);
parts.forEach(function(part, i) {
if (i % 2 === 0) {
frag.appendChild(document.createTextNode(part));
} else {
var span = document.createElement('span');
span.className = 'userscript-highlight';
span.textContent = part;
frag.appendChild(span);
}
});
node.parentNode.replaceChild(frag, node);
}
});
} else if (node.nodeType === 1 && node.childNodes) { // element
var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];
if (!skipTags.includes(node.tagName)) {
Array.from(node.childNodes).forEach(highlight);
}
}
}
highlight(document.body);
// Re-highlight on dynamic content
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1 || node.nodeType === 3) highlight(node);
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
[mono][infra] Enable runtime tests in fullAOT LLVM mode on linux-x64 by kotlarmilos · Pull Request #92057 · dotnet/runtime · GitHub
Skip to content
[mono][infra] Enable runtime tests in fullAOT LLVM mode on linux-x64 - #92057
Merged
kotlarmilos merged 110 commits into
dotnet:mainfrom 
kotlarmilos:bugfix/linux-x64-fullaot-job
Apr 5, 2024
Merged
[mono][infra] Enable runtime tests in fullAOT LLVM mode on linux-x64#92057
kotlarmilos merged 110 commits into
dotnet:mainfrom 
kotlarmilos:bugfix/linux-x64-fullaot-job
Conversation
@kotlarmilos
@kotlarmiloskotlarmilos
commented
Sep 14, 2023•

edited

Loading

Copy link


Copy Markdown

Member
Description
This PR enables jobs to run runtime tests in fullAOT LLVM mode on linux-x64. The jobs utilize the CBL-Mariner docker image, which contains the clang toolchain instead of binutils. Due to OOM and timeout issues, the tests are split into two jobs: intrinsics and other runtime tests.
Failing tests have been disabled and will be triaged in subsequent PRs. The arm64 support will be introduced in subsequent PRs. A list of disabled tests is included in #90427.
@kotlarmiloskotlarmilos
added

NO-REVIEW
Experimental/testing PR, do NOT review it
area-Infrastructure-mono

labels
Sep 14, 2023
@kotlarmiloskotlarmilos
added this to the 9.0.0 milestone
Sep 14, 2023
@ghost
ghost
commented
Sep 14, 2023

Copy link


Copy Markdown

Tagging subscribers to this area: @directhex

See info in area-owners.md if you want to be subscribed.
Issue Details
Work in progress.
This PR tests the linux-x64 fullAOT CI job.
Author:kotlarmilos
Assignees:-
Labels:
NO-REVIEW, area-Infrastructure-mono
Milestone:-
@kotlarmiloskotlarmilos
added
the 
NO-MERGE
The PR is not ready for merge yet (see discussion for detailed reasons)
label
Sep 14, 2023
@kotlarmilos
kotlarmilos
commented
Sep 14, 2023

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-extra-platforms
@azure-pipelines
azure-pipelinesBot
commented
Sep 14, 2023

Copy link


Copy Markdown


Azure Pipelines successfully started running 1 pipeline(s).

This was referenced Sep 14, 2023

Open


Closed


Closed


Closed


Closed


Closed


Closed


Open


Open

@kotlarmilos
kotlarmilos
commented
Sep 15, 2023

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-extra-platforms
@azure-pipelines
azure-pipelinesBot
commented
Sep 15, 2023

Copy link


Copy Markdown


Azure Pipelines successfully started running 1 pipeline(s).
@build-analysisbuild-analysisBot
mentioned this pull request
Sep 15, 2023

Closed

@kotlarmilos
kotlarmilos
commented
Sep 15, 2023

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-extra-platforms
@azure-pipelines
azure-pipelinesBot
commented
Sep 15, 2023

Copy link


Copy Markdown


Azure Pipelines successfully started running 1 pipeline(s).
@kotlarmilos
kotlarmilos
commented
Sep 15, 2023

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-extra-platforms
@azure-pipelines
azure-pipelinesBot
commented
Sep 15, 2023

Copy link


Copy Markdown


Azure Pipelines successfully started running 1 pipeline(s).
@kotlarmilos
kotlarmilos
commented
Sep 16, 2023

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-extra-platforms
@kotlarmilos
kotlarmilos
commented
Apr 2, 2024

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-extra-platforms
@kotlarmilos
kotlarmilos
commented
Apr 3, 2024

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-extra-platforms
@kotlarmilos
kotlarmilos
commented
Apr 3, 2024

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-extra-platforms
@kotlarmilos
kotlarmilos
commented
Apr 3, 2024

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-extra-platforms
@kotlarmilos
kotlarmilos
commented
Apr 3, 2024

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-extra-platforms
@kotlarmiloskotlarmilos
changed the title
[mono][infra] Enable linux-x64 Mono fullAOT llvm job on extra platforms[mono][infra] Enable runtime tests in fullAOT LLVM mode on linux-x64Apr 3, 2024
@kotlarmilos
kotlarmilos
marked this pull request as ready for review
April 3, 2024 18:17
@kotlarmiloskotlarmilos
removed
the 
blocked
Issue/PR is blocked on something - see comments
label
Apr 3, 2024
@kotlarmilos
kotlarmilos
commented
Apr 4, 2024

Copy link


Copy Markdown

MemberAuthor
This PR is ready for review. Failures shouldn't be related.
Runtime tests: https://dev.azure.com/dnceng-public/public/_build/results?buildId=629337&view=logs&jobId=d29f5ac9-2776-59d5-99ad-be6d4bcd416c&j=d29f5ac9-2776-59d5-99ad-be6d4bcd416c

Runtime intrinsics tests: https://dev.azure.com/dnceng-public/public/_build/results?buildId=629337&view=logs&jobId=d29f5ac9-2776-59d5-99ad-be6d4bcd416c&j=8c8e10f8-cac0-5393-e608-4ea6cc126022
matouskozak
matouskozak
approved these changes
Apr 4, 2024
View reviewed changes
@matouskozakmatouskozak
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! good job
Comment threadeng/pipelines/extra-platforms/runtime-extra-platforms-other.yml
Outdated

 testGroup: innerloop
 nameSuffix: AllSubsets_Mono_LLVMFullAot_RuntimeTests_Tests
 runtimeVariant: llvmfullaot
 buildArgs: -s mono+libs+clr.hosts+clr.iltools -c Release /p:MonoEnableLLVM=true /p:MonoBundleLLVMOptimizer=true
@matouskozakmatouskozakApr 4, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if it is needed in this PR but we should consolidate which flags are used for fullAOT/AOT jobs across our builds. For example, for perf jobs we use /p:MonoAOTEnableLLVM=true /p:MonoEnableLLVM=true /p:BuildMonoAOTCrossCompiler=truehttps://github.com/matouskozak/runtime/blob/12f546416d4aeb23b980a92e407fbf693823236d/eng/pipelines/coreclr/perf-non-wasm-jobs.yml#L39
@kotlarmiloskotlarmilosApr 4, 2024

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. This command only builds the runtime.
The AOT compiler is built in an optional step which contains required parameters:
runtime/eng/pipelines/common/templates/runtimes/build-runtime-tests-and-send-to-helix.yml

Lines 42 to 63
in
e610649
 - ${{ if and(eq(parameters.runtimeFlavor, 'mono'), in(parameters.runtimeVariant, 'llvmaot', 'llvmfullaot', 'minifullaot')) }}: 
 - ${{ if eq(parameters.archType, 'arm64') }}: 
 - ${{ if eq(parameters.runtimeVariant, 'minifullaot') }}: 
 - script: ./build.sh
-subset mono
-c ${{ parameters.buildConfig }}
-arch ${{ parameters.archType }}
/p:BuildMonoAotCrossCompiler=true
/p:BuildMonoAotCrossCompilerOnly=true
/p:CrossBuild=true
displayName: "Build Mono Mini AOT cross compiler"
 - ${{ else }}: 
 - script: ./build.sh
-subset mono
-c ${{ parameters.buildConfig }}
-arch ${{ parameters.archType }}
/p:BuildMonoAotCrossCompiler=true
/p:BuildMonoAotCrossCompilerOnly=true
/p:MonoLibClang="/usr/local/lib/libclang.so.16"
/p:MonoAOTEnableLLVM=true
/p:CrossBuild=true
displayName: "Build Mono LLVM AOT cross compiler"
@matouskozakmatouskozakApr 4, 2024•

edited

Loading

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we align on the use of MonoBundleLLVMOptimizer vs MonoAOTEnableLLVM then? Is the MonoBundleLLVMOptimizer=true necessary then? Looks like MonoEnableLLVM=true should be enough 
runtime/src/mono/mono.proj

Line 46
in
c7ada75
 <MonoBundleLLVMOptimizerCondition="'$(MonoEnableLLVM)' == 'true'">true</MonoBundleLLVMOptimizer> 
Comment threadeng/pipelines/extra-platforms/runtime-extra-platforms-other.yml
Outdated

 testGroup: innerloop
 nameSuffix: AllSubsets_Mono_LLVMFullAot_RuntimeIntrinsicsTests
 runtimeVariant: llvmfullaot
 buildArgs: -s mono+libs+clr.hosts+clr.iltools -c Release /p:MonoEnableLLVM=true /p:MonoBundleLLVMOptimizer=true
@matouskozakmatouskozakApr 4, 2024•

edited

Loading

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if it is possible with Helix, but could we re-use the build from AllSubsets_Mono_LLVMFullAot_RuntimeTests_Tests job? The runtime build should be the same, the difference is only which tests get build...
@kotlarmiloskotlarmilosApr 4, 2024•

edited

Loading

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct, perhaps we can create a template. @fanyang-mono Should this be moved to runtime-llvm.yml?
@kotlarmiloskotlarmilosApr 5, 2024

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved to the runtime-llvm.yml and fixed llvmaot job.
@kotlarmilos
kotlarmilos
commented
Apr 4, 2024•

edited

Loading

Copy link


Copy Markdown

MemberAuthor
These jobs have been moved to runtime-llvm.yml. Test run: https://dev.azure.com/dnceng-public/public/_build/results?buildId=630560&view=results
@kotlarmilos
kotlarmilos
commented
Apr 5, 2024

Copy link


Copy Markdown

MemberAuthor
The failures shouldn't be related.
@kotlarmilos
kotlarmilos
merged commit ddae9bd
into
dotnet:mainApr 5, 2024
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@matouskozakmatouskozakmatouskozak approved these changes
@lambdageeklambdageekAwaiting requested review from lambdageek
@SamMonoRTSamMonoRTAwaiting requested review from SamMonoRT
@directhexdirecthexAwaiting requested review from directhex
@steveisoksteveisokAwaiting requested review from steveisok
@fanyang-monofanyang-monoAwaiting requested review from fanyang-mono

+1 more reviewer

@vargazvargazvargaz approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

@kotlarmiloskotlarmilos

Labels

area-Infrastructure-mono

Projects


None yet


Milestone

9.0.0

Development

Successfully merging this pull request may close these issues.

5 participants

@kotlarmilos@vargaz@lambdageek@fanyang-mono@matouskozak

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Strip utm_, fbclid, gclid, etc. from all links on page
(function() {
var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',
'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',
'ref', 'ref_src', 'source', 'medium', 'campaign'];
function cleanUrl(url) {
try {
var u = new URL(url, window.location.origin);
var changed = false;
trackingParams.forEach(function(p) {
if (u.searchParams.has(p)) {
u.searchParams.delete(p);
changed = true;
}
});
return changed ? u.toString() : url;
} catch (e) {
return url;
}
}
function cleanLinks() {
document.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
cleanLinks();
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1) {
if (node.tagName === 'A') cleanLinks();
node.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '
[mono][infra] Enable runtime tests in fullAOT LLVM mode on linux-x64 by kotlarmilos · Pull Request #92057 · dotnet/runtime · GitHub
Skip to content
[mono][infra] Enable runtime tests in fullAOT LLVM mode on linux-x64 - #92057
Merged
kotlarmilos merged 110 commits into
dotnet:mainfrom 
kotlarmilos:bugfix/linux-x64-fullaot-job
Apr 5, 2024
Merged
[mono][infra] Enable runtime tests in fullAOT LLVM mode on linux-x64#92057
kotlarmilos merged 110 commits into
dotnet:mainfrom 
kotlarmilos:bugfix/linux-x64-fullaot-job
Conversation
@kotlarmilos
@kotlarmiloskotlarmilos
commented
Sep 14, 2023•

edited

Loading

Copy link


Copy Markdown

Member
Description
This PR enables jobs to run runtime tests in fullAOT LLVM mode on linux-x64. The jobs utilize the CBL-Mariner docker image, which contains the clang toolchain instead of binutils. Due to OOM and timeout issues, the tests are split into two jobs: intrinsics and other runtime tests.
Failing tests have been disabled and will be triaged in subsequent PRs. The arm64 support will be introduced in subsequent PRs. A list of disabled tests is included in #90427.
@kotlarmiloskotlarmilos
added

NO-REVIEW
Experimental/testing PR, do NOT review it
area-Infrastructure-mono

labels
Sep 14, 2023
@kotlarmiloskotlarmilos
added this to the 9.0.0 milestone
Sep 14, 2023
@ghost
ghost
commented
Sep 14, 2023

Copy link


Copy Markdown

Tagging subscribers to this area: @directhex

See info in area-owners.md if you want to be subscribed.
Issue Details
Work in progress.
This PR tests the linux-x64 fullAOT CI job.
Author:kotlarmilos
Assignees:-
Labels:
NO-REVIEW, area-Infrastructure-mono
Milestone:-
@kotlarmiloskotlarmilos
added
the 
NO-MERGE
The PR is not ready for merge yet (see discussion for detailed reasons)
label
Sep 14, 2023
@kotlarmilos
kotlarmilos
commented
Sep 14, 2023

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-extra-platforms
@azure-pipelines
azure-pipelinesBot
commented
Sep 14, 2023

Copy link


Copy Markdown


Azure Pipelines successfully started running 1 pipeline(s).

This was referenced Sep 14, 2023

Open


Closed


Closed


Closed


Closed


Closed


Closed


Open


Open

@kotlarmilos
kotlarmilos
commented
Sep 15, 2023

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-extra-platforms
@azure-pipelines
azure-pipelinesBot
commented
Sep 15, 2023

Copy link


Copy Markdown


Azure Pipelines successfully started running 1 pipeline(s).
@build-analysisbuild-analysisBot
mentioned this pull request
Sep 15, 2023

Closed

@kotlarmilos
kotlarmilos
commented
Sep 15, 2023

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-extra-platforms
@azure-pipelines
azure-pipelinesBot
commented
Sep 15, 2023

Copy link


Copy Markdown


Azure Pipelines successfully started running 1 pipeline(s).
@kotlarmilos
kotlarmilos
commented
Sep 15, 2023

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-extra-platforms
@azure-pipelines
azure-pipelinesBot
commented
Sep 15, 2023

Copy link


Copy Markdown


Azure Pipelines successfully started running 1 pipeline(s).
@kotlarmilos
kotlarmilos
commented
Sep 16, 2023

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-extra-platforms
@kotlarmilos
kotlarmilos
commented
Apr 2, 2024

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-extra-platforms
@kotlarmilos
kotlarmilos
commented
Apr 3, 2024

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-extra-platforms
@kotlarmilos
kotlarmilos
commented
Apr 3, 2024

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-extra-platforms
@kotlarmilos
kotlarmilos
commented
Apr 3, 2024

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-extra-platforms
@kotlarmilos
kotlarmilos
commented
Apr 3, 2024

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-extra-platforms
@kotlarmiloskotlarmilos
changed the title
[mono][infra] Enable linux-x64 Mono fullAOT llvm job on extra platforms[mono][infra] Enable runtime tests in fullAOT LLVM mode on linux-x64Apr 3, 2024
@kotlarmilos
kotlarmilos
marked this pull request as ready for review
April 3, 2024 18:17
@kotlarmiloskotlarmilos
removed
the 
blocked
Issue/PR is blocked on something - see comments
label
Apr 3, 2024
@kotlarmilos
kotlarmilos
commented
Apr 4, 2024

Copy link


Copy Markdown

MemberAuthor
This PR is ready for review. Failures shouldn't be related.
Runtime tests: https://dev.azure.com/dnceng-public/public/_build/results?buildId=629337&view=logs&jobId=d29f5ac9-2776-59d5-99ad-be6d4bcd416c&j=d29f5ac9-2776-59d5-99ad-be6d4bcd416c

Runtime intrinsics tests: https://dev.azure.com/dnceng-public/public/_build/results?buildId=629337&view=logs&jobId=d29f5ac9-2776-59d5-99ad-be6d4bcd416c&j=8c8e10f8-cac0-5393-e608-4ea6cc126022
matouskozak
matouskozak
approved these changes
Apr 4, 2024
View reviewed changes
@matouskozakmatouskozak
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! good job
Comment threadeng/pipelines/extra-platforms/runtime-extra-platforms-other.yml
Outdated

 testGroup: innerloop
 nameSuffix: AllSubsets_Mono_LLVMFullAot_RuntimeTests_Tests
 runtimeVariant: llvmfullaot
 buildArgs: -s mono+libs+clr.hosts+clr.iltools -c Release /p:MonoEnableLLVM=true /p:MonoBundleLLVMOptimizer=true
@matouskozakmatouskozakApr 4, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if it is needed in this PR but we should consolidate which flags are used for fullAOT/AOT jobs across our builds. For example, for perf jobs we use /p:MonoAOTEnableLLVM=true /p:MonoEnableLLVM=true /p:BuildMonoAOTCrossCompiler=truehttps://github.com/matouskozak/runtime/blob/12f546416d4aeb23b980a92e407fbf693823236d/eng/pipelines/coreclr/perf-non-wasm-jobs.yml#L39
@kotlarmiloskotlarmilosApr 4, 2024

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. This command only builds the runtime.
The AOT compiler is built in an optional step which contains required parameters:
runtime/eng/pipelines/common/templates/runtimes/build-runtime-tests-and-send-to-helix.yml

Lines 42 to 63
in
e610649
 - ${{ if and(eq(parameters.runtimeFlavor, 'mono'), in(parameters.runtimeVariant, 'llvmaot', 'llvmfullaot', 'minifullaot')) }}: 
 - ${{ if eq(parameters.archType, 'arm64') }}: 
 - ${{ if eq(parameters.runtimeVariant, 'minifullaot') }}: 
 - script: ./build.sh
-subset mono
-c ${{ parameters.buildConfig }}
-arch ${{ parameters.archType }}
/p:BuildMonoAotCrossCompiler=true
/p:BuildMonoAotCrossCompilerOnly=true
/p:CrossBuild=true
displayName: "Build Mono Mini AOT cross compiler"
 - ${{ else }}: 
 - script: ./build.sh
-subset mono
-c ${{ parameters.buildConfig }}
-arch ${{ parameters.archType }}
/p:BuildMonoAotCrossCompiler=true
/p:BuildMonoAotCrossCompilerOnly=true
/p:MonoLibClang="/usr/local/lib/libclang.so.16"
/p:MonoAOTEnableLLVM=true
/p:CrossBuild=true
displayName: "Build Mono LLVM AOT cross compiler"
@matouskozakmatouskozakApr 4, 2024•

edited

Loading

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we align on the use of MonoBundleLLVMOptimizer vs MonoAOTEnableLLVM then? Is the MonoBundleLLVMOptimizer=true necessary then? Looks like MonoEnableLLVM=true should be enough 
runtime/src/mono/mono.proj

Line 46
in
c7ada75
 <MonoBundleLLVMOptimizerCondition="'$(MonoEnableLLVM)' == 'true'">true</MonoBundleLLVMOptimizer> 
Comment threadeng/pipelines/extra-platforms/runtime-extra-platforms-other.yml
Outdated

 testGroup: innerloop
 nameSuffix: AllSubsets_Mono_LLVMFullAot_RuntimeIntrinsicsTests
 runtimeVariant: llvmfullaot
 buildArgs: -s mono+libs+clr.hosts+clr.iltools -c Release /p:MonoEnableLLVM=true /p:MonoBundleLLVMOptimizer=true
@matouskozakmatouskozakApr 4, 2024•

edited

Loading

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if it is possible with Helix, but could we re-use the build from AllSubsets_Mono_LLVMFullAot_RuntimeTests_Tests job? The runtime build should be the same, the difference is only which tests get build...
@kotlarmiloskotlarmilosApr 4, 2024•

edited

Loading

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct, perhaps we can create a template. @fanyang-mono Should this be moved to runtime-llvm.yml?
@kotlarmiloskotlarmilosApr 5, 2024

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved to the runtime-llvm.yml and fixed llvmaot job.
@kotlarmilos
kotlarmilos
commented
Apr 4, 2024•

edited

Loading

Copy link


Copy Markdown

MemberAuthor
These jobs have been moved to runtime-llvm.yml. Test run: https://dev.azure.com/dnceng-public/public/_build/results?buildId=630560&view=results
@kotlarmilos
kotlarmilos
commented
Apr 5, 2024

Copy link


Copy Markdown

MemberAuthor
The failures shouldn't be related.
@kotlarmilos
kotlarmilos
merged commit ddae9bd
into
dotnet:mainApr 5, 2024
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@matouskozakmatouskozakmatouskozak approved these changes
@lambdageeklambdageekAwaiting requested review from lambdageek
@SamMonoRTSamMonoRTAwaiting requested review from SamMonoRT
@directhexdirecthexAwaiting requested review from directhex
@steveisoksteveisokAwaiting requested review from steveisok
@fanyang-monofanyang-monoAwaiting requested review from fanyang-mono

+1 more reviewer

@vargazvargazvargaz approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

@kotlarmiloskotlarmilos

Labels

area-Infrastructure-mono

Projects


None yet


Milestone

9.0.0

Development

Successfully merging this pull request may close these issues.

5 participants

@kotlarmilos@vargaz@lambdageek@fanyang-mono@matouskozak

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Auto-enable theater mode on YouTube
(function() {
function tryTheater() {
var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]');
if (btn && !btn.classList.contains('activated')) {
btn.click();
}
}
// Try immediately
tryTheater();
// Try after navigation (SPA)
var lastUrl = location.href;
setInterval(function() {
if (location.href !== lastUrl) {
lastUrl = location.href;
setTimeout(tryTheater, 500);
}
}, 1000);
// Also try on player load
var observer = new MutationObserver(tryTheater);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
[mono][infra] Enable runtime tests in fullAOT LLVM mode on linux-x64 by kotlarmilos · Pull Request #92057 · dotnet/runtime · GitHub
Skip to content
[mono][infra] Enable runtime tests in fullAOT LLVM mode on linux-x64 - #92057
Merged
kotlarmilos merged 110 commits into
dotnet:mainfrom 
kotlarmilos:bugfix/linux-x64-fullaot-job
Apr 5, 2024
Merged
[mono][infra] Enable runtime tests in fullAOT LLVM mode on linux-x64#92057
kotlarmilos merged 110 commits into
dotnet:mainfrom 
kotlarmilos:bugfix/linux-x64-fullaot-job
Conversation
@kotlarmilos
@kotlarmiloskotlarmilos
commented
Sep 14, 2023•

edited

Loading

Copy link


Copy Markdown

Member
Description
This PR enables jobs to run runtime tests in fullAOT LLVM mode on linux-x64. The jobs utilize the CBL-Mariner docker image, which contains the clang toolchain instead of binutils. Due to OOM and timeout issues, the tests are split into two jobs: intrinsics and other runtime tests.
Failing tests have been disabled and will be triaged in subsequent PRs. The arm64 support will be introduced in subsequent PRs. A list of disabled tests is included in #90427.
@kotlarmiloskotlarmilos
added

NO-REVIEW
Experimental/testing PR, do NOT review it
area-Infrastructure-mono

labels
Sep 14, 2023
@kotlarmiloskotlarmilos
added this to the 9.0.0 milestone
Sep 14, 2023
@ghost
ghost
commented
Sep 14, 2023

Copy link


Copy Markdown

Tagging subscribers to this area: @directhex

See info in area-owners.md if you want to be subscribed.
Issue Details
Work in progress.
This PR tests the linux-x64 fullAOT CI job.
Author:kotlarmilos
Assignees:-
Labels:
NO-REVIEW, area-Infrastructure-mono
Milestone:-
@kotlarmiloskotlarmilos
added
the 
NO-MERGE
The PR is not ready for merge yet (see discussion for detailed reasons)
label
Sep 14, 2023
@kotlarmilos
kotlarmilos
commented
Sep 14, 2023

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-extra-platforms
@azure-pipelines
azure-pipelinesBot
commented
Sep 14, 2023

Copy link


Copy Markdown


Azure Pipelines successfully started running 1 pipeline(s).

This was referenced Sep 14, 2023

Open


Closed


Closed


Closed


Closed


Closed


Closed


Open


Open

@kotlarmilos
kotlarmilos
commented
Sep 15, 2023

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-extra-platforms
@azure-pipelines
azure-pipelinesBot
commented
Sep 15, 2023

Copy link


Copy Markdown


Azure Pipelines successfully started running 1 pipeline(s).
@build-analysisbuild-analysisBot
mentioned this pull request
Sep 15, 2023

Closed

@kotlarmilos
kotlarmilos
commented
Sep 15, 2023

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-extra-platforms
@azure-pipelines
azure-pipelinesBot
commented
Sep 15, 2023

Copy link


Copy Markdown


Azure Pipelines successfully started running 1 pipeline(s).
@kotlarmilos
kotlarmilos
commented
Sep 15, 2023

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-extra-platforms
@azure-pipelines
azure-pipelinesBot
commented
Sep 15, 2023

Copy link


Copy Markdown


Azure Pipelines successfully started running 1 pipeline(s).
@kotlarmilos
kotlarmilos
commented
Sep 16, 2023

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-extra-platforms
@kotlarmilos
kotlarmilos
commented
Apr 2, 2024

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-extra-platforms
@kotlarmilos
kotlarmilos
commented
Apr 3, 2024

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-extra-platforms
@kotlarmilos
kotlarmilos
commented
Apr 3, 2024

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-extra-platforms
@kotlarmilos
kotlarmilos
commented
Apr 3, 2024

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-extra-platforms
@kotlarmilos
kotlarmilos
commented
Apr 3, 2024

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-extra-platforms
@kotlarmiloskotlarmilos
changed the title
[mono][infra] Enable linux-x64 Mono fullAOT llvm job on extra platforms[mono][infra] Enable runtime tests in fullAOT LLVM mode on linux-x64Apr 3, 2024
@kotlarmilos
kotlarmilos
marked this pull request as ready for review
April 3, 2024 18:17
@kotlarmiloskotlarmilos
removed
the 
blocked
Issue/PR is blocked on something - see comments
label
Apr 3, 2024
@kotlarmilos
kotlarmilos
commented
Apr 4, 2024

Copy link


Copy Markdown

MemberAuthor
This PR is ready for review. Failures shouldn't be related.
Runtime tests: https://dev.azure.com/dnceng-public/public/_build/results?buildId=629337&view=logs&jobId=d29f5ac9-2776-59d5-99ad-be6d4bcd416c&j=d29f5ac9-2776-59d5-99ad-be6d4bcd416c

Runtime intrinsics tests: https://dev.azure.com/dnceng-public/public/_build/results?buildId=629337&view=logs&jobId=d29f5ac9-2776-59d5-99ad-be6d4bcd416c&j=8c8e10f8-cac0-5393-e608-4ea6cc126022
matouskozak
matouskozak
approved these changes
Apr 4, 2024
View reviewed changes
@matouskozakmatouskozak
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! good job
Comment threadeng/pipelines/extra-platforms/runtime-extra-platforms-other.yml
Outdated

 testGroup: innerloop
 nameSuffix: AllSubsets_Mono_LLVMFullAot_RuntimeTests_Tests
 runtimeVariant: llvmfullaot
 buildArgs: -s mono+libs+clr.hosts+clr.iltools -c Release /p:MonoEnableLLVM=true /p:MonoBundleLLVMOptimizer=true
@matouskozakmatouskozakApr 4, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if it is needed in this PR but we should consolidate which flags are used for fullAOT/AOT jobs across our builds. For example, for perf jobs we use /p:MonoAOTEnableLLVM=true /p:MonoEnableLLVM=true /p:BuildMonoAOTCrossCompiler=truehttps://github.com/matouskozak/runtime/blob/12f546416d4aeb23b980a92e407fbf693823236d/eng/pipelines/coreclr/perf-non-wasm-jobs.yml#L39
@kotlarmiloskotlarmilosApr 4, 2024

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. This command only builds the runtime.
The AOT compiler is built in an optional step which contains required parameters:
runtime/eng/pipelines/common/templates/runtimes/build-runtime-tests-and-send-to-helix.yml

Lines 42 to 63
in
e610649
 - ${{ if and(eq(parameters.runtimeFlavor, 'mono'), in(parameters.runtimeVariant, 'llvmaot', 'llvmfullaot', 'minifullaot')) }}: 
 - ${{ if eq(parameters.archType, 'arm64') }}: 
 - ${{ if eq(parameters.runtimeVariant, 'minifullaot') }}: 
 - script: ./build.sh
-subset mono
-c ${{ parameters.buildConfig }}
-arch ${{ parameters.archType }}
/p:BuildMonoAotCrossCompiler=true
/p:BuildMonoAotCrossCompilerOnly=true
/p:CrossBuild=true
displayName: "Build Mono Mini AOT cross compiler"
 - ${{ else }}: 
 - script: ./build.sh
-subset mono
-c ${{ parameters.buildConfig }}
-arch ${{ parameters.archType }}
/p:BuildMonoAotCrossCompiler=true
/p:BuildMonoAotCrossCompilerOnly=true
/p:MonoLibClang="/usr/local/lib/libclang.so.16"
/p:MonoAOTEnableLLVM=true
/p:CrossBuild=true
displayName: "Build Mono LLVM AOT cross compiler"
@matouskozakmatouskozakApr 4, 2024•

edited

Loading

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we align on the use of MonoBundleLLVMOptimizer vs MonoAOTEnableLLVM then? Is the MonoBundleLLVMOptimizer=true necessary then? Looks like MonoEnableLLVM=true should be enough 
runtime/src/mono/mono.proj

Line 46
in
c7ada75
 <MonoBundleLLVMOptimizerCondition="'$(MonoEnableLLVM)' == 'true'">true</MonoBundleLLVMOptimizer> 
Comment threadeng/pipelines/extra-platforms/runtime-extra-platforms-other.yml
Outdated

 testGroup: innerloop
 nameSuffix: AllSubsets_Mono_LLVMFullAot_RuntimeIntrinsicsTests
 runtimeVariant: llvmfullaot
 buildArgs: -s mono+libs+clr.hosts+clr.iltools -c Release /p:MonoEnableLLVM=true /p:MonoBundleLLVMOptimizer=true
@matouskozakmatouskozakApr 4, 2024•

edited

Loading

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if it is possible with Helix, but could we re-use the build from AllSubsets_Mono_LLVMFullAot_RuntimeTests_Tests job? The runtime build should be the same, the difference is only which tests get build...
@kotlarmiloskotlarmilosApr 4, 2024•

edited

Loading

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct, perhaps we can create a template. @fanyang-mono Should this be moved to runtime-llvm.yml?
@kotlarmiloskotlarmilosApr 5, 2024

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved to the runtime-llvm.yml and fixed llvmaot job.
@kotlarmilos
kotlarmilos
commented
Apr 4, 2024•

edited

Loading

Copy link


Copy Markdown

MemberAuthor
These jobs have been moved to runtime-llvm.yml. Test run: https://dev.azure.com/dnceng-public/public/_build/results?buildId=630560&view=results
@kotlarmilos
kotlarmilos
commented
Apr 5, 2024

Copy link


Copy Markdown

MemberAuthor
The failures shouldn't be related.
@kotlarmilos
kotlarmilos
merged commit ddae9bd
into
dotnet:mainApr 5, 2024
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@matouskozakmatouskozakmatouskozak approved these changes
@lambdageeklambdageekAwaiting requested review from lambdageek
@SamMonoRTSamMonoRTAwaiting requested review from SamMonoRT
@directhexdirecthexAwaiting requested review from directhex
@steveisoksteveisokAwaiting requested review from steveisok
@fanyang-monofanyang-monoAwaiting requested review from fanyang-mono

+1 more reviewer

@vargazvargazvargaz approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

@kotlarmiloskotlarmilos

Labels

area-Infrastructure-mono

Projects


None yet


Milestone

9.0.0

Development

Successfully merging this pull request may close these issues.

5 participants

@kotlarmilos@vargaz@lambdageek@fanyang-mono@matouskozak

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Remove or un-stick sticky/fixed headers that block content
(function() {
function unstick() {
document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) {
if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {
el.style.position = 'static';
el.style.top = 'auto';
el.style.zIndex = 'auto';
}
});
}
unstick();
var observer = new MutationObserver(unstick);
observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });
})();
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
[mono][infra] Enable runtime tests in fullAOT LLVM mode on linux-x64 by kotlarmilos · Pull Request #92057 · dotnet/runtime · GitHub
Skip to content
[mono][infra] Enable runtime tests in fullAOT LLVM mode on linux-x64 - #92057
Merged
kotlarmilos merged 110 commits into
dotnet:mainfrom 
kotlarmilos:bugfix/linux-x64-fullaot-job
Apr 5, 2024
Merged
[mono][infra] Enable runtime tests in fullAOT LLVM mode on linux-x64#92057
kotlarmilos merged 110 commits into
dotnet:mainfrom 
kotlarmilos:bugfix/linux-x64-fullaot-job
Conversation
@kotlarmilos
@kotlarmiloskotlarmilos
commented
Sep 14, 2023•

edited

Loading

Copy link


Copy Markdown

Member
Description
This PR enables jobs to run runtime tests in fullAOT LLVM mode on linux-x64. The jobs utilize the CBL-Mariner docker image, which contains the clang toolchain instead of binutils. Due to OOM and timeout issues, the tests are split into two jobs: intrinsics and other runtime tests.
Failing tests have been disabled and will be triaged in subsequent PRs. The arm64 support will be introduced in subsequent PRs. A list of disabled tests is included in #90427.
@kotlarmiloskotlarmilos
added

NO-REVIEW
Experimental/testing PR, do NOT review it
area-Infrastructure-mono

labels
Sep 14, 2023
@kotlarmiloskotlarmilos
added this to the 9.0.0 milestone
Sep 14, 2023
@ghost
ghost
commented
Sep 14, 2023

Copy link


Copy Markdown

Tagging subscribers to this area: @directhex

See info in area-owners.md if you want to be subscribed.
Issue Details
Work in progress.
This PR tests the linux-x64 fullAOT CI job.
Author:kotlarmilos
Assignees:-
Labels:
NO-REVIEW, area-Infrastructure-mono
Milestone:-
@kotlarmiloskotlarmilos
added
the 
NO-MERGE
The PR is not ready for merge yet (see discussion for detailed reasons)
label
Sep 14, 2023
@kotlarmilos
kotlarmilos
commented
Sep 14, 2023

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-extra-platforms
@azure-pipelines
azure-pipelinesBot
commented
Sep 14, 2023

Copy link


Copy Markdown


Azure Pipelines successfully started running 1 pipeline(s).

This was referenced Sep 14, 2023

Open


Closed


Closed


Closed


Closed


Closed


Closed


Open


Open

@kotlarmilos
kotlarmilos
commented
Sep 15, 2023

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-extra-platforms
@azure-pipelines
azure-pipelinesBot
commented
Sep 15, 2023

Copy link


Copy Markdown


Azure Pipelines successfully started running 1 pipeline(s).
@build-analysisbuild-analysisBot
mentioned this pull request
Sep 15, 2023

Closed

@kotlarmilos
kotlarmilos
commented
Sep 15, 2023

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-extra-platforms
@azure-pipelines
azure-pipelinesBot
commented
Sep 15, 2023

Copy link


Copy Markdown


Azure Pipelines successfully started running 1 pipeline(s).
@kotlarmilos
kotlarmilos
commented
Sep 15, 2023

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-extra-platforms
@azure-pipelines
azure-pipelinesBot
commented
Sep 15, 2023

Copy link


Copy Markdown


Azure Pipelines successfully started running 1 pipeline(s).
@kotlarmilos
kotlarmilos
commented
Sep 16, 2023

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-extra-platforms
@kotlarmilos
kotlarmilos
commented
Apr 2, 2024

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-extra-platforms
@kotlarmilos
kotlarmilos
commented
Apr 3, 2024

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-extra-platforms
@kotlarmilos
kotlarmilos
commented
Apr 3, 2024

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-extra-platforms
@kotlarmilos
kotlarmilos
commented
Apr 3, 2024

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-extra-platforms
@kotlarmilos
kotlarmilos
commented
Apr 3, 2024

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-extra-platforms
@kotlarmiloskotlarmilos
changed the title
[mono][infra] Enable linux-x64 Mono fullAOT llvm job on extra platforms[mono][infra] Enable runtime tests in fullAOT LLVM mode on linux-x64Apr 3, 2024
@kotlarmilos
kotlarmilos
marked this pull request as ready for review
April 3, 2024 18:17
@kotlarmiloskotlarmilos
removed
the 
blocked
Issue/PR is blocked on something - see comments
label
Apr 3, 2024
@kotlarmilos
kotlarmilos
commented
Apr 4, 2024

Copy link


Copy Markdown

MemberAuthor
This PR is ready for review. Failures shouldn't be related.
Runtime tests: https://dev.azure.com/dnceng-public/public/_build/results?buildId=629337&view=logs&jobId=d29f5ac9-2776-59d5-99ad-be6d4bcd416c&j=d29f5ac9-2776-59d5-99ad-be6d4bcd416c

Runtime intrinsics tests: https://dev.azure.com/dnceng-public/public/_build/results?buildId=629337&view=logs&jobId=d29f5ac9-2776-59d5-99ad-be6d4bcd416c&j=8c8e10f8-cac0-5393-e608-4ea6cc126022
matouskozak
matouskozak
approved these changes
Apr 4, 2024
View reviewed changes
@matouskozakmatouskozak
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! good job
Comment threadeng/pipelines/extra-platforms/runtime-extra-platforms-other.yml
Outdated

 testGroup: innerloop
 nameSuffix: AllSubsets_Mono_LLVMFullAot_RuntimeTests_Tests
 runtimeVariant: llvmfullaot
 buildArgs: -s mono+libs+clr.hosts+clr.iltools -c Release /p:MonoEnableLLVM=true /p:MonoBundleLLVMOptimizer=true
@matouskozakmatouskozakApr 4, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if it is needed in this PR but we should consolidate which flags are used for fullAOT/AOT jobs across our builds. For example, for perf jobs we use /p:MonoAOTEnableLLVM=true /p:MonoEnableLLVM=true /p:BuildMonoAOTCrossCompiler=truehttps://github.com/matouskozak/runtime/blob/12f546416d4aeb23b980a92e407fbf693823236d/eng/pipelines/coreclr/perf-non-wasm-jobs.yml#L39
@kotlarmiloskotlarmilosApr 4, 2024

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. This command only builds the runtime.
The AOT compiler is built in an optional step which contains required parameters:
runtime/eng/pipelines/common/templates/runtimes/build-runtime-tests-and-send-to-helix.yml

Lines 42 to 63
in
e610649
 - ${{ if and(eq(parameters.runtimeFlavor, 'mono'), in(parameters.runtimeVariant, 'llvmaot', 'llvmfullaot', 'minifullaot')) }}: 
 - ${{ if eq(parameters.archType, 'arm64') }}: 
 - ${{ if eq(parameters.runtimeVariant, 'minifullaot') }}: 
 - script: ./build.sh
-subset mono
-c ${{ parameters.buildConfig }}
-arch ${{ parameters.archType }}
/p:BuildMonoAotCrossCompiler=true
/p:BuildMonoAotCrossCompilerOnly=true
/p:CrossBuild=true
displayName: "Build Mono Mini AOT cross compiler"
 - ${{ else }}: 
 - script: ./build.sh
-subset mono
-c ${{ parameters.buildConfig }}
-arch ${{ parameters.archType }}
/p:BuildMonoAotCrossCompiler=true
/p:BuildMonoAotCrossCompilerOnly=true
/p:MonoLibClang="/usr/local/lib/libclang.so.16"
/p:MonoAOTEnableLLVM=true
/p:CrossBuild=true
displayName: "Build Mono LLVM AOT cross compiler"
@matouskozakmatouskozakApr 4, 2024•

edited

Loading

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we align on the use of MonoBundleLLVMOptimizer vs MonoAOTEnableLLVM then? Is the MonoBundleLLVMOptimizer=true necessary then? Looks like MonoEnableLLVM=true should be enough 
runtime/src/mono/mono.proj

Line 46
in
c7ada75
 <MonoBundleLLVMOptimizerCondition="'$(MonoEnableLLVM)' == 'true'">true</MonoBundleLLVMOptimizer> 
Comment threadeng/pipelines/extra-platforms/runtime-extra-platforms-other.yml
Outdated

 testGroup: innerloop
 nameSuffix: AllSubsets_Mono_LLVMFullAot_RuntimeIntrinsicsTests
 runtimeVariant: llvmfullaot
 buildArgs: -s mono+libs+clr.hosts+clr.iltools -c Release /p:MonoEnableLLVM=true /p:MonoBundleLLVMOptimizer=true
@matouskozakmatouskozakApr 4, 2024•

edited

Loading

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if it is possible with Helix, but could we re-use the build from AllSubsets_Mono_LLVMFullAot_RuntimeTests_Tests job? The runtime build should be the same, the difference is only which tests get build...
@kotlarmiloskotlarmilosApr 4, 2024•

edited

Loading

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct, perhaps we can create a template. @fanyang-mono Should this be moved to runtime-llvm.yml?
@kotlarmiloskotlarmilosApr 5, 2024

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved to the runtime-llvm.yml and fixed llvmaot job.
@kotlarmilos
kotlarmilos
commented
Apr 4, 2024•

edited

Loading

Copy link


Copy Markdown

MemberAuthor
These jobs have been moved to runtime-llvm.yml. Test run: https://dev.azure.com/dnceng-public/public/_build/results?buildId=630560&view=results
@kotlarmilos
kotlarmilos
commented
Apr 5, 2024

Copy link


Copy Markdown

MemberAuthor
The failures shouldn't be related.
@kotlarmilos
kotlarmilos
merged commit ddae9bd
into
dotnet:mainApr 5, 2024
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@matouskozakmatouskozakmatouskozak approved these changes
@lambdageeklambdageekAwaiting requested review from lambdageek
@SamMonoRTSamMonoRTAwaiting requested review from SamMonoRT
@directhexdirecthexAwaiting requested review from directhex
@steveisoksteveisokAwaiting requested review from steveisok
@fanyang-monofanyang-monoAwaiting requested review from fanyang-mono

+1 more reviewer

@vargazvargazvargaz approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

@kotlarmiloskotlarmilos

Labels

area-Infrastructure-mono

Projects


None yet


Milestone

9.0.0

Development

Successfully merging this pull request may close these issues.

5 participants

@kotlarmilos@vargaz@lambdageek@fanyang-mono@matouskozak

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Universal Dark Mode - works on any site
(function() {
var enabled = true;
function applyDarkMode() {
if (!enabled) return;
// Create style element if it doesn't exist
var style = document.getElementById('universal-dark-mode-style');
if (!style) {
style = document.createElement('style');
style.id = 'universal-dark-mode-style';
document.head.appendChild(style);
}
// Dark mode CSS - inverts colors but preserves images/video
style.textContent = '
/* Invert everything except media */
html {
filter: invert(1) hue-rotate(180deg) !important;
background: #1a1a2e !important;
}
/* Restore images, videos, iframes, canvas */
img, video, iframe, canvas, svg, picture, [style*="background-image"] {
filter: invert(1) hue-rotate(180deg) !important;
}
/* Preserve specific elements that should not be inverted */
.no-dark-mode, .no-dark-mode *,
[data-theme="light"], [data-theme="light"],
.ace_editor, .ace_editor *,
.CodeMirror, .CodeMirror *,
.monaco-editor, .monaco-editor *,
.markdown-body pre, .markdown-body pre *,
.highlight, .highlight *,
pre code, pre code * {
filter: none !important;
}
/* Fix common UI elements */
.modal, .popup, .dropdown-menu, .tooltip, .popover {
filter: invert(1) hue-rotate(180deg) !important;
background: #2d2d44 !important;
border-color: #444 !important;
}
/* Scrollbars */
::-webkit-scrollbar { background: #1a1a2e !important; }
::-webkit-scrollbar-thumb { background: #444 !important; }
::-webkit-scrollbar-thumb:hover { background: #555 !important; }
/* Selection */
::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }
::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }
';
}
function removeDarkMode() {
var style = document.getElementById('universal-dark-mode-style');
if (style) style.remove();
}
// Toggle with Alt+Shift+D
document.addEventListener('keydown', function(e) {
if (e.altKey && e.shiftKey && e.key === 'D') {
e.preventDefault();
enabled = !enabled;
if (enabled) {
applyDarkMode();
console.log('[Universal Dark Mode] Enabled');
} else {
removeDarkMode();
console.log('[Universal Dark Mode] Disabled');
}
}
});
// Apply on load
applyDarkMode();
// Re-apply on dynamic content
var observer = new MutationObserver(function(mutations) {
if (enabled && !document.getElementById('universal-dark-mode-style')) {
applyDarkMode();
}
});
observer.observe(document.head, { childList: true });
console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');
})();
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();
[mono][infra] Enable runtime tests in fullAOT LLVM mode on linux-x64 by kotlarmilos · Pull Request #92057 · dotnet/runtime · GitHub
Skip to content
[mono][infra] Enable runtime tests in fullAOT LLVM mode on linux-x64 - #92057
Merged
kotlarmilos merged 110 commits into
dotnet:mainfrom 
kotlarmilos:bugfix/linux-x64-fullaot-job
Apr 5, 2024
Merged
[mono][infra] Enable runtime tests in fullAOT LLVM mode on linux-x64#92057
kotlarmilos merged 110 commits into
dotnet:mainfrom 
kotlarmilos:bugfix/linux-x64-fullaot-job
Conversation
@kotlarmilos
@kotlarmiloskotlarmilos
commented
Sep 14, 2023•

edited

Loading

Copy link


Copy Markdown

Member
Description
This PR enables jobs to run runtime tests in fullAOT LLVM mode on linux-x64. The jobs utilize the CBL-Mariner docker image, which contains the clang toolchain instead of binutils. Due to OOM and timeout issues, the tests are split into two jobs: intrinsics and other runtime tests.
Failing tests have been disabled and will be triaged in subsequent PRs. The arm64 support will be introduced in subsequent PRs. A list of disabled tests is included in #90427.
@kotlarmiloskotlarmilos
added

NO-REVIEW
Experimental/testing PR, do NOT review it
area-Infrastructure-mono

labels
Sep 14, 2023
@kotlarmiloskotlarmilos
added this to the 9.0.0 milestone
Sep 14, 2023
@ghost
ghost
commented
Sep 14, 2023

Copy link


Copy Markdown

Tagging subscribers to this area: @directhex

See info in area-owners.md if you want to be subscribed.
Issue Details
Work in progress.
This PR tests the linux-x64 fullAOT CI job.
Author:kotlarmilos
Assignees:-
Labels:
NO-REVIEW, area-Infrastructure-mono
Milestone:-
@kotlarmiloskotlarmilos
added
the 
NO-MERGE
The PR is not ready for merge yet (see discussion for detailed reasons)
label
Sep 14, 2023
@kotlarmilos
kotlarmilos
commented
Sep 14, 2023

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-extra-platforms
@azure-pipelines
azure-pipelinesBot
commented
Sep 14, 2023

Copy link


Copy Markdown


Azure Pipelines successfully started running 1 pipeline(s).

This was referenced Sep 14, 2023

Open


Closed


Closed


Closed


Closed


Closed


Closed


Open


Open

@kotlarmilos
kotlarmilos
commented
Sep 15, 2023

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-extra-platforms
@azure-pipelines
azure-pipelinesBot
commented
Sep 15, 2023

Copy link


Copy Markdown


Azure Pipelines successfully started running 1 pipeline(s).
@build-analysisbuild-analysisBot
mentioned this pull request
Sep 15, 2023

Closed

@kotlarmilos
kotlarmilos
commented
Sep 15, 2023

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-extra-platforms
@azure-pipelines
azure-pipelinesBot
commented
Sep 15, 2023

Copy link


Copy Markdown


Azure Pipelines successfully started running 1 pipeline(s).
@kotlarmilos
kotlarmilos
commented
Sep 15, 2023

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-extra-platforms
@azure-pipelines
azure-pipelinesBot
commented
Sep 15, 2023

Copy link


Copy Markdown


Azure Pipelines successfully started running 1 pipeline(s).
@kotlarmilos
kotlarmilos
commented
Sep 16, 2023

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-extra-platforms
@kotlarmilos
kotlarmilos
commented
Apr 2, 2024

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-extra-platforms
@kotlarmilos
kotlarmilos
commented
Apr 3, 2024

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-extra-platforms
@kotlarmilos
kotlarmilos
commented
Apr 3, 2024

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-extra-platforms
@kotlarmilos
kotlarmilos
commented
Apr 3, 2024

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-extra-platforms
@kotlarmilos
kotlarmilos
commented
Apr 3, 2024

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-extra-platforms
@kotlarmiloskotlarmilos
changed the title
[mono][infra] Enable linux-x64 Mono fullAOT llvm job on extra platforms[mono][infra] Enable runtime tests in fullAOT LLVM mode on linux-x64Apr 3, 2024
@kotlarmilos
kotlarmilos
marked this pull request as ready for review
April 3, 2024 18:17
@kotlarmiloskotlarmilos
removed
the 
blocked
Issue/PR is blocked on something - see comments
label
Apr 3, 2024
@kotlarmilos
kotlarmilos
commented
Apr 4, 2024

Copy link


Copy Markdown

MemberAuthor
This PR is ready for review. Failures shouldn't be related.
Runtime tests: https://dev.azure.com/dnceng-public/public/_build/results?buildId=629337&view=logs&jobId=d29f5ac9-2776-59d5-99ad-be6d4bcd416c&j=d29f5ac9-2776-59d5-99ad-be6d4bcd416c

Runtime intrinsics tests: https://dev.azure.com/dnceng-public/public/_build/results?buildId=629337&view=logs&jobId=d29f5ac9-2776-59d5-99ad-be6d4bcd416c&j=8c8e10f8-cac0-5393-e608-4ea6cc126022
matouskozak
matouskozak
approved these changes
Apr 4, 2024
View reviewed changes
@matouskozakmatouskozak
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! good job
Comment threadeng/pipelines/extra-platforms/runtime-extra-platforms-other.yml
Outdated

 testGroup: innerloop
 nameSuffix: AllSubsets_Mono_LLVMFullAot_RuntimeTests_Tests
 runtimeVariant: llvmfullaot
 buildArgs: -s mono+libs+clr.hosts+clr.iltools -c Release /p:MonoEnableLLVM=true /p:MonoBundleLLVMOptimizer=true
@matouskozakmatouskozakApr 4, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if it is needed in this PR but we should consolidate which flags are used for fullAOT/AOT jobs across our builds. For example, for perf jobs we use /p:MonoAOTEnableLLVM=true /p:MonoEnableLLVM=true /p:BuildMonoAOTCrossCompiler=truehttps://github.com/matouskozak/runtime/blob/12f546416d4aeb23b980a92e407fbf693823236d/eng/pipelines/coreclr/perf-non-wasm-jobs.yml#L39
@kotlarmiloskotlarmilosApr 4, 2024

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. This command only builds the runtime.
The AOT compiler is built in an optional step which contains required parameters:
runtime/eng/pipelines/common/templates/runtimes/build-runtime-tests-and-send-to-helix.yml

Lines 42 to 63
in
e610649
 - ${{ if and(eq(parameters.runtimeFlavor, 'mono'), in(parameters.runtimeVariant, 'llvmaot', 'llvmfullaot', 'minifullaot')) }}: 
 - ${{ if eq(parameters.archType, 'arm64') }}: 
 - ${{ if eq(parameters.runtimeVariant, 'minifullaot') }}: 
 - script: ./build.sh
-subset mono
-c ${{ parameters.buildConfig }}
-arch ${{ parameters.archType }}
/p:BuildMonoAotCrossCompiler=true
/p:BuildMonoAotCrossCompilerOnly=true
/p:CrossBuild=true
displayName: "Build Mono Mini AOT cross compiler"
 - ${{ else }}: 
 - script: ./build.sh
-subset mono
-c ${{ parameters.buildConfig }}
-arch ${{ parameters.archType }}
/p:BuildMonoAotCrossCompiler=true
/p:BuildMonoAotCrossCompilerOnly=true
/p:MonoLibClang="/usr/local/lib/libclang.so.16"
/p:MonoAOTEnableLLVM=true
/p:CrossBuild=true
displayName: "Build Mono LLVM AOT cross compiler"
@matouskozakmatouskozakApr 4, 2024•

edited

Loading

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we align on the use of MonoBundleLLVMOptimizer vs MonoAOTEnableLLVM then? Is the MonoBundleLLVMOptimizer=true necessary then? Looks like MonoEnableLLVM=true should be enough 
runtime/src/mono/mono.proj

Line 46
in
c7ada75
 <MonoBundleLLVMOptimizerCondition="'$(MonoEnableLLVM)' == 'true'">true</MonoBundleLLVMOptimizer> 
Comment threadeng/pipelines/extra-platforms/runtime-extra-platforms-other.yml
Outdated

 testGroup: innerloop
 nameSuffix: AllSubsets_Mono_LLVMFullAot_RuntimeIntrinsicsTests
 runtimeVariant: llvmfullaot
 buildArgs: -s mono+libs+clr.hosts+clr.iltools -c Release /p:MonoEnableLLVM=true /p:MonoBundleLLVMOptimizer=true
@matouskozakmatouskozakApr 4, 2024•

edited

Loading

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if it is possible with Helix, but could we re-use the build from AllSubsets_Mono_LLVMFullAot_RuntimeTests_Tests job? The runtime build should be the same, the difference is only which tests get build...
@kotlarmiloskotlarmilosApr 4, 2024•

edited

Loading

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct, perhaps we can create a template. @fanyang-mono Should this be moved to runtime-llvm.yml?
@kotlarmiloskotlarmilosApr 5, 2024

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved to the runtime-llvm.yml and fixed llvmaot job.
@kotlarmilos
kotlarmilos
commented
Apr 4, 2024•

edited

Loading

Copy link


Copy Markdown

MemberAuthor
These jobs have been moved to runtime-llvm.yml. Test run: https://dev.azure.com/dnceng-public/public/_build/results?buildId=630560&view=results
@kotlarmilos
kotlarmilos
commented
Apr 5, 2024

Copy link


Copy Markdown

MemberAuthor
The failures shouldn't be related.
@kotlarmilos
kotlarmilos
merged commit ddae9bd
into
dotnet:mainApr 5, 2024
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@matouskozakmatouskozakmatouskozak approved these changes
@lambdageeklambdageekAwaiting requested review from lambdageek
@SamMonoRTSamMonoRTAwaiting requested review from SamMonoRT
@directhexdirecthexAwaiting requested review from directhex
@steveisoksteveisokAwaiting requested review from steveisok
@fanyang-monofanyang-monoAwaiting requested review from fanyang-mono

+1 more reviewer

@vargazvargazvargaz approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

@kotlarmiloskotlarmilos

Labels

area-Infrastructure-mono

Projects


None yet


Milestone

9.0.0

Development

Successfully merging this pull request may close these issues.

5 participants

@kotlarmilos@vargaz@lambdageek@fanyang-mono@matouskozak