[ci-scan] Build break: crossgen2 comparison NativeOrReadyToRunImage hash mismatch on linux arm #129013

Description

@github-actions

Build Information

Build: https://dev.azure.com/dnceng-public/public/_build/results?buildId=1447200
Build error leg or test failing: Test crossgen2-comparison linux arm Release to arm Linux - WorkItem

Error Details

crossgen2_comparison.py detects non-deterministic R2R image output when crossgen2 compiles the same assemblies twice targeting linux-arm. 14 of 236 compared assemblies produce different SHA-256 hashes between the baseline and re-compilation pass.

File hash sum mismatch for "Microsoft.Extensions.Logging.Console" assembly for files of type "NativeOrReadyToRunImage":
- "/root/helix/work/workitem/u/prebuiltWork/log" has "14ec4b8afd76ac66aa4432a5b6ba1247fc6bbc7b1a434d35a7f0ba95a13e06ce"
- "/root/helix/work/workitem/uploads" has "57c98b4b4dba1f7c24354e679283d84d7967819eae931debe694e24abcccc042"
File hash sum mismatch for "System.Collections.Immutable" assembly for files of type "NativeOrReadyToRunImage":
...
Number of omitted results: 0
Number of mismatched results: 14
Total number of files compared: 236

Affected pipelines (all comparison legs for linux-arm target):

  • runtime-coreclr crossgen2 outerloop (def 134)
  • runtime-coreclr crossgen2-composite (def 136) — likely same root cause
  • runtime-coreclr r2r-extra (def 114) — likely same root cause

Related (closed, .NET 10): #111972

Error Message

{
"ErrorMessage": "File hash sum mismatch for",
"ErrorPattern": "",
"BuildRetry": false,
"ExcludeConsoleLog": false
}

First build it occurred


Filed by ci-failure-scan, which scans dnceng-public outer-loop pipelines on main and converts stable failures into KBEs.

Note

🔒 Integrity filter blocked 42 items

The following items were blocked because they don't meet the GitHub integrity level.

  • #5158search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
  • #7096search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
  • #5984search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
  • #5101search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
  • #5318search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
  • #44158search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
  • #5637search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
  • #7231search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
  • #5130search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
  • #128959search_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
  • #67082search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
  • #114222search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
  • #128365search_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
  • #48070search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
  • #88027search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
  • #96968search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
  • ... and 26 more items

To allow these resources, lower min-integrity in your GitHub frontmatter:

tools:
github:
min-integrity: approved # merged | approved | unapproved | none

Generated by CI Outer-Loop Failure Scanner · ● 93.5M ·

Report

Summary

24-Hour Hit Count7-Day Hit Count1-Month Count
000

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions

      , 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
       blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
      }
      } catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
      })();
      (function(){
      try {
      var __m = "github.com";
      var __re = new RegExp('^' + "github\\.com" + '
      
      Skip to content

      [ci-scan] Build break: crossgen2 comparison NativeOrReadyToRunImage hash mismatch on linux arm #129013

      Description

      @github-actions

      Build Information

      Build: https://dev.azure.com/dnceng-public/public/_build/results?buildId=1447200
      Build error leg or test failing: Test crossgen2-comparison linux arm Release to arm Linux - WorkItem

      Error Details

      crossgen2_comparison.py detects non-deterministic R2R image output when crossgen2 compiles the same assemblies twice targeting linux-arm. 14 of 236 compared assemblies produce different SHA-256 hashes between the baseline and re-compilation pass.

      File hash sum mismatch for "Microsoft.Extensions.Logging.Console" assembly for files of type "NativeOrReadyToRunImage":
      - "/root/helix/work/workitem/u/prebuiltWork/log" has "14ec4b8afd76ac66aa4432a5b6ba1247fc6bbc7b1a434d35a7f0ba95a13e06ce"
      - "/root/helix/work/workitem/uploads" has "57c98b4b4dba1f7c24354e679283d84d7967819eae931debe694e24abcccc042"
      File hash sum mismatch for "System.Collections.Immutable" assembly for files of type "NativeOrReadyToRunImage":
      ...
      Number of omitted results: 0
      Number of mismatched results: 14
      Total number of files compared: 236
      

      Affected pipelines (all comparison legs for linux-arm target):

      • runtime-coreclr crossgen2 outerloop (def 134)
      • runtime-coreclr crossgen2-composite (def 136) — likely same root cause
      • runtime-coreclr r2r-extra (def 114) — likely same root cause

      Related (closed, .NET 10): #111972

      Error Message

      {
      "ErrorMessage": "File hash sum mismatch for",
      "ErrorPattern": "",
      "BuildRetry": false,
      "ExcludeConsoleLog": false
      }

      First build it occurred


      Filed by ci-failure-scan, which scans dnceng-public outer-loop pipelines on main and converts stable failures into KBEs.

      Note

      🔒 Integrity filter blocked 42 items

      The following items were blocked because they don't meet the GitHub integrity level.

      • #5158search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
      • #7096search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
      • #5984search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
      • #5101search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
      • #5318search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
      • #44158search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
      • #5637search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
      • #7231search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
      • #5130search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
      • #128959search_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
      • #67082search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
      • #114222search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
      • #128365search_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
      • #48070search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
      • #88027search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
      • #96968search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
      • ... and 26 more items

      To allow these resources, lower min-integrity in your GitHub frontmatter:

      tools:
      github:
      min-integrity: approved # merged | approved | unapproved | none

      Generated by CI Outer-Loop Failure Scanner · ● 93.5M ·

      Report

      Summary

      24-Hour Hit Count7-Day Hit Count1-Month Count
      000

      Metadata

      Metadata

      Assignees

      No one assigned

        Labels

        Type

        No type

        Projects

        No projects

          Milestone

          No milestone

          Relationships

          None yet

          Development

          No branches or pull requests

          Issue actions

          , 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
          Skip to content

          [ci-scan] Build break: crossgen2 comparison NativeOrReadyToRunImage hash mismatch on linux arm #129013

          Description

          @github-actions

          Build Information

          Build: https://dev.azure.com/dnceng-public/public/_build/results?buildId=1447200
          Build error leg or test failing: Test crossgen2-comparison linux arm Release to arm Linux - WorkItem

          Error Details

          crossgen2_comparison.py detects non-deterministic R2R image output when crossgen2 compiles the same assemblies twice targeting linux-arm. 14 of 236 compared assemblies produce different SHA-256 hashes between the baseline and re-compilation pass.

          File hash sum mismatch for "Microsoft.Extensions.Logging.Console" assembly for files of type "NativeOrReadyToRunImage":
          - "/root/helix/work/workitem/u/prebuiltWork/log" has "14ec4b8afd76ac66aa4432a5b6ba1247fc6bbc7b1a434d35a7f0ba95a13e06ce"
          - "/root/helix/work/workitem/uploads" has "57c98b4b4dba1f7c24354e679283d84d7967819eae931debe694e24abcccc042"
          File hash sum mismatch for "System.Collections.Immutable" assembly for files of type "NativeOrReadyToRunImage":
          ...
          Number of omitted results: 0
          Number of mismatched results: 14
          Total number of files compared: 236
          

          Affected pipelines (all comparison legs for linux-arm target):

          • runtime-coreclr crossgen2 outerloop (def 134)
          • runtime-coreclr crossgen2-composite (def 136) — likely same root cause
          • runtime-coreclr r2r-extra (def 114) — likely same root cause

          Related (closed, .NET 10): #111972

          Error Message

          {
          "ErrorMessage": "File hash sum mismatch for",
          "ErrorPattern": "",
          "BuildRetry": false,
          "ExcludeConsoleLog": false
          }

          First build it occurred


          Filed by ci-failure-scan, which scans dnceng-public outer-loop pipelines on main and converts stable failures into KBEs.

          Note

          🔒 Integrity filter blocked 42 items

          The following items were blocked because they don't meet the GitHub integrity level.

          • #5158search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
          • #7096search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
          • #5984search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
          • #5101search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
          • #5318search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
          • #44158search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
          • #5637search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
          • #7231search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
          • #5130search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
          • #128959search_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
          • #67082search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
          • #114222search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
          • #128365search_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
          • #48070search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
          • #88027search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
          • #96968search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
          • ... and 26 more items

          To allow these resources, lower min-integrity in your GitHub frontmatter:

          tools:
          github:
          min-integrity: approved # merged | approved | unapproved | none

          Generated by CI Outer-Loop Failure Scanner · ● 93.5M ·

          Report

          Summary

          24-Hour Hit Count7-Day Hit Count1-Month Count
          000

          Metadata

          Metadata

          Assignees

          No one assigned

            Labels

            Type

            No type

            Projects

            No projects

              Milestone

              No milestone

              Relationships

              None yet

              Development

              No branches or pull requests

              Issue actions

              , 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
              Skip to content

              [ci-scan] Build break: crossgen2 comparison NativeOrReadyToRunImage hash mismatch on linux arm #129013

              Description

              @github-actions

              Build Information

              Build: https://dev.azure.com/dnceng-public/public/_build/results?buildId=1447200
              Build error leg or test failing: Test crossgen2-comparison linux arm Release to arm Linux - WorkItem

              Error Details

              crossgen2_comparison.py detects non-deterministic R2R image output when crossgen2 compiles the same assemblies twice targeting linux-arm. 14 of 236 compared assemblies produce different SHA-256 hashes between the baseline and re-compilation pass.

              File hash sum mismatch for "Microsoft.Extensions.Logging.Console" assembly for files of type "NativeOrReadyToRunImage":
              - "/root/helix/work/workitem/u/prebuiltWork/log" has "14ec4b8afd76ac66aa4432a5b6ba1247fc6bbc7b1a434d35a7f0ba95a13e06ce"
              - "/root/helix/work/workitem/uploads" has "57c98b4b4dba1f7c24354e679283d84d7967819eae931debe694e24abcccc042"
              File hash sum mismatch for "System.Collections.Immutable" assembly for files of type "NativeOrReadyToRunImage":
              ...
              Number of omitted results: 0
              Number of mismatched results: 14
              Total number of files compared: 236
              

              Affected pipelines (all comparison legs for linux-arm target):

              • runtime-coreclr crossgen2 outerloop (def 134)
              • runtime-coreclr crossgen2-composite (def 136) — likely same root cause
              • runtime-coreclr r2r-extra (def 114) — likely same root cause

              Related (closed, .NET 10): #111972

              Error Message

              {
              "ErrorMessage": "File hash sum mismatch for",
              "ErrorPattern": "",
              "BuildRetry": false,
              "ExcludeConsoleLog": false
              }

              First build it occurred


              Filed by ci-failure-scan, which scans dnceng-public outer-loop pipelines on main and converts stable failures into KBEs.

              Note

              🔒 Integrity filter blocked 42 items

              The following items were blocked because they don't meet the GitHub integrity level.

              • #5158search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
              • #7096search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
              • #5984search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
              • #5101search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
              • #5318search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
              • #44158search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
              • #5637search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
              • #7231search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
              • #5130search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
              • #128959search_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
              • #67082search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
              • #114222search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
              • #128365search_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
              • #48070search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
              • #88027search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
              • #96968search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
              • ... and 26 more items

              To allow these resources, lower min-integrity in your GitHub frontmatter:

              tools:
              github:
              min-integrity: approved # merged | approved | unapproved | none

              Generated by CI Outer-Loop Failure Scanner · ● 93.5M ·

              Report

              Summary

              24-Hour Hit Count7-Day Hit Count1-Month Count
              000

              Metadata

              Metadata

              Assignees

              No one assigned

                Labels

                Type

                No type

                Projects

                No projects

                  Milestone

                  No milestone

                  Relationships

                  None yet

                  Development

                  No branches or pull requests

                  Issue actions

                  , 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
                  Skip to content

                  [ci-scan] Build break: crossgen2 comparison NativeOrReadyToRunImage hash mismatch on linux arm #129013

                  Description

                  @github-actions

                  Build Information

                  Build: https://dev.azure.com/dnceng-public/public/_build/results?buildId=1447200
                  Build error leg or test failing: Test crossgen2-comparison linux arm Release to arm Linux - WorkItem

                  Error Details

                  crossgen2_comparison.py detects non-deterministic R2R image output when crossgen2 compiles the same assemblies twice targeting linux-arm. 14 of 236 compared assemblies produce different SHA-256 hashes between the baseline and re-compilation pass.

                  File hash sum mismatch for "Microsoft.Extensions.Logging.Console" assembly for files of type "NativeOrReadyToRunImage":
                  - "/root/helix/work/workitem/u/prebuiltWork/log" has "14ec4b8afd76ac66aa4432a5b6ba1247fc6bbc7b1a434d35a7f0ba95a13e06ce"
                  - "/root/helix/work/workitem/uploads" has "57c98b4b4dba1f7c24354e679283d84d7967819eae931debe694e24abcccc042"
                  File hash sum mismatch for "System.Collections.Immutable" assembly for files of type "NativeOrReadyToRunImage":
                  ...
                  Number of omitted results: 0
                  Number of mismatched results: 14
                  Total number of files compared: 236
                  

                  Affected pipelines (all comparison legs for linux-arm target):

                  • runtime-coreclr crossgen2 outerloop (def 134)
                  • runtime-coreclr crossgen2-composite (def 136) — likely same root cause
                  • runtime-coreclr r2r-extra (def 114) — likely same root cause

                  Related (closed, .NET 10): #111972

                  Error Message

                  {
                  "ErrorMessage": "File hash sum mismatch for",
                  "ErrorPattern": "",
                  "BuildRetry": false,
                  "ExcludeConsoleLog": false
                  }

                  First build it occurred


                  Filed by ci-failure-scan, which scans dnceng-public outer-loop pipelines on main and converts stable failures into KBEs.

                  Note

                  🔒 Integrity filter blocked 42 items

                  The following items were blocked because they don't meet the GitHub integrity level.

                  • #5158search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
                  • #7096search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
                  • #5984search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
                  • #5101search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
                  • #5318search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
                  • #44158search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
                  • #5637search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
                  • #7231search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
                  • #5130search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
                  • #128959search_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
                  • #67082search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
                  • #114222search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
                  • #128365search_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
                  • #48070search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
                  • #88027search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
                  • #96968search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
                  • ... and 26 more items

                  To allow these resources, lower min-integrity in your GitHub frontmatter:

                  tools:
                  github:
                  min-integrity: approved # merged | approved | unapproved | none

                  Generated by CI Outer-Loop Failure Scanner · ● 93.5M ·

                  Report

                  Summary

                  24-Hour Hit Count7-Day Hit Count1-Month Count
                  000

                  Metadata

                  Metadata

                  Assignees

                  No one assigned

                    Labels

                    Type

                    No type

                    Projects

                    No projects

                      Milestone

                      No milestone

                      Relationships

                      None yet

                      Development

                      No branches or pull requests

                      Issue actions

                      , 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
                      Skip to content

                      [ci-scan] Build break: crossgen2 comparison NativeOrReadyToRunImage hash mismatch on linux arm #129013

                      Description

                      @github-actions

                      Build Information

                      Build: https://dev.azure.com/dnceng-public/public/_build/results?buildId=1447200
                      Build error leg or test failing: Test crossgen2-comparison linux arm Release to arm Linux - WorkItem

                      Error Details

                      crossgen2_comparison.py detects non-deterministic R2R image output when crossgen2 compiles the same assemblies twice targeting linux-arm. 14 of 236 compared assemblies produce different SHA-256 hashes between the baseline and re-compilation pass.

                      File hash sum mismatch for "Microsoft.Extensions.Logging.Console" assembly for files of type "NativeOrReadyToRunImage":
                      - "/root/helix/work/workitem/u/prebuiltWork/log" has "14ec4b8afd76ac66aa4432a5b6ba1247fc6bbc7b1a434d35a7f0ba95a13e06ce"
                      - "/root/helix/work/workitem/uploads" has "57c98b4b4dba1f7c24354e679283d84d7967819eae931debe694e24abcccc042"
                      File hash sum mismatch for "System.Collections.Immutable" assembly for files of type "NativeOrReadyToRunImage":
                      ...
                      Number of omitted results: 0
                      Number of mismatched results: 14
                      Total number of files compared: 236
                      

                      Affected pipelines (all comparison legs for linux-arm target):

                      • runtime-coreclr crossgen2 outerloop (def 134)
                      • runtime-coreclr crossgen2-composite (def 136) — likely same root cause
                      • runtime-coreclr r2r-extra (def 114) — likely same root cause

                      Related (closed, .NET 10): #111972

                      Error Message

                      {
                      "ErrorMessage": "File hash sum mismatch for",
                      "ErrorPattern": "",
                      "BuildRetry": false,
                      "ExcludeConsoleLog": false
                      }

                      First build it occurred


                      Filed by ci-failure-scan, which scans dnceng-public outer-loop pipelines on main and converts stable failures into KBEs.

                      Note

                      🔒 Integrity filter blocked 42 items

                      The following items were blocked because they don't meet the GitHub integrity level.

                      • #5158search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
                      • #7096search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
                      • #5984search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
                      • #5101search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
                      • #5318search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
                      • #44158search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
                      • #5637search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
                      • #7231search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
                      • #5130search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
                      • #128959search_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
                      • #67082search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
                      • #114222search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
                      • #128365search_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
                      • #48070search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
                      • #88027search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
                      • #96968search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
                      • ... and 26 more items

                      To allow these resources, lower min-integrity in your GitHub frontmatter:

                      tools:
                      github:
                      min-integrity: approved # merged | approved | unapproved | none

                      Generated by CI Outer-Loop Failure Scanner · ● 93.5M ·

                      Report

                      Summary

                      24-Hour Hit Count7-Day Hit Count1-Month Count
                      000

                      Metadata

                      Metadata

                      Assignees

                      No one assigned

                        Labels

                        Type

                        No type

                        Projects

                        No projects

                          Milestone

                          No milestone

                          Relationships

                          None yet

                          Development

                          No branches or pull requests

                          Issue actions

                          , 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
                          Skip to content

                          [ci-scan] Build break: crossgen2 comparison NativeOrReadyToRunImage hash mismatch on linux arm #129013

                          Description

                          @github-actions

                          Build Information

                          Build: https://dev.azure.com/dnceng-public/public/_build/results?buildId=1447200
                          Build error leg or test failing: Test crossgen2-comparison linux arm Release to arm Linux - WorkItem

                          Error Details

                          crossgen2_comparison.py detects non-deterministic R2R image output when crossgen2 compiles the same assemblies twice targeting linux-arm. 14 of 236 compared assemblies produce different SHA-256 hashes between the baseline and re-compilation pass.

                          File hash sum mismatch for "Microsoft.Extensions.Logging.Console" assembly for files of type "NativeOrReadyToRunImage":
                          - "/root/helix/work/workitem/u/prebuiltWork/log" has "14ec4b8afd76ac66aa4432a5b6ba1247fc6bbc7b1a434d35a7f0ba95a13e06ce"
                          - "/root/helix/work/workitem/uploads" has "57c98b4b4dba1f7c24354e679283d84d7967819eae931debe694e24abcccc042"
                          File hash sum mismatch for "System.Collections.Immutable" assembly for files of type "NativeOrReadyToRunImage":
                          ...
                          Number of omitted results: 0
                          Number of mismatched results: 14
                          Total number of files compared: 236
                          

                          Affected pipelines (all comparison legs for linux-arm target):

                          • runtime-coreclr crossgen2 outerloop (def 134)
                          • runtime-coreclr crossgen2-composite (def 136) — likely same root cause
                          • runtime-coreclr r2r-extra (def 114) — likely same root cause

                          Related (closed, .NET 10): #111972

                          Error Message

                          {
                          "ErrorMessage": "File hash sum mismatch for",
                          "ErrorPattern": "",
                          "BuildRetry": false,
                          "ExcludeConsoleLog": false
                          }

                          First build it occurred


                          Filed by ci-failure-scan, which scans dnceng-public outer-loop pipelines on main and converts stable failures into KBEs.

                          Note

                          🔒 Integrity filter blocked 42 items

                          The following items were blocked because they don't meet the GitHub integrity level.

                          • #5158search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
                          • #7096search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
                          • #5984search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
                          • #5101search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
                          • #5318search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
                          • #44158search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
                          • #5637search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
                          • #7231search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
                          • #5130search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
                          • #128959search_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
                          • #67082search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
                          • #114222search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
                          • #128365search_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
                          • #48070search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
                          • #88027search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
                          • #96968search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
                          • ... and 26 more items

                          To allow these resources, lower min-integrity in your GitHub frontmatter:

                          tools:
                          github:
                          min-integrity: approved # merged | approved | unapproved | none

                          Generated by CI Outer-Loop Failure Scanner · ● 93.5M ·

                          Report

                          Summary

                          24-Hour Hit Count7-Day Hit Count1-Month Count
                          000

                          Metadata

                          Metadata

                          Assignees

                          No one assigned

                            Labels

                            Type

                            No type

                            Projects

                            No projects

                              Milestone

                              No milestone

                              Relationships

                              None yet

                              Development

                              No branches or pull requests

                              Issue actions

                              , 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
                              Skip to content

                              [ci-scan] Build break: crossgen2 comparison NativeOrReadyToRunImage hash mismatch on linux arm #129013

                              Description

                              @github-actions

                              Build Information

                              Build: https://dev.azure.com/dnceng-public/public/_build/results?buildId=1447200
                              Build error leg or test failing: Test crossgen2-comparison linux arm Release to arm Linux - WorkItem

                              Error Details

                              crossgen2_comparison.py detects non-deterministic R2R image output when crossgen2 compiles the same assemblies twice targeting linux-arm. 14 of 236 compared assemblies produce different SHA-256 hashes between the baseline and re-compilation pass.

                              File hash sum mismatch for "Microsoft.Extensions.Logging.Console" assembly for files of type "NativeOrReadyToRunImage":
                              - "/root/helix/work/workitem/u/prebuiltWork/log" has "14ec4b8afd76ac66aa4432a5b6ba1247fc6bbc7b1a434d35a7f0ba95a13e06ce"
                              - "/root/helix/work/workitem/uploads" has "57c98b4b4dba1f7c24354e679283d84d7967819eae931debe694e24abcccc042"
                              File hash sum mismatch for "System.Collections.Immutable" assembly for files of type "NativeOrReadyToRunImage":
                              ...
                              Number of omitted results: 0
                              Number of mismatched results: 14
                              Total number of files compared: 236
                              

                              Affected pipelines (all comparison legs for linux-arm target):

                              • runtime-coreclr crossgen2 outerloop (def 134)
                              • runtime-coreclr crossgen2-composite (def 136) — likely same root cause
                              • runtime-coreclr r2r-extra (def 114) — likely same root cause

                              Related (closed, .NET 10): #111972

                              Error Message

                              {
                              "ErrorMessage": "File hash sum mismatch for",
                              "ErrorPattern": "",
                              "BuildRetry": false,
                              "ExcludeConsoleLog": false
                              }

                              First build it occurred


                              Filed by ci-failure-scan, which scans dnceng-public outer-loop pipelines on main and converts stable failures into KBEs.

                              Note

                              🔒 Integrity filter blocked 42 items

                              The following items were blocked because they don't meet the GitHub integrity level.

                              • #5158search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
                              • #7096search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
                              • #5984search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
                              • #5101search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
                              • #5318search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
                              • #44158search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
                              • #5637search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
                              • #7231search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
                              • #5130search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
                              • #128959search_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
                              • #67082search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
                              • #114222search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
                              • #128365search_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
                              • #48070search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
                              • #88027search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
                              • #96968search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
                              • ... and 26 more items

                              To allow these resources, lower min-integrity in your GitHub frontmatter:

                              tools:
                              github:
                              min-integrity: approved # merged | approved | unapproved | none

                              Generated by CI Outer-Loop Failure Scanner · ● 93.5M ·

                              Report

                              Summary

                              24-Hour Hit Count7-Day Hit Count1-Month Count
                              000

                              Metadata

                              Metadata

                              Assignees

                              No one assigned

                                Labels

                                Type

                                No type

                                Projects

                                No projects

                                  Milestone

                                  No milestone

                                  Relationships

                                  None yet

                                  Development

                                  No branches or pull requests

                                  Issue actions