Set MSBUILDDEBUGPATH in build scripts for reliable crash diagnostics - #126806

Closed
danmoseley wants to merge 3 commits into
dotnet:mainfrom
danmoseley:fix-msbuilddebugpath
Closed

Set MSBUILDDEBUGPATH in build scripts for reliable crash diagnostics#126806
danmoseley wants to merge 3 commits into
dotnet:mainfrom
danmoseley:fix-msbuilddebugpath

Conversation

@danmoseley

@danmoseleydanmoseley commented Apr 12, 2026

Copy link
Copy Markdown
Contributor

Set MSBUILDDEBUGPATH directly in eng/build.sh for reliable crash diagnostics

PR #126012 added MSBUILDDEBUGPATH as an AzDO pipeline variable in global-build-job.yml so that MSBuild crash diagnostics (MSBuild_*.failure.txt) would be written to artifacts/log/ instead of the system temp directory. However, recent builds that hit the MSB4166 crash (e.g., build 1370423) still show MSBuild writing to the default temp path, indicating the pipeline variable is not reaching the MSBuild process.

Other build scripts (src/coreclr/build-runtime.sh, src/tests/build.sh) already set and export MSBUILDDEBUGPATH independently, which is why "Build Tests" steps work. The "Build product" step calls eng/build.sh, which did not set it.

This PR sets MSBUILDDEBUGPATH directly in eng/build.sh and eng/build.ps1 before any common/build.sh/common/build.ps1 invocations, matching the pattern used by the other scripts. This also:

  • Creates the directory with mkdir -p before MSBuild starts (MSBuild's EnsureDirectoryExists has no try/catch in its static constructor, so a failed directory creation would silently fall back to the temp dir)
  • Echoes the path for diagnostic visibility in build logs
  • Covers both the bootstrap and product build invocations

The existing pipeline variable in global-build-job.yml is left in place as defense-in-depth.

Relates to getting data to fix#92290

…rash diagnostics
The MSBUILDDEBUGPATH pipeline variable added in PR dotnet#126012 is not reaching
the MSBuild process during Build product steps. Set the env var directly in
the build scripts, matching the existing pattern in build-runtime.sh and
tests/build.sh. Also create the directory with mkdir -p / New-Item -Force
to avoid issues with MSBuild's EnsureDirectoryExists (which has no try/catch
in its static constructor).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR ensures MSBuild crash diagnostics (e.g., MSBuild_*.failure.txt from MSB4166) are consistently written under artifacts/log/ by setting up and exporting MSBUILDDEBUGPATH directly in the top-level build scripts (eng/build.sh and eng/build.ps1), rather than relying solely on CI pipeline variables.

Changes:

  • Set and export MSBUILDDEBUGPATH in eng/build.sh and create the target directory before invoking MSBuild.
  • Set and export MSBUILDDEBUGPATH in eng/build.ps1, create the target directory, and update it per-configuration when building multiple configurations.
Show a summary per file
FileDescription
eng/build.shAdds MSBUILDDEBUGPATH initialization + mkdir -p so MSBuild crash logs land under artifacts/log/<Config>/MsbuildDebugLogs.
eng/build.ps1Adds MSBUILDDEBUGPATH initialization + directory creation, and updates the env var per configuration in multi-config builds.

Copilot's findings

Comments suppressed due to low confidence (1)

eng/build.ps1:451

  • Same as above: inside the per-configuration loop, New-Item errors are non-terminating by default. If directory creation fails, the build continues and MSBuild crash diagnostics may not be captured under artifacts/log. Make directory creation terminating (e.g., -ErrorAction Stop) to ensure reliability.
 $msbuildDebugLogsDir = "$PSScriptRoot/../artifacts/log/$titleCaseConfig/MsbuildDebugLogs"
New-Item -ItemType Directory -Force -Path $msbuildDebugLogsDir | Out-Null
$env:MSBUILDDEBUGPATH = $msbuildDebugLogsDir
  • Files reviewed: 2/2 changed files
  • Comments generated: 2

Comment threadeng/build.ps1
Comment threadeng/build.ps1
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates top-level build entrypoints to ensure MSBuild crash diagnostics (MSBuild_*.failure.txt from MSB4166/node crashes) are consistently written under artifacts/log/ so CI reliably publishes them.

Changes:

  • Set and export MSBUILDDEBUGPATH early in eng/build.sh, create the directory, and print it for diagnostics.
  • Set and export MSBUILDDEBUGPATH early in eng/build.ps1, create the directory, print it, and update it per configuration during multi-config builds.
Show a summary per file
FileDescription
eng/build.shExports MSBUILDDEBUGPATH to a stable artifacts/log/<config>/MsbuildDebugLogs directory before invoking eng/common/build.sh.
eng/build.ps1Exports MSBUILDDEBUGPATH to artifacts/log/<config>/MsbuildDebugLogs, ensures the directory exists, and adjusts it when iterating configurations.

Copilot's findings

  • Files reviewed: 2/2 changed files
  • Comments generated: 1

Comment threadeng/build.ps1
Comment on lines +445 to +451
$titleCaseConfig = $((Get-Culture).TextInfo.ToTitleCase($config))

# Update MSBuild debug logs directory for this configuration.
$msbuildDebugLogsDir = "$PSScriptRoot/../artifacts/log/$titleCaseConfig/MsbuildDebugLogs"
New-Item -ItemType Directory -Force -Path $msbuildDebugLogsDir | Out-Null
$env:MSBUILDDEBUGPATH = $msbuildDebugLogsDir
Write-Host "MSBUILDDEBUGPATH=$msbuildDebugLogsDir"

CopilotAIApr 12, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the multi-configuration loop you update $env:MSBUILDDEBUGPATH per config, but MSBuild node reuse is enabled by default for non-CI runs (/nr:true via eng/common/tools.ps1 where nodeReuse defaults to !$ci). Reused nodes won’t see the updated environment variable, so crash diagnostics may still be written to the previous configuration’s directory, making the per-config directory update potentially misleading. Consider either keeping a single MSBUILDDEBUGPATH for the whole build.ps1 invocation, or disabling node reuse when switching configurations so each config reliably writes to its intended directory.

See below for a potential fix:

# Use a single MSBuild debug logs directory for the entire invocation because
# reused MSBuild nodes do not observe environment variable updates made between
# configurations.
$msbuildDebugLogsDir = "$PSScriptRoot/../artifacts/log/MsbuildDebugLogs"
New-Item -ItemType Directory -Force -Path $msbuildDebugLogsDir | Out-Null
$env:MSBUILDDEBUGPATH = $msbuildDebugLogsDir
Write-Host "MSBUILDDEBUGPATH=$msbuildDebugLogsDir"
foreach ($config in $configuration) {
$titleCaseConfig = $((Get-Culture).TextInfo.ToTitleCase($config))

Copilot uses AI. Check for mistakes.
Comment threadeng/build.sh
# Set up the directory for MSBuild debug logs, so that if MSBuild crashes (MSB4166)
# the failure.txt diagnostics are written to a known location under artifacts/log
# where they'll be captured as build artifacts.
MSBUILDDEBUGPATH="$scriptroot/../artifacts/log/${bootstrapConfig:-Debug}/MsbuildDebugLogs"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why does this use ${bootstrapConfig:-Debug} ? I would expect it to use actual config for non-bootstrap builds.

Comment threadeng/build.sh
done

# Set a different path for prebuilt usage tracking for the bootstrap build.
"$scriptroot/common/build.sh" ${bootstrapArguments[@]+"${bootstrapArguments[@]}"} /p:Subset=bootstrap /p:TrackPrebuiltUsageReportFile=$scriptroot/../artifacts/log/bootstrap-prebuilt-usage.xml -bl:$scriptroot/../artifacts/log/$bootstrapConfig/bootstrap.binlog

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be better to put this instrumentation into the common scripts like "$scriptroot/common/build.sh?

@jkotas

Copy link
Copy Markdown
Member

The existing pipeline variable in global-build-job.yml is left in place as defense-in-depth.

Defense-in-depth against what?

@danmoseley

Copy link
Copy Markdown
ContributorAuthor

Let's see whether we can do in arcade instead

Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

MSBuild crashing in the build

3 participants

@danmoseley@jkotas
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

Set MSBUILDDEBUGPATH in build scripts for reliable crash diagnostics - #126806

Closed
danmoseley wants to merge 3 commits into
dotnet:mainfrom
danmoseley:fix-msbuilddebugpath
Closed

Set MSBUILDDEBUGPATH in build scripts for reliable crash diagnostics#126806
danmoseley wants to merge 3 commits into
dotnet:mainfrom
danmoseley:fix-msbuilddebugpath

Conversation

@danmoseley

@danmoseleydanmoseley commented Apr 12, 2026

Copy link
Copy Markdown
Contributor

Set MSBUILDDEBUGPATH directly in eng/build.sh for reliable crash diagnostics

PR #126012 added MSBUILDDEBUGPATH as an AzDO pipeline variable in global-build-job.yml so that MSBuild crash diagnostics (MSBuild_*.failure.txt) would be written to artifacts/log/ instead of the system temp directory. However, recent builds that hit the MSB4166 crash (e.g., build 1370423) still show MSBuild writing to the default temp path, indicating the pipeline variable is not reaching the MSBuild process.

Other build scripts (src/coreclr/build-runtime.sh, src/tests/build.sh) already set and export MSBUILDDEBUGPATH independently, which is why "Build Tests" steps work. The "Build product" step calls eng/build.sh, which did not set it.

This PR sets MSBUILDDEBUGPATH directly in eng/build.sh and eng/build.ps1 before any common/build.sh/common/build.ps1 invocations, matching the pattern used by the other scripts. This also:

  • Creates the directory with mkdir -p before MSBuild starts (MSBuild's EnsureDirectoryExists has no try/catch in its static constructor, so a failed directory creation would silently fall back to the temp dir)
  • Echoes the path for diagnostic visibility in build logs
  • Covers both the bootstrap and product build invocations

The existing pipeline variable in global-build-job.yml is left in place as defense-in-depth.

Relates to getting data to fix#92290

…rash diagnostics
The MSBUILDDEBUGPATH pipeline variable added in PR dotnet#126012 is not reaching
the MSBuild process during Build product steps. Set the env var directly in
the build scripts, matching the existing pattern in build-runtime.sh and
tests/build.sh. Also create the directory with mkdir -p / New-Item -Force
to avoid issues with MSBuild's EnsureDirectoryExists (which has no try/catch
in its static constructor).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR ensures MSBuild crash diagnostics (e.g., MSBuild_*.failure.txt from MSB4166) are consistently written under artifacts/log/ by setting up and exporting MSBUILDDEBUGPATH directly in the top-level build scripts (eng/build.sh and eng/build.ps1), rather than relying solely on CI pipeline variables.

Changes:

  • Set and export MSBUILDDEBUGPATH in eng/build.sh and create the target directory before invoking MSBuild.
  • Set and export MSBUILDDEBUGPATH in eng/build.ps1, create the target directory, and update it per-configuration when building multiple configurations.
Show a summary per file
FileDescription
eng/build.shAdds MSBUILDDEBUGPATH initialization + mkdir -p so MSBuild crash logs land under artifacts/log/<Config>/MsbuildDebugLogs.
eng/build.ps1Adds MSBUILDDEBUGPATH initialization + directory creation, and updates the env var per configuration in multi-config builds.

Copilot's findings

Comments suppressed due to low confidence (1)

eng/build.ps1:451

  • Same as above: inside the per-configuration loop, New-Item errors are non-terminating by default. If directory creation fails, the build continues and MSBuild crash diagnostics may not be captured under artifacts/log. Make directory creation terminating (e.g., -ErrorAction Stop) to ensure reliability.
 $msbuildDebugLogsDir = "$PSScriptRoot/../artifacts/log/$titleCaseConfig/MsbuildDebugLogs"
New-Item -ItemType Directory -Force -Path $msbuildDebugLogsDir | Out-Null
$env:MSBUILDDEBUGPATH = $msbuildDebugLogsDir
  • Files reviewed: 2/2 changed files
  • Comments generated: 2

Comment threadeng/build.ps1
Comment threadeng/build.ps1
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates top-level build entrypoints to ensure MSBuild crash diagnostics (MSBuild_*.failure.txt from MSB4166/node crashes) are consistently written under artifacts/log/ so CI reliably publishes them.

Changes:

  • Set and export MSBUILDDEBUGPATH early in eng/build.sh, create the directory, and print it for diagnostics.
  • Set and export MSBUILDDEBUGPATH early in eng/build.ps1, create the directory, print it, and update it per configuration during multi-config builds.
Show a summary per file
FileDescription
eng/build.shExports MSBUILDDEBUGPATH to a stable artifacts/log/<config>/MsbuildDebugLogs directory before invoking eng/common/build.sh.
eng/build.ps1Exports MSBUILDDEBUGPATH to artifacts/log/<config>/MsbuildDebugLogs, ensures the directory exists, and adjusts it when iterating configurations.

Copilot's findings

  • Files reviewed: 2/2 changed files
  • Comments generated: 1

Comment threadeng/build.ps1
Comment on lines +445 to +451
$titleCaseConfig = $((Get-Culture).TextInfo.ToTitleCase($config))

# Update MSBuild debug logs directory for this configuration.
$msbuildDebugLogsDir = "$PSScriptRoot/../artifacts/log/$titleCaseConfig/MsbuildDebugLogs"
New-Item -ItemType Directory -Force -Path $msbuildDebugLogsDir | Out-Null
$env:MSBUILDDEBUGPATH = $msbuildDebugLogsDir
Write-Host "MSBUILDDEBUGPATH=$msbuildDebugLogsDir"

CopilotAIApr 12, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the multi-configuration loop you update $env:MSBUILDDEBUGPATH per config, but MSBuild node reuse is enabled by default for non-CI runs (/nr:true via eng/common/tools.ps1 where nodeReuse defaults to !$ci). Reused nodes won’t see the updated environment variable, so crash diagnostics may still be written to the previous configuration’s directory, making the per-config directory update potentially misleading. Consider either keeping a single MSBUILDDEBUGPATH for the whole build.ps1 invocation, or disabling node reuse when switching configurations so each config reliably writes to its intended directory.

See below for a potential fix:

# Use a single MSBuild debug logs directory for the entire invocation because
# reused MSBuild nodes do not observe environment variable updates made between
# configurations.
$msbuildDebugLogsDir = "$PSScriptRoot/../artifacts/log/MsbuildDebugLogs"
New-Item -ItemType Directory -Force -Path $msbuildDebugLogsDir | Out-Null
$env:MSBUILDDEBUGPATH = $msbuildDebugLogsDir
Write-Host "MSBUILDDEBUGPATH=$msbuildDebugLogsDir"
foreach ($config in $configuration) {
$titleCaseConfig = $((Get-Culture).TextInfo.ToTitleCase($config))

Copilot uses AI. Check for mistakes.
Comment threadeng/build.sh
# Set up the directory for MSBuild debug logs, so that if MSBuild crashes (MSB4166)
# the failure.txt diagnostics are written to a known location under artifacts/log
# where they'll be captured as build artifacts.
MSBUILDDEBUGPATH="$scriptroot/../artifacts/log/${bootstrapConfig:-Debug}/MsbuildDebugLogs"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why does this use ${bootstrapConfig:-Debug} ? I would expect it to use actual config for non-bootstrap builds.

Comment threadeng/build.sh
done

# Set a different path for prebuilt usage tracking for the bootstrap build.
"$scriptroot/common/build.sh" ${bootstrapArguments[@]+"${bootstrapArguments[@]}"} /p:Subset=bootstrap /p:TrackPrebuiltUsageReportFile=$scriptroot/../artifacts/log/bootstrap-prebuilt-usage.xml -bl:$scriptroot/../artifacts/log/$bootstrapConfig/bootstrap.binlog

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be better to put this instrumentation into the common scripts like "$scriptroot/common/build.sh?

@jkotas

Copy link
Copy Markdown
Member

The existing pipeline variable in global-build-job.yml is left in place as defense-in-depth.

Defense-in-depth against what?

@danmoseley

Copy link
Copy Markdown
ContributorAuthor

Let's see whether we can do in arcade instead

Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

MSBuild crashing in the build

3 participants

@danmoseley@jkotas
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Set MSBUILDDEBUGPATH in build scripts for reliable crash diagnostics - #126806

Closed
danmoseley wants to merge 3 commits into
dotnet:mainfrom
danmoseley:fix-msbuilddebugpath
Closed

Set MSBUILDDEBUGPATH in build scripts for reliable crash diagnostics#126806
danmoseley wants to merge 3 commits into
dotnet:mainfrom
danmoseley:fix-msbuilddebugpath

Conversation

@danmoseley

@danmoseleydanmoseley commented Apr 12, 2026

Copy link
Copy Markdown
Contributor

Set MSBUILDDEBUGPATH directly in eng/build.sh for reliable crash diagnostics

PR #126012 added MSBUILDDEBUGPATH as an AzDO pipeline variable in global-build-job.yml so that MSBuild crash diagnostics (MSBuild_*.failure.txt) would be written to artifacts/log/ instead of the system temp directory. However, recent builds that hit the MSB4166 crash (e.g., build 1370423) still show MSBuild writing to the default temp path, indicating the pipeline variable is not reaching the MSBuild process.

Other build scripts (src/coreclr/build-runtime.sh, src/tests/build.sh) already set and export MSBUILDDEBUGPATH independently, which is why "Build Tests" steps work. The "Build product" step calls eng/build.sh, which did not set it.

This PR sets MSBUILDDEBUGPATH directly in eng/build.sh and eng/build.ps1 before any common/build.sh/common/build.ps1 invocations, matching the pattern used by the other scripts. This also:

  • Creates the directory with mkdir -p before MSBuild starts (MSBuild's EnsureDirectoryExists has no try/catch in its static constructor, so a failed directory creation would silently fall back to the temp dir)
  • Echoes the path for diagnostic visibility in build logs
  • Covers both the bootstrap and product build invocations

The existing pipeline variable in global-build-job.yml is left in place as defense-in-depth.

Relates to getting data to fix#92290

…rash diagnostics
The MSBUILDDEBUGPATH pipeline variable added in PR dotnet#126012 is not reaching
the MSBuild process during Build product steps. Set the env var directly in
the build scripts, matching the existing pattern in build-runtime.sh and
tests/build.sh. Also create the directory with mkdir -p / New-Item -Force
to avoid issues with MSBuild's EnsureDirectoryExists (which has no try/catch
in its static constructor).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR ensures MSBuild crash diagnostics (e.g., MSBuild_*.failure.txt from MSB4166) are consistently written under artifacts/log/ by setting up and exporting MSBUILDDEBUGPATH directly in the top-level build scripts (eng/build.sh and eng/build.ps1), rather than relying solely on CI pipeline variables.

Changes:

  • Set and export MSBUILDDEBUGPATH in eng/build.sh and create the target directory before invoking MSBuild.
  • Set and export MSBUILDDEBUGPATH in eng/build.ps1, create the target directory, and update it per-configuration when building multiple configurations.
Show a summary per file
FileDescription
eng/build.shAdds MSBUILDDEBUGPATH initialization + mkdir -p so MSBuild crash logs land under artifacts/log/<Config>/MsbuildDebugLogs.
eng/build.ps1Adds MSBUILDDEBUGPATH initialization + directory creation, and updates the env var per configuration in multi-config builds.

Copilot's findings

Comments suppressed due to low confidence (1)

eng/build.ps1:451

  • Same as above: inside the per-configuration loop, New-Item errors are non-terminating by default. If directory creation fails, the build continues and MSBuild crash diagnostics may not be captured under artifacts/log. Make directory creation terminating (e.g., -ErrorAction Stop) to ensure reliability.
 $msbuildDebugLogsDir = "$PSScriptRoot/../artifacts/log/$titleCaseConfig/MsbuildDebugLogs"
New-Item -ItemType Directory -Force -Path $msbuildDebugLogsDir | Out-Null
$env:MSBUILDDEBUGPATH = $msbuildDebugLogsDir
  • Files reviewed: 2/2 changed files
  • Comments generated: 2

Comment threadeng/build.ps1
Comment threadeng/build.ps1
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates top-level build entrypoints to ensure MSBuild crash diagnostics (MSBuild_*.failure.txt from MSB4166/node crashes) are consistently written under artifacts/log/ so CI reliably publishes them.

Changes:

  • Set and export MSBUILDDEBUGPATH early in eng/build.sh, create the directory, and print it for diagnostics.
  • Set and export MSBUILDDEBUGPATH early in eng/build.ps1, create the directory, print it, and update it per configuration during multi-config builds.
Show a summary per file
FileDescription
eng/build.shExports MSBUILDDEBUGPATH to a stable artifacts/log/<config>/MsbuildDebugLogs directory before invoking eng/common/build.sh.
eng/build.ps1Exports MSBUILDDEBUGPATH to artifacts/log/<config>/MsbuildDebugLogs, ensures the directory exists, and adjusts it when iterating configurations.

Copilot's findings

  • Files reviewed: 2/2 changed files
  • Comments generated: 1

Comment threadeng/build.ps1
Comment on lines +445 to +451
$titleCaseConfig = $((Get-Culture).TextInfo.ToTitleCase($config))

# Update MSBuild debug logs directory for this configuration.
$msbuildDebugLogsDir = "$PSScriptRoot/../artifacts/log/$titleCaseConfig/MsbuildDebugLogs"
New-Item -ItemType Directory -Force -Path $msbuildDebugLogsDir | Out-Null
$env:MSBUILDDEBUGPATH = $msbuildDebugLogsDir
Write-Host "MSBUILDDEBUGPATH=$msbuildDebugLogsDir"

CopilotAIApr 12, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the multi-configuration loop you update $env:MSBUILDDEBUGPATH per config, but MSBuild node reuse is enabled by default for non-CI runs (/nr:true via eng/common/tools.ps1 where nodeReuse defaults to !$ci). Reused nodes won’t see the updated environment variable, so crash diagnostics may still be written to the previous configuration’s directory, making the per-config directory update potentially misleading. Consider either keeping a single MSBUILDDEBUGPATH for the whole build.ps1 invocation, or disabling node reuse when switching configurations so each config reliably writes to its intended directory.

See below for a potential fix:

# Use a single MSBuild debug logs directory for the entire invocation because
# reused MSBuild nodes do not observe environment variable updates made between
# configurations.
$msbuildDebugLogsDir = "$PSScriptRoot/../artifacts/log/MsbuildDebugLogs"
New-Item -ItemType Directory -Force -Path $msbuildDebugLogsDir | Out-Null
$env:MSBUILDDEBUGPATH = $msbuildDebugLogsDir
Write-Host "MSBUILDDEBUGPATH=$msbuildDebugLogsDir"
foreach ($config in $configuration) {
$titleCaseConfig = $((Get-Culture).TextInfo.ToTitleCase($config))

Copilot uses AI. Check for mistakes.
Comment threadeng/build.sh
# Set up the directory for MSBuild debug logs, so that if MSBuild crashes (MSB4166)
# the failure.txt diagnostics are written to a known location under artifacts/log
# where they'll be captured as build artifacts.
MSBUILDDEBUGPATH="$scriptroot/../artifacts/log/${bootstrapConfig:-Debug}/MsbuildDebugLogs"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why does this use ${bootstrapConfig:-Debug} ? I would expect it to use actual config for non-bootstrap builds.

Comment threadeng/build.sh
done

# Set a different path for prebuilt usage tracking for the bootstrap build.
"$scriptroot/common/build.sh" ${bootstrapArguments[@]+"${bootstrapArguments[@]}"} /p:Subset=bootstrap /p:TrackPrebuiltUsageReportFile=$scriptroot/../artifacts/log/bootstrap-prebuilt-usage.xml -bl:$scriptroot/../artifacts/log/$bootstrapConfig/bootstrap.binlog

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be better to put this instrumentation into the common scripts like "$scriptroot/common/build.sh?

@jkotas

Copy link
Copy Markdown
Member

The existing pipeline variable in global-build-job.yml is left in place as defense-in-depth.

Defense-in-depth against what?

@danmoseley

Copy link
Copy Markdown
ContributorAuthor

Let's see whether we can do in arcade instead

Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

MSBuild crashing in the build

3 participants

@danmoseley@jkotas
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Set MSBUILDDEBUGPATH in build scripts for reliable crash diagnostics - #126806

Closed
danmoseley wants to merge 3 commits into
dotnet:mainfrom
danmoseley:fix-msbuilddebugpath
Closed

Set MSBUILDDEBUGPATH in build scripts for reliable crash diagnostics#126806
danmoseley wants to merge 3 commits into
dotnet:mainfrom
danmoseley:fix-msbuilddebugpath

Conversation

@danmoseley

@danmoseleydanmoseley commented Apr 12, 2026

Copy link
Copy Markdown
Contributor

Set MSBUILDDEBUGPATH directly in eng/build.sh for reliable crash diagnostics

PR #126012 added MSBUILDDEBUGPATH as an AzDO pipeline variable in global-build-job.yml so that MSBuild crash diagnostics (MSBuild_*.failure.txt) would be written to artifacts/log/ instead of the system temp directory. However, recent builds that hit the MSB4166 crash (e.g., build 1370423) still show MSBuild writing to the default temp path, indicating the pipeline variable is not reaching the MSBuild process.

Other build scripts (src/coreclr/build-runtime.sh, src/tests/build.sh) already set and export MSBUILDDEBUGPATH independently, which is why "Build Tests" steps work. The "Build product" step calls eng/build.sh, which did not set it.

This PR sets MSBUILDDEBUGPATH directly in eng/build.sh and eng/build.ps1 before any common/build.sh/common/build.ps1 invocations, matching the pattern used by the other scripts. This also:

  • Creates the directory with mkdir -p before MSBuild starts (MSBuild's EnsureDirectoryExists has no try/catch in its static constructor, so a failed directory creation would silently fall back to the temp dir)
  • Echoes the path for diagnostic visibility in build logs
  • Covers both the bootstrap and product build invocations

The existing pipeline variable in global-build-job.yml is left in place as defense-in-depth.

Relates to getting data to fix#92290

…rash diagnostics
The MSBUILDDEBUGPATH pipeline variable added in PR dotnet#126012 is not reaching
the MSBuild process during Build product steps. Set the env var directly in
the build scripts, matching the existing pattern in build-runtime.sh and
tests/build.sh. Also create the directory with mkdir -p / New-Item -Force
to avoid issues with MSBuild's EnsureDirectoryExists (which has no try/catch
in its static constructor).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR ensures MSBuild crash diagnostics (e.g., MSBuild_*.failure.txt from MSB4166) are consistently written under artifacts/log/ by setting up and exporting MSBUILDDEBUGPATH directly in the top-level build scripts (eng/build.sh and eng/build.ps1), rather than relying solely on CI pipeline variables.

Changes:

  • Set and export MSBUILDDEBUGPATH in eng/build.sh and create the target directory before invoking MSBuild.
  • Set and export MSBUILDDEBUGPATH in eng/build.ps1, create the target directory, and update it per-configuration when building multiple configurations.
Show a summary per file
FileDescription
eng/build.shAdds MSBUILDDEBUGPATH initialization + mkdir -p so MSBuild crash logs land under artifacts/log/<Config>/MsbuildDebugLogs.
eng/build.ps1Adds MSBUILDDEBUGPATH initialization + directory creation, and updates the env var per configuration in multi-config builds.

Copilot's findings

Comments suppressed due to low confidence (1)

eng/build.ps1:451

  • Same as above: inside the per-configuration loop, New-Item errors are non-terminating by default. If directory creation fails, the build continues and MSBuild crash diagnostics may not be captured under artifacts/log. Make directory creation terminating (e.g., -ErrorAction Stop) to ensure reliability.
 $msbuildDebugLogsDir = "$PSScriptRoot/../artifacts/log/$titleCaseConfig/MsbuildDebugLogs"
New-Item -ItemType Directory -Force -Path $msbuildDebugLogsDir | Out-Null
$env:MSBUILDDEBUGPATH = $msbuildDebugLogsDir
  • Files reviewed: 2/2 changed files
  • Comments generated: 2

Comment threadeng/build.ps1
Comment threadeng/build.ps1
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates top-level build entrypoints to ensure MSBuild crash diagnostics (MSBuild_*.failure.txt from MSB4166/node crashes) are consistently written under artifacts/log/ so CI reliably publishes them.

Changes:

  • Set and export MSBUILDDEBUGPATH early in eng/build.sh, create the directory, and print it for diagnostics.
  • Set and export MSBUILDDEBUGPATH early in eng/build.ps1, create the directory, print it, and update it per configuration during multi-config builds.
Show a summary per file
FileDescription
eng/build.shExports MSBUILDDEBUGPATH to a stable artifacts/log/<config>/MsbuildDebugLogs directory before invoking eng/common/build.sh.
eng/build.ps1Exports MSBUILDDEBUGPATH to artifacts/log/<config>/MsbuildDebugLogs, ensures the directory exists, and adjusts it when iterating configurations.

Copilot's findings

  • Files reviewed: 2/2 changed files
  • Comments generated: 1

Comment threadeng/build.ps1
Comment on lines +445 to +451
$titleCaseConfig = $((Get-Culture).TextInfo.ToTitleCase($config))

# Update MSBuild debug logs directory for this configuration.
$msbuildDebugLogsDir = "$PSScriptRoot/../artifacts/log/$titleCaseConfig/MsbuildDebugLogs"
New-Item -ItemType Directory -Force -Path $msbuildDebugLogsDir | Out-Null
$env:MSBUILDDEBUGPATH = $msbuildDebugLogsDir
Write-Host "MSBUILDDEBUGPATH=$msbuildDebugLogsDir"

CopilotAIApr 12, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the multi-configuration loop you update $env:MSBUILDDEBUGPATH per config, but MSBuild node reuse is enabled by default for non-CI runs (/nr:true via eng/common/tools.ps1 where nodeReuse defaults to !$ci). Reused nodes won’t see the updated environment variable, so crash diagnostics may still be written to the previous configuration’s directory, making the per-config directory update potentially misleading. Consider either keeping a single MSBUILDDEBUGPATH for the whole build.ps1 invocation, or disabling node reuse when switching configurations so each config reliably writes to its intended directory.

See below for a potential fix:

# Use a single MSBuild debug logs directory for the entire invocation because
# reused MSBuild nodes do not observe environment variable updates made between
# configurations.
$msbuildDebugLogsDir = "$PSScriptRoot/../artifacts/log/MsbuildDebugLogs"
New-Item -ItemType Directory -Force -Path $msbuildDebugLogsDir | Out-Null
$env:MSBUILDDEBUGPATH = $msbuildDebugLogsDir
Write-Host "MSBUILDDEBUGPATH=$msbuildDebugLogsDir"
foreach ($config in $configuration) {
$titleCaseConfig = $((Get-Culture).TextInfo.ToTitleCase($config))

Copilot uses AI. Check for mistakes.
Comment threadeng/build.sh
# Set up the directory for MSBuild debug logs, so that if MSBuild crashes (MSB4166)
# the failure.txt diagnostics are written to a known location under artifacts/log
# where they'll be captured as build artifacts.
MSBUILDDEBUGPATH="$scriptroot/../artifacts/log/${bootstrapConfig:-Debug}/MsbuildDebugLogs"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why does this use ${bootstrapConfig:-Debug} ? I would expect it to use actual config for non-bootstrap builds.

Comment threadeng/build.sh
done

# Set a different path for prebuilt usage tracking for the bootstrap build.
"$scriptroot/common/build.sh" ${bootstrapArguments[@]+"${bootstrapArguments[@]}"} /p:Subset=bootstrap /p:TrackPrebuiltUsageReportFile=$scriptroot/../artifacts/log/bootstrap-prebuilt-usage.xml -bl:$scriptroot/../artifacts/log/$bootstrapConfig/bootstrap.binlog

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be better to put this instrumentation into the common scripts like "$scriptroot/common/build.sh?

@jkotas

Copy link
Copy Markdown
Member

The existing pipeline variable in global-build-job.yml is left in place as defense-in-depth.

Defense-in-depth against what?

@danmoseley

Copy link
Copy Markdown
ContributorAuthor

Let's see whether we can do in arcade instead

Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

MSBuild crashing in the build

3 participants

@danmoseley@jkotas
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

Set MSBUILDDEBUGPATH in build scripts for reliable crash diagnostics - #126806

Closed
danmoseley wants to merge 3 commits into
dotnet:mainfrom
danmoseley:fix-msbuilddebugpath
Closed

Set MSBUILDDEBUGPATH in build scripts for reliable crash diagnostics#126806
danmoseley wants to merge 3 commits into
dotnet:mainfrom
danmoseley:fix-msbuilddebugpath

Conversation

@danmoseley

@danmoseleydanmoseley commented Apr 12, 2026

Copy link
Copy Markdown
Contributor

Set MSBUILDDEBUGPATH directly in eng/build.sh for reliable crash diagnostics

PR #126012 added MSBUILDDEBUGPATH as an AzDO pipeline variable in global-build-job.yml so that MSBuild crash diagnostics (MSBuild_*.failure.txt) would be written to artifacts/log/ instead of the system temp directory. However, recent builds that hit the MSB4166 crash (e.g., build 1370423) still show MSBuild writing to the default temp path, indicating the pipeline variable is not reaching the MSBuild process.

Other build scripts (src/coreclr/build-runtime.sh, src/tests/build.sh) already set and export MSBUILDDEBUGPATH independently, which is why "Build Tests" steps work. The "Build product" step calls eng/build.sh, which did not set it.

This PR sets MSBUILDDEBUGPATH directly in eng/build.sh and eng/build.ps1 before any common/build.sh/common/build.ps1 invocations, matching the pattern used by the other scripts. This also:

  • Creates the directory with mkdir -p before MSBuild starts (MSBuild's EnsureDirectoryExists has no try/catch in its static constructor, so a failed directory creation would silently fall back to the temp dir)
  • Echoes the path for diagnostic visibility in build logs
  • Covers both the bootstrap and product build invocations

The existing pipeline variable in global-build-job.yml is left in place as defense-in-depth.

Relates to getting data to fix#92290

…rash diagnostics
The MSBUILDDEBUGPATH pipeline variable added in PR dotnet#126012 is not reaching
the MSBuild process during Build product steps. Set the env var directly in
the build scripts, matching the existing pattern in build-runtime.sh and
tests/build.sh. Also create the directory with mkdir -p / New-Item -Force
to avoid issues with MSBuild's EnsureDirectoryExists (which has no try/catch
in its static constructor).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR ensures MSBuild crash diagnostics (e.g., MSBuild_*.failure.txt from MSB4166) are consistently written under artifacts/log/ by setting up and exporting MSBUILDDEBUGPATH directly in the top-level build scripts (eng/build.sh and eng/build.ps1), rather than relying solely on CI pipeline variables.

Changes:

  • Set and export MSBUILDDEBUGPATH in eng/build.sh and create the target directory before invoking MSBuild.
  • Set and export MSBUILDDEBUGPATH in eng/build.ps1, create the target directory, and update it per-configuration when building multiple configurations.
Show a summary per file
FileDescription
eng/build.shAdds MSBUILDDEBUGPATH initialization + mkdir -p so MSBuild crash logs land under artifacts/log/<Config>/MsbuildDebugLogs.
eng/build.ps1Adds MSBUILDDEBUGPATH initialization + directory creation, and updates the env var per configuration in multi-config builds.

Copilot's findings

Comments suppressed due to low confidence (1)

eng/build.ps1:451

  • Same as above: inside the per-configuration loop, New-Item errors are non-terminating by default. If directory creation fails, the build continues and MSBuild crash diagnostics may not be captured under artifacts/log. Make directory creation terminating (e.g., -ErrorAction Stop) to ensure reliability.
 $msbuildDebugLogsDir = "$PSScriptRoot/../artifacts/log/$titleCaseConfig/MsbuildDebugLogs"
New-Item -ItemType Directory -Force -Path $msbuildDebugLogsDir | Out-Null
$env:MSBUILDDEBUGPATH = $msbuildDebugLogsDir
  • Files reviewed: 2/2 changed files
  • Comments generated: 2

Comment threadeng/build.ps1
Comment threadeng/build.ps1
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates top-level build entrypoints to ensure MSBuild crash diagnostics (MSBuild_*.failure.txt from MSB4166/node crashes) are consistently written under artifacts/log/ so CI reliably publishes them.

Changes:

  • Set and export MSBUILDDEBUGPATH early in eng/build.sh, create the directory, and print it for diagnostics.
  • Set and export MSBUILDDEBUGPATH early in eng/build.ps1, create the directory, print it, and update it per configuration during multi-config builds.
Show a summary per file
FileDescription
eng/build.shExports MSBUILDDEBUGPATH to a stable artifacts/log/<config>/MsbuildDebugLogs directory before invoking eng/common/build.sh.
eng/build.ps1Exports MSBUILDDEBUGPATH to artifacts/log/<config>/MsbuildDebugLogs, ensures the directory exists, and adjusts it when iterating configurations.

Copilot's findings

  • Files reviewed: 2/2 changed files
  • Comments generated: 1

Comment threadeng/build.ps1
Comment on lines +445 to +451
$titleCaseConfig = $((Get-Culture).TextInfo.ToTitleCase($config))

# Update MSBuild debug logs directory for this configuration.
$msbuildDebugLogsDir = "$PSScriptRoot/../artifacts/log/$titleCaseConfig/MsbuildDebugLogs"
New-Item -ItemType Directory -Force -Path $msbuildDebugLogsDir | Out-Null
$env:MSBUILDDEBUGPATH = $msbuildDebugLogsDir
Write-Host "MSBUILDDEBUGPATH=$msbuildDebugLogsDir"

CopilotAIApr 12, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the multi-configuration loop you update $env:MSBUILDDEBUGPATH per config, but MSBuild node reuse is enabled by default for non-CI runs (/nr:true via eng/common/tools.ps1 where nodeReuse defaults to !$ci). Reused nodes won’t see the updated environment variable, so crash diagnostics may still be written to the previous configuration’s directory, making the per-config directory update potentially misleading. Consider either keeping a single MSBUILDDEBUGPATH for the whole build.ps1 invocation, or disabling node reuse when switching configurations so each config reliably writes to its intended directory.

See below for a potential fix:

# Use a single MSBuild debug logs directory for the entire invocation because
# reused MSBuild nodes do not observe environment variable updates made between
# configurations.
$msbuildDebugLogsDir = "$PSScriptRoot/../artifacts/log/MsbuildDebugLogs"
New-Item -ItemType Directory -Force -Path $msbuildDebugLogsDir | Out-Null
$env:MSBUILDDEBUGPATH = $msbuildDebugLogsDir
Write-Host "MSBUILDDEBUGPATH=$msbuildDebugLogsDir"
foreach ($config in $configuration) {
$titleCaseConfig = $((Get-Culture).TextInfo.ToTitleCase($config))

Copilot uses AI. Check for mistakes.
Comment threadeng/build.sh
# Set up the directory for MSBuild debug logs, so that if MSBuild crashes (MSB4166)
# the failure.txt diagnostics are written to a known location under artifacts/log
# where they'll be captured as build artifacts.
MSBUILDDEBUGPATH="$scriptroot/../artifacts/log/${bootstrapConfig:-Debug}/MsbuildDebugLogs"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why does this use ${bootstrapConfig:-Debug} ? I would expect it to use actual config for non-bootstrap builds.

Comment threadeng/build.sh
done

# Set a different path for prebuilt usage tracking for the bootstrap build.
"$scriptroot/common/build.sh" ${bootstrapArguments[@]+"${bootstrapArguments[@]}"} /p:Subset=bootstrap /p:TrackPrebuiltUsageReportFile=$scriptroot/../artifacts/log/bootstrap-prebuilt-usage.xml -bl:$scriptroot/../artifacts/log/$bootstrapConfig/bootstrap.binlog

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be better to put this instrumentation into the common scripts like "$scriptroot/common/build.sh?

@jkotas

Copy link
Copy Markdown
Member

The existing pipeline variable in global-build-job.yml is left in place as defense-in-depth.

Defense-in-depth against what?

@danmoseley

Copy link
Copy Markdown
ContributorAuthor

Let's see whether we can do in arcade instead

Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

MSBuild crashing in the build

3 participants

@danmoseley@jkotas
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Set MSBUILDDEBUGPATH in build scripts for reliable crash diagnostics - #126806

Closed
danmoseley wants to merge 3 commits into
dotnet:mainfrom
danmoseley:fix-msbuilddebugpath
Closed

Set MSBUILDDEBUGPATH in build scripts for reliable crash diagnostics#126806
danmoseley wants to merge 3 commits into
dotnet:mainfrom
danmoseley:fix-msbuilddebugpath

Conversation

@danmoseley

@danmoseleydanmoseley commented Apr 12, 2026

Copy link
Copy Markdown
Contributor

Set MSBUILDDEBUGPATH directly in eng/build.sh for reliable crash diagnostics

PR #126012 added MSBUILDDEBUGPATH as an AzDO pipeline variable in global-build-job.yml so that MSBuild crash diagnostics (MSBuild_*.failure.txt) would be written to artifacts/log/ instead of the system temp directory. However, recent builds that hit the MSB4166 crash (e.g., build 1370423) still show MSBuild writing to the default temp path, indicating the pipeline variable is not reaching the MSBuild process.

Other build scripts (src/coreclr/build-runtime.sh, src/tests/build.sh) already set and export MSBUILDDEBUGPATH independently, which is why "Build Tests" steps work. The "Build product" step calls eng/build.sh, which did not set it.

This PR sets MSBUILDDEBUGPATH directly in eng/build.sh and eng/build.ps1 before any common/build.sh/common/build.ps1 invocations, matching the pattern used by the other scripts. This also:

  • Creates the directory with mkdir -p before MSBuild starts (MSBuild's EnsureDirectoryExists has no try/catch in its static constructor, so a failed directory creation would silently fall back to the temp dir)
  • Echoes the path for diagnostic visibility in build logs
  • Covers both the bootstrap and product build invocations

The existing pipeline variable in global-build-job.yml is left in place as defense-in-depth.

Relates to getting data to fix#92290

…rash diagnostics
The MSBUILDDEBUGPATH pipeline variable added in PR dotnet#126012 is not reaching
the MSBuild process during Build product steps. Set the env var directly in
the build scripts, matching the existing pattern in build-runtime.sh and
tests/build.sh. Also create the directory with mkdir -p / New-Item -Force
to avoid issues with MSBuild's EnsureDirectoryExists (which has no try/catch
in its static constructor).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR ensures MSBuild crash diagnostics (e.g., MSBuild_*.failure.txt from MSB4166) are consistently written under artifacts/log/ by setting up and exporting MSBUILDDEBUGPATH directly in the top-level build scripts (eng/build.sh and eng/build.ps1), rather than relying solely on CI pipeline variables.

Changes:

  • Set and export MSBUILDDEBUGPATH in eng/build.sh and create the target directory before invoking MSBuild.
  • Set and export MSBUILDDEBUGPATH in eng/build.ps1, create the target directory, and update it per-configuration when building multiple configurations.
Show a summary per file
FileDescription
eng/build.shAdds MSBUILDDEBUGPATH initialization + mkdir -p so MSBuild crash logs land under artifacts/log/<Config>/MsbuildDebugLogs.
eng/build.ps1Adds MSBUILDDEBUGPATH initialization + directory creation, and updates the env var per configuration in multi-config builds.

Copilot's findings

Comments suppressed due to low confidence (1)

eng/build.ps1:451

  • Same as above: inside the per-configuration loop, New-Item errors are non-terminating by default. If directory creation fails, the build continues and MSBuild crash diagnostics may not be captured under artifacts/log. Make directory creation terminating (e.g., -ErrorAction Stop) to ensure reliability.
 $msbuildDebugLogsDir = "$PSScriptRoot/../artifacts/log/$titleCaseConfig/MsbuildDebugLogs"
New-Item -ItemType Directory -Force -Path $msbuildDebugLogsDir | Out-Null
$env:MSBUILDDEBUGPATH = $msbuildDebugLogsDir
  • Files reviewed: 2/2 changed files
  • Comments generated: 2

Comment threadeng/build.ps1
Comment threadeng/build.ps1
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates top-level build entrypoints to ensure MSBuild crash diagnostics (MSBuild_*.failure.txt from MSB4166/node crashes) are consistently written under artifacts/log/ so CI reliably publishes them.

Changes:

  • Set and export MSBUILDDEBUGPATH early in eng/build.sh, create the directory, and print it for diagnostics.
  • Set and export MSBUILDDEBUGPATH early in eng/build.ps1, create the directory, print it, and update it per configuration during multi-config builds.
Show a summary per file
FileDescription
eng/build.shExports MSBUILDDEBUGPATH to a stable artifacts/log/<config>/MsbuildDebugLogs directory before invoking eng/common/build.sh.
eng/build.ps1Exports MSBUILDDEBUGPATH to artifacts/log/<config>/MsbuildDebugLogs, ensures the directory exists, and adjusts it when iterating configurations.

Copilot's findings

  • Files reviewed: 2/2 changed files
  • Comments generated: 1

Comment threadeng/build.ps1
Comment on lines +445 to +451
$titleCaseConfig = $((Get-Culture).TextInfo.ToTitleCase($config))

# Update MSBuild debug logs directory for this configuration.
$msbuildDebugLogsDir = "$PSScriptRoot/../artifacts/log/$titleCaseConfig/MsbuildDebugLogs"
New-Item -ItemType Directory -Force -Path $msbuildDebugLogsDir | Out-Null
$env:MSBUILDDEBUGPATH = $msbuildDebugLogsDir
Write-Host "MSBUILDDEBUGPATH=$msbuildDebugLogsDir"

CopilotAIApr 12, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the multi-configuration loop you update $env:MSBUILDDEBUGPATH per config, but MSBuild node reuse is enabled by default for non-CI runs (/nr:true via eng/common/tools.ps1 where nodeReuse defaults to !$ci). Reused nodes won’t see the updated environment variable, so crash diagnostics may still be written to the previous configuration’s directory, making the per-config directory update potentially misleading. Consider either keeping a single MSBUILDDEBUGPATH for the whole build.ps1 invocation, or disabling node reuse when switching configurations so each config reliably writes to its intended directory.

See below for a potential fix:

# Use a single MSBuild debug logs directory for the entire invocation because
# reused MSBuild nodes do not observe environment variable updates made between
# configurations.
$msbuildDebugLogsDir = "$PSScriptRoot/../artifacts/log/MsbuildDebugLogs"
New-Item -ItemType Directory -Force -Path $msbuildDebugLogsDir | Out-Null
$env:MSBUILDDEBUGPATH = $msbuildDebugLogsDir
Write-Host "MSBUILDDEBUGPATH=$msbuildDebugLogsDir"
foreach ($config in $configuration) {
$titleCaseConfig = $((Get-Culture).TextInfo.ToTitleCase($config))

Copilot uses AI. Check for mistakes.
Comment threadeng/build.sh
# Set up the directory for MSBuild debug logs, so that if MSBuild crashes (MSB4166)
# the failure.txt diagnostics are written to a known location under artifacts/log
# where they'll be captured as build artifacts.
MSBUILDDEBUGPATH="$scriptroot/../artifacts/log/${bootstrapConfig:-Debug}/MsbuildDebugLogs"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why does this use ${bootstrapConfig:-Debug} ? I would expect it to use actual config for non-bootstrap builds.

Comment threadeng/build.sh
done

# Set a different path for prebuilt usage tracking for the bootstrap build.
"$scriptroot/common/build.sh" ${bootstrapArguments[@]+"${bootstrapArguments[@]}"} /p:Subset=bootstrap /p:TrackPrebuiltUsageReportFile=$scriptroot/../artifacts/log/bootstrap-prebuilt-usage.xml -bl:$scriptroot/../artifacts/log/$bootstrapConfig/bootstrap.binlog

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be better to put this instrumentation into the common scripts like "$scriptroot/common/build.sh?

@jkotas

Copy link
Copy Markdown
Member

The existing pipeline variable in global-build-job.yml is left in place as defense-in-depth.

Defense-in-depth against what?

@danmoseley

Copy link
Copy Markdown
ContributorAuthor

Let's see whether we can do in arcade instead

Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

MSBuild crashing in the build

3 participants

@danmoseley@jkotas
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Set MSBUILDDEBUGPATH in build scripts for reliable crash diagnostics - #126806

Closed
danmoseley wants to merge 3 commits into
dotnet:mainfrom
danmoseley:fix-msbuilddebugpath
Closed

Set MSBUILDDEBUGPATH in build scripts for reliable crash diagnostics#126806
danmoseley wants to merge 3 commits into
dotnet:mainfrom
danmoseley:fix-msbuilddebugpath

Conversation

@danmoseley

@danmoseleydanmoseley commented Apr 12, 2026

Copy link
Copy Markdown
Contributor

Set MSBUILDDEBUGPATH directly in eng/build.sh for reliable crash diagnostics

PR #126012 added MSBUILDDEBUGPATH as an AzDO pipeline variable in global-build-job.yml so that MSBuild crash diagnostics (MSBuild_*.failure.txt) would be written to artifacts/log/ instead of the system temp directory. However, recent builds that hit the MSB4166 crash (e.g., build 1370423) still show MSBuild writing to the default temp path, indicating the pipeline variable is not reaching the MSBuild process.

Other build scripts (src/coreclr/build-runtime.sh, src/tests/build.sh) already set and export MSBUILDDEBUGPATH independently, which is why "Build Tests" steps work. The "Build product" step calls eng/build.sh, which did not set it.

This PR sets MSBUILDDEBUGPATH directly in eng/build.sh and eng/build.ps1 before any common/build.sh/common/build.ps1 invocations, matching the pattern used by the other scripts. This also:

  • Creates the directory with mkdir -p before MSBuild starts (MSBuild's EnsureDirectoryExists has no try/catch in its static constructor, so a failed directory creation would silently fall back to the temp dir)
  • Echoes the path for diagnostic visibility in build logs
  • Covers both the bootstrap and product build invocations

The existing pipeline variable in global-build-job.yml is left in place as defense-in-depth.

Relates to getting data to fix#92290

…rash diagnostics
The MSBUILDDEBUGPATH pipeline variable added in PR dotnet#126012 is not reaching
the MSBuild process during Build product steps. Set the env var directly in
the build scripts, matching the existing pattern in build-runtime.sh and
tests/build.sh. Also create the directory with mkdir -p / New-Item -Force
to avoid issues with MSBuild's EnsureDirectoryExists (which has no try/catch
in its static constructor).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR ensures MSBuild crash diagnostics (e.g., MSBuild_*.failure.txt from MSB4166) are consistently written under artifacts/log/ by setting up and exporting MSBUILDDEBUGPATH directly in the top-level build scripts (eng/build.sh and eng/build.ps1), rather than relying solely on CI pipeline variables.

Changes:

  • Set and export MSBUILDDEBUGPATH in eng/build.sh and create the target directory before invoking MSBuild.
  • Set and export MSBUILDDEBUGPATH in eng/build.ps1, create the target directory, and update it per-configuration when building multiple configurations.
Show a summary per file
FileDescription
eng/build.shAdds MSBUILDDEBUGPATH initialization + mkdir -p so MSBuild crash logs land under artifacts/log/<Config>/MsbuildDebugLogs.
eng/build.ps1Adds MSBUILDDEBUGPATH initialization + directory creation, and updates the env var per configuration in multi-config builds.

Copilot's findings

Comments suppressed due to low confidence (1)

eng/build.ps1:451

  • Same as above: inside the per-configuration loop, New-Item errors are non-terminating by default. If directory creation fails, the build continues and MSBuild crash diagnostics may not be captured under artifacts/log. Make directory creation terminating (e.g., -ErrorAction Stop) to ensure reliability.
 $msbuildDebugLogsDir = "$PSScriptRoot/../artifacts/log/$titleCaseConfig/MsbuildDebugLogs"
New-Item -ItemType Directory -Force -Path $msbuildDebugLogsDir | Out-Null
$env:MSBUILDDEBUGPATH = $msbuildDebugLogsDir
  • Files reviewed: 2/2 changed files
  • Comments generated: 2

Comment threadeng/build.ps1
Comment threadeng/build.ps1
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates top-level build entrypoints to ensure MSBuild crash diagnostics (MSBuild_*.failure.txt from MSB4166/node crashes) are consistently written under artifacts/log/ so CI reliably publishes them.

Changes:

  • Set and export MSBUILDDEBUGPATH early in eng/build.sh, create the directory, and print it for diagnostics.
  • Set and export MSBUILDDEBUGPATH early in eng/build.ps1, create the directory, print it, and update it per configuration during multi-config builds.
Show a summary per file
FileDescription
eng/build.shExports MSBUILDDEBUGPATH to a stable artifacts/log/<config>/MsbuildDebugLogs directory before invoking eng/common/build.sh.
eng/build.ps1Exports MSBUILDDEBUGPATH to artifacts/log/<config>/MsbuildDebugLogs, ensures the directory exists, and adjusts it when iterating configurations.

Copilot's findings

  • Files reviewed: 2/2 changed files
  • Comments generated: 1

Comment threadeng/build.ps1
Comment on lines +445 to +451
$titleCaseConfig = $((Get-Culture).TextInfo.ToTitleCase($config))

# Update MSBuild debug logs directory for this configuration.
$msbuildDebugLogsDir = "$PSScriptRoot/../artifacts/log/$titleCaseConfig/MsbuildDebugLogs"
New-Item -ItemType Directory -Force -Path $msbuildDebugLogsDir | Out-Null
$env:MSBUILDDEBUGPATH = $msbuildDebugLogsDir
Write-Host "MSBUILDDEBUGPATH=$msbuildDebugLogsDir"

CopilotAIApr 12, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the multi-configuration loop you update $env:MSBUILDDEBUGPATH per config, but MSBuild node reuse is enabled by default for non-CI runs (/nr:true via eng/common/tools.ps1 where nodeReuse defaults to !$ci). Reused nodes won’t see the updated environment variable, so crash diagnostics may still be written to the previous configuration’s directory, making the per-config directory update potentially misleading. Consider either keeping a single MSBUILDDEBUGPATH for the whole build.ps1 invocation, or disabling node reuse when switching configurations so each config reliably writes to its intended directory.

See below for a potential fix:

# Use a single MSBuild debug logs directory for the entire invocation because
# reused MSBuild nodes do not observe environment variable updates made between
# configurations.
$msbuildDebugLogsDir = "$PSScriptRoot/../artifacts/log/MsbuildDebugLogs"
New-Item -ItemType Directory -Force -Path $msbuildDebugLogsDir | Out-Null
$env:MSBUILDDEBUGPATH = $msbuildDebugLogsDir
Write-Host "MSBUILDDEBUGPATH=$msbuildDebugLogsDir"
foreach ($config in $configuration) {
$titleCaseConfig = $((Get-Culture).TextInfo.ToTitleCase($config))

Copilot uses AI. Check for mistakes.
Comment threadeng/build.sh
# Set up the directory for MSBuild debug logs, so that if MSBuild crashes (MSB4166)
# the failure.txt diagnostics are written to a known location under artifacts/log
# where they'll be captured as build artifacts.
MSBUILDDEBUGPATH="$scriptroot/../artifacts/log/${bootstrapConfig:-Debug}/MsbuildDebugLogs"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why does this use ${bootstrapConfig:-Debug} ? I would expect it to use actual config for non-bootstrap builds.

Comment threadeng/build.sh
done

# Set a different path for prebuilt usage tracking for the bootstrap build.
"$scriptroot/common/build.sh" ${bootstrapArguments[@]+"${bootstrapArguments[@]}"} /p:Subset=bootstrap /p:TrackPrebuiltUsageReportFile=$scriptroot/../artifacts/log/bootstrap-prebuilt-usage.xml -bl:$scriptroot/../artifacts/log/$bootstrapConfig/bootstrap.binlog

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be better to put this instrumentation into the common scripts like "$scriptroot/common/build.sh?

@jkotas

Copy link
Copy Markdown
Member

The existing pipeline variable in global-build-job.yml is left in place as defense-in-depth.

Defense-in-depth against what?

@danmoseley

Copy link
Copy Markdown
ContributorAuthor

Let's see whether we can do in arcade instead

Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

MSBuild crashing in the build

3 participants

@danmoseley@jkotas
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

Set MSBUILDDEBUGPATH in build scripts for reliable crash diagnostics - #126806

Closed
danmoseley wants to merge 3 commits into
dotnet:mainfrom
danmoseley:fix-msbuilddebugpath
Closed

Set MSBUILDDEBUGPATH in build scripts for reliable crash diagnostics#126806
danmoseley wants to merge 3 commits into
dotnet:mainfrom
danmoseley:fix-msbuilddebugpath

Conversation

@danmoseley

@danmoseleydanmoseley commented Apr 12, 2026

Copy link
Copy Markdown
Contributor

Set MSBUILDDEBUGPATH directly in eng/build.sh for reliable crash diagnostics

PR #126012 added MSBUILDDEBUGPATH as an AzDO pipeline variable in global-build-job.yml so that MSBuild crash diagnostics (MSBuild_*.failure.txt) would be written to artifacts/log/ instead of the system temp directory. However, recent builds that hit the MSB4166 crash (e.g., build 1370423) still show MSBuild writing to the default temp path, indicating the pipeline variable is not reaching the MSBuild process.

Other build scripts (src/coreclr/build-runtime.sh, src/tests/build.sh) already set and export MSBUILDDEBUGPATH independently, which is why "Build Tests" steps work. The "Build product" step calls eng/build.sh, which did not set it.

This PR sets MSBUILDDEBUGPATH directly in eng/build.sh and eng/build.ps1 before any common/build.sh/common/build.ps1 invocations, matching the pattern used by the other scripts. This also:

  • Creates the directory with mkdir -p before MSBuild starts (MSBuild's EnsureDirectoryExists has no try/catch in its static constructor, so a failed directory creation would silently fall back to the temp dir)
  • Echoes the path for diagnostic visibility in build logs
  • Covers both the bootstrap and product build invocations

The existing pipeline variable in global-build-job.yml is left in place as defense-in-depth.

Relates to getting data to fix#92290

…rash diagnostics
The MSBUILDDEBUGPATH pipeline variable added in PR dotnet#126012 is not reaching
the MSBuild process during Build product steps. Set the env var directly in
the build scripts, matching the existing pattern in build-runtime.sh and
tests/build.sh. Also create the directory with mkdir -p / New-Item -Force
to avoid issues with MSBuild's EnsureDirectoryExists (which has no try/catch
in its static constructor).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR ensures MSBuild crash diagnostics (e.g., MSBuild_*.failure.txt from MSB4166) are consistently written under artifacts/log/ by setting up and exporting MSBUILDDEBUGPATH directly in the top-level build scripts (eng/build.sh and eng/build.ps1), rather than relying solely on CI pipeline variables.

Changes:

  • Set and export MSBUILDDEBUGPATH in eng/build.sh and create the target directory before invoking MSBuild.
  • Set and export MSBUILDDEBUGPATH in eng/build.ps1, create the target directory, and update it per-configuration when building multiple configurations.
Show a summary per file
FileDescription
eng/build.shAdds MSBUILDDEBUGPATH initialization + mkdir -p so MSBuild crash logs land under artifacts/log/<Config>/MsbuildDebugLogs.
eng/build.ps1Adds MSBUILDDEBUGPATH initialization + directory creation, and updates the env var per configuration in multi-config builds.

Copilot's findings

Comments suppressed due to low confidence (1)

eng/build.ps1:451

  • Same as above: inside the per-configuration loop, New-Item errors are non-terminating by default. If directory creation fails, the build continues and MSBuild crash diagnostics may not be captured under artifacts/log. Make directory creation terminating (e.g., -ErrorAction Stop) to ensure reliability.
 $msbuildDebugLogsDir = "$PSScriptRoot/../artifacts/log/$titleCaseConfig/MsbuildDebugLogs"
New-Item -ItemType Directory -Force -Path $msbuildDebugLogsDir | Out-Null
$env:MSBUILDDEBUGPATH = $msbuildDebugLogsDir
  • Files reviewed: 2/2 changed files
  • Comments generated: 2

Comment threadeng/build.ps1
Comment threadeng/build.ps1
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates top-level build entrypoints to ensure MSBuild crash diagnostics (MSBuild_*.failure.txt from MSB4166/node crashes) are consistently written under artifacts/log/ so CI reliably publishes them.

Changes:

  • Set and export MSBUILDDEBUGPATH early in eng/build.sh, create the directory, and print it for diagnostics.
  • Set and export MSBUILDDEBUGPATH early in eng/build.ps1, create the directory, print it, and update it per configuration during multi-config builds.
Show a summary per file
FileDescription
eng/build.shExports MSBUILDDEBUGPATH to a stable artifacts/log/<config>/MsbuildDebugLogs directory before invoking eng/common/build.sh.
eng/build.ps1Exports MSBUILDDEBUGPATH to artifacts/log/<config>/MsbuildDebugLogs, ensures the directory exists, and adjusts it when iterating configurations.

Copilot's findings

  • Files reviewed: 2/2 changed files
  • Comments generated: 1

Comment threadeng/build.ps1
Comment on lines +445 to +451
$titleCaseConfig = $((Get-Culture).TextInfo.ToTitleCase($config))

# Update MSBuild debug logs directory for this configuration.
$msbuildDebugLogsDir = "$PSScriptRoot/../artifacts/log/$titleCaseConfig/MsbuildDebugLogs"
New-Item -ItemType Directory -Force -Path $msbuildDebugLogsDir | Out-Null
$env:MSBUILDDEBUGPATH = $msbuildDebugLogsDir
Write-Host "MSBUILDDEBUGPATH=$msbuildDebugLogsDir"

CopilotAIApr 12, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the multi-configuration loop you update $env:MSBUILDDEBUGPATH per config, but MSBuild node reuse is enabled by default for non-CI runs (/nr:true via eng/common/tools.ps1 where nodeReuse defaults to !$ci). Reused nodes won’t see the updated environment variable, so crash diagnostics may still be written to the previous configuration’s directory, making the per-config directory update potentially misleading. Consider either keeping a single MSBUILDDEBUGPATH for the whole build.ps1 invocation, or disabling node reuse when switching configurations so each config reliably writes to its intended directory.

See below for a potential fix:

# Use a single MSBuild debug logs directory for the entire invocation because
# reused MSBuild nodes do not observe environment variable updates made between
# configurations.
$msbuildDebugLogsDir = "$PSScriptRoot/../artifacts/log/MsbuildDebugLogs"
New-Item -ItemType Directory -Force -Path $msbuildDebugLogsDir | Out-Null
$env:MSBUILDDEBUGPATH = $msbuildDebugLogsDir
Write-Host "MSBUILDDEBUGPATH=$msbuildDebugLogsDir"
foreach ($config in $configuration) {
$titleCaseConfig = $((Get-Culture).TextInfo.ToTitleCase($config))

Copilot uses AI. Check for mistakes.
Comment threadeng/build.sh
# Set up the directory for MSBuild debug logs, so that if MSBuild crashes (MSB4166)
# the failure.txt diagnostics are written to a known location under artifacts/log
# where they'll be captured as build artifacts.
MSBUILDDEBUGPATH="$scriptroot/../artifacts/log/${bootstrapConfig:-Debug}/MsbuildDebugLogs"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why does this use ${bootstrapConfig:-Debug} ? I would expect it to use actual config for non-bootstrap builds.

Comment threadeng/build.sh
done

# Set a different path for prebuilt usage tracking for the bootstrap build.
"$scriptroot/common/build.sh" ${bootstrapArguments[@]+"${bootstrapArguments[@]}"} /p:Subset=bootstrap /p:TrackPrebuiltUsageReportFile=$scriptroot/../artifacts/log/bootstrap-prebuilt-usage.xml -bl:$scriptroot/../artifacts/log/$bootstrapConfig/bootstrap.binlog

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be better to put this instrumentation into the common scripts like "$scriptroot/common/build.sh?

@jkotas

Copy link
Copy Markdown
Member

The existing pipeline variable in global-build-job.yml is left in place as defense-in-depth.

Defense-in-depth against what?

@danmoseley

Copy link
Copy Markdown
ContributorAuthor

Let's see whether we can do in arcade instead

Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

MSBuild crashing in the build

3 participants

@danmoseley@jkotas