Auto-upload test artifacts and build tags to Azure DevOps - #8295

Merged
Amaury Levé (Evangelink) merged 9 commits into
mainfrom
dev/amauryleve/azdo-artifact-upload
May 19, 2026
Merged

Auto-upload test artifacts and build tags to Azure DevOps#8295
Amaury Levé (Evangelink) merged 9 commits into
mainfrom
dev/amauryleve/azdo-artifact-upload

Conversation

@Evangelink

Copy link
Copy Markdown
Member

Part 3 of the brainstorm in #5951 — adds opt-in pipeline-level artifact upload + build tags to Microsoft.Testing.Extensions.AzureDevOpsReport. Pure ##vso logging commands — no Azure DevOps authentication required.

This is one of three PRs derived from the #5951 brainstorm. The other two:

  • Live test-result publishing
  • Flaky-history annotations + quarantine awareness

See comment for the broader plan.

Why

Today HangDump, CrashDump, CodeCoverage, and TRX all drop files into TestResultsDirectory, but nothing teaches Azure DevOps to publish them as build artifacts — users have to add a separate PublishBuildArtifacts@1 task. Similarly, build queries can't easily filter "runs that produced a hang/crash dump" without grepping logs.

What

Four new opt-in CLI options on Microsoft.Testing.Extensions.AzureDevOpsReport:

OptionTypePurpose
--report-azdo-upload-artifacts <mode>off (default) | tags-only | files | allEnable build tags, file upload, or both.
--report-azdo-upload-artifact-name <name>stringOverride the artifact container name. Default: TestResults_<assemblyName>_<tfm>.
--report-azdo-upload-artifact-include <glob>repeatableFile globs (relative to TestResultsDirectory) to include. Default: **/*.
--report-azdo-upload-artifact-exclude <glob>repeatableExclusion globs. Default: none.

At session end (when TF_BUILD=true and the mode is not off), the new AzureDevOpsArtifactUploader:

  • Emits ##vso[build.addbuildtag]has-crashdump and/or has-hangdump based on the producer UID of consumed TestNodeFileArtifact events (CrashDumpProcessLifetimeHandler / HangDumpProcessLifetimeHandler — robust to localization).
  • Emits ##vso[build.addbuildtag]has-test-failures if the session contained any failed test.
  • Enumerates TestResultsDirectory via IFileSystem, applies include/exclude globs (case-sensitive on Linux, case-insensitive on Windows), and emits ##vso[artifact.upload containerfolder=…;artifactname=…]<path> for each match — properly escaping ;, \r, \n via the existing AzDoEscaper.

All output goes through IOutputDevice (no direct Console.WriteLine). All env/path/file access goes through IEnvironment / IConfiguration / IFileSystem. Exception isolation: callbacks never propagate non-cancellation exceptions out of the lifetime handler.

Highlights from the expert-reviewer round

The implementation went through one round of expert-reviewer. Critical/major issues addressed:

  • Failure isolation: all callbacks now try/catch everything except OperationCanceledException; warnings logged via ILogger.
  • Validator: rejects --report-azdo-upload-artifact-{include,exclude,name} when --report-azdo-upload-artifacts is absent or off.
  • Race-free flags: monotonic crash/hang/failure flags use Interlocked.Exchange + Volatile.Read.
  • Cross-platform paths: new PathComparison helper (OrdinalIgnoreCase on Windows, Ordinal elsewhere) applied to glob construction, prefix matching, and sort.
  • Dump detection: producer-UID-based only — no longer matches on localized DisplayName or path substring.
  • Out-of-dir filter: files resolved outside TestResultsDirectory (e.g. via symlinks) are skipped.
  • Lazy TFM resolution via new shared TargetFrameworkMonikerHelper (also adopted by the existing AzureDevOpsReporter).
  • Pre-existing typo EanbleEnable fixed in Resources/AzureDevOpsResources.resx.

Tests

  • 12 new unit tests in test/UnitTests/Microsoft.Testing.Extensions.UnitTests/AzureDevOpsArtifactUploaderTests.cs covering mode wiring (incl. off no-op), tag emission (exactly once, even under concurrent message events), glob include/exclude, escaping, missing TF_BUILD, null/whitespace TestResultsDirectory, out-of-dir filtering, and validator rejection.
  • HelpInfoAllExtensionsTests expectations updated for new options (both --help and --info blocks, alphabetical order preserved).

Build status (local)

  • .\.dotnet\dotnet.exe build src\Platform\Microsoft.Testing.Extensions.AzureDevOpsReport\Microsoft.Testing.Extensions.AzureDevOpsReport.csproj -c Debug0 warnings, 0 errors.
  • .\.dotnet\dotnet.exe test test\UnitTests\Microsoft.Testing.Extensions.UnitTests\Microsoft.Testing.Extensions.UnitTests.csproj566/566 passed.
  • .\build.cmd -pack0 warnings, 0 errors.

Out of scope (deliberate)

  • Result-level attachments (attach a dump to its failing test in the AzDO Tests tab) — depends on the REST client introduced by the live-publishing PR.
  • Code-coverage attachment under the Coverage tab — same reason.

Both can land as small follow-ups once that PR is merged.

Checklist

  • Critical & Major review findings addressed
  • Localized via resx + xlf (regenerated with /t:UpdateXlf, not hand-edited)
  • Help/info acceptance test expectations updated
  • No new public API
  • .\build.cmd green (0 warnings, 0 errors)
  • PR feedback addressed

Refs #5951

Add opt-in Azure DevOps artifact upload support, help/info updates, and unit coverage for the new reporter behavior.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
CopilotAI review requested due to automatic review settings May 16, 2026 16:33

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds an opt-in pipeline-level artifact upload + build-tag feature to Microsoft.Testing.Extensions.AzureDevOpsReport. It introduces four new CLI options that, when running under Azure DevOps (TF_BUILD=true), emit ##vso[artifact.upload …] and ##vso[build.addbuildtag]… logging commands at session end based on consumed FileArtifact/TestNodeUpdateMessage events and a glob filter over the test results directory.

Changes:

  • New AzureDevOpsArtifactUploader (data consumer + session lifetime handler) plus supporting PathComparison / TargetFrameworkMonikerHelper helpers and AzureDevOpsArtifactUploadMode enum.
  • New CLI options (--report-azdo-upload-artifacts, --report-azdo-upload-artifact-{name,include,exclude}) with validator that rejects absolute globs, unknown modes, and artifact options when uploads are off; resx/xlf strings added; pre-existing Eanble→Enable typo fixed.
  • Help/info acceptance expectations updated and 12 new unit tests covering modes, tag emission, escaping, globbing, missing TF_BUILD, and validator rejection.
Show a summary per file
FileDescription
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsArtifactUploader.csNew uploader implementing the artifact-upload + build-tag logic.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsCommandLineOptions.csNew option/mode constants.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsCommandLineProvider.csRegisters and validates the new options.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsExtensions.csRegisters the uploader as data consumer + session lifetime handler.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsReporter.csSwitched TFM resolution to shared helper.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/PathComparison.csOS-aware path string comparer/comparison.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/TargetFrameworkMonikerHelper.csShared TFM resolution helper.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Microsoft.Testing.Extensions.AzureDevOpsReport.csprojAdds Microsoft.Extensions.FileSystemGlobbing reference.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/AzureDevOpsResources.resxNew strings + Eanble→Enable typo fix.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/*.xlfRegenerated translations for all locales.
test/UnitTests/Microsoft.Testing.Extensions.UnitTests/AzureDevOpsArtifactUploaderTests.cs12 new unit tests for uploader + validator.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/HelpInfoAllExtensionsTests.csAdds AzDO extension + new option entries to help/info expectations.

Copilot's findings

  • Files reviewed: 24/24 changed files
  • Comments generated: 1

@Evangelink

Copy link
Copy Markdown
MemberAuthor

/review

@github-actions

github-actionsBot commented May 17, 2026

Copy link
Copy Markdown
Contributor

Expert Code Review (command) completed successfully!

Expert-reviewer launched in background for PR #8295. It will post the review directly.

@Evangelink
Amaury Levé (Evangelink) marked this pull request as ready for review May 17, 2026 19:13
CopilotAI review requested due to automatic review settings May 18, 2026 07:13
@Evangelink

Copy link
Copy Markdown
MemberAuthor

Copilot address review comments

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot's findings

  • Files reviewed: 24/24 changed files
  • Comments generated: 0 new

Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com>

CopilotAI commented May 18, 2026

Copy link
Copy Markdown
Contributor

Copilot address review comments

Addressed in 79f309a. I updated AzureDevOpsArtifactUploaderTests to use OS-agnostic paths (Path.Combine-based helpers) and adjusted assertions/globs accordingly so the files-mode tests pass cross-platform.

Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com>
CopilotAI review requested due to automatic review settings May 18, 2026 07:21
@Evangelink
Amaury Levé (Evangelink) removed the request for review from CopilotMay 18, 2026 07:21
Resolve the AzureDevOpsReporter merge conflict and fix net462-compatible escaping in AzureDevOpsArtifactUploaderTests.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
CopilotAI review requested due to automatic review settings May 18, 2026 19:08

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot's findings

  • Files reviewed: 24/24 changed files
  • Comments generated: 0 new

@Evangelink

Copy link
Copy Markdown
MemberAuthor

Copilot resolve the merge conflicts in this pull request

…rtifact-upload
# Conflicts:
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsCommandLineProvider.cs
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsExtensions.cs
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsReporter.cs
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Microsoft.Testing.Extensions.AzureDevOpsReport.csproj
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/AzureDevOpsResources.resx
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.cs.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.de.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.es.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.fr.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.it.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.ja.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.ko.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.pl.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.pt-BR.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.ru.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.tr.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.zh-Hans.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.zh-Hant.xlf
#	test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/HelpInfoAllExtensionsTests.cs
Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com>

CopilotAI commented May 19, 2026

Copy link
Copy Markdown
Contributor

Copilot resolve the merge conflicts in this pull request

Merge conflicts are resolved in commit bb8d863 by merging origin/main and reconciling the Azure DevOps report, resources, and help-info expectations.

CopilotAI review requested due to automatic review settings May 19, 2026 12:42

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot's findings

  • Files reviewed: 11/11 changed files
  • Comments generated: 0 new

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@Evangelink

Copy link
Copy Markdown
MemberAuthor

Addressed in b5bd216:

  • Cross-platform paths in tests (the only inline review comment from Copilot): already fixed in earlier 79f309a / bbe254cAzureDevOpsArtifactUploaderTests uses Path.Combine-based InResults/OutsideResults helpers rooted at Path.GetTempPath(). Verified.
  • Merge fallout from ff7261f05 (broken state on origin):
    • AzureDevOpsExtensions.cs: restored using Microsoft.Testing.Platform.Extensions; (required for CompositeExtensionFactory<>).
    • AzureDevOpsReporter.cs: removed leftover dead GetTargetFrameworkMoniker method (call site now uses TargetFrameworkMonikerHelper).
    • Resources/xlf/AzureDevOpsResources.*.xlf: regenerated via /t:UpdateXlf so all 14 locales have the 7 new state="new" trans-units that the merge dropped.

Verification:

  • .\build.cmd — Build succeeded, 0 Warning(s), 0 Error(s).
  • All 23 AzureDevOpsArtifactUploaderTests pass (full extensions unit-test suite: 174 passed, 2 skipped, 0 failed).

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@Evangelink
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

Auto-upload test artifacts and build tags to Azure DevOps - #8295

Merged
Amaury Levé (Evangelink) merged 9 commits into
mainfrom
dev/amauryleve/azdo-artifact-upload
May 19, 2026
Merged

Auto-upload test artifacts and build tags to Azure DevOps#8295
Amaury Levé (Evangelink) merged 9 commits into
mainfrom
dev/amauryleve/azdo-artifact-upload

Conversation

@Evangelink

Copy link
Copy Markdown
Member

Part 3 of the brainstorm in #5951 — adds opt-in pipeline-level artifact upload + build tags to Microsoft.Testing.Extensions.AzureDevOpsReport. Pure ##vso logging commands — no Azure DevOps authentication required.

This is one of three PRs derived from the #5951 brainstorm. The other two:

  • Live test-result publishing
  • Flaky-history annotations + quarantine awareness

See comment for the broader plan.

Why

Today HangDump, CrashDump, CodeCoverage, and TRX all drop files into TestResultsDirectory, but nothing teaches Azure DevOps to publish them as build artifacts — users have to add a separate PublishBuildArtifacts@1 task. Similarly, build queries can't easily filter "runs that produced a hang/crash dump" without grepping logs.

What

Four new opt-in CLI options on Microsoft.Testing.Extensions.AzureDevOpsReport:

OptionTypePurpose
--report-azdo-upload-artifacts <mode>off (default) | tags-only | files | allEnable build tags, file upload, or both.
--report-azdo-upload-artifact-name <name>stringOverride the artifact container name. Default: TestResults_<assemblyName>_<tfm>.
--report-azdo-upload-artifact-include <glob>repeatableFile globs (relative to TestResultsDirectory) to include. Default: **/*.
--report-azdo-upload-artifact-exclude <glob>repeatableExclusion globs. Default: none.

At session end (when TF_BUILD=true and the mode is not off), the new AzureDevOpsArtifactUploader:

  • Emits ##vso[build.addbuildtag]has-crashdump and/or has-hangdump based on the producer UID of consumed TestNodeFileArtifact events (CrashDumpProcessLifetimeHandler / HangDumpProcessLifetimeHandler — robust to localization).
  • Emits ##vso[build.addbuildtag]has-test-failures if the session contained any failed test.
  • Enumerates TestResultsDirectory via IFileSystem, applies include/exclude globs (case-sensitive on Linux, case-insensitive on Windows), and emits ##vso[artifact.upload containerfolder=…;artifactname=…]<path> for each match — properly escaping ;, \r, \n via the existing AzDoEscaper.

All output goes through IOutputDevice (no direct Console.WriteLine). All env/path/file access goes through IEnvironment / IConfiguration / IFileSystem. Exception isolation: callbacks never propagate non-cancellation exceptions out of the lifetime handler.

Highlights from the expert-reviewer round

The implementation went through one round of expert-reviewer. Critical/major issues addressed:

  • Failure isolation: all callbacks now try/catch everything except OperationCanceledException; warnings logged via ILogger.
  • Validator: rejects --report-azdo-upload-artifact-{include,exclude,name} when --report-azdo-upload-artifacts is absent or off.
  • Race-free flags: monotonic crash/hang/failure flags use Interlocked.Exchange + Volatile.Read.
  • Cross-platform paths: new PathComparison helper (OrdinalIgnoreCase on Windows, Ordinal elsewhere) applied to glob construction, prefix matching, and sort.
  • Dump detection: producer-UID-based only — no longer matches on localized DisplayName or path substring.
  • Out-of-dir filter: files resolved outside TestResultsDirectory (e.g. via symlinks) are skipped.
  • Lazy TFM resolution via new shared TargetFrameworkMonikerHelper (also adopted by the existing AzureDevOpsReporter).
  • Pre-existing typo EanbleEnable fixed in Resources/AzureDevOpsResources.resx.

Tests

  • 12 new unit tests in test/UnitTests/Microsoft.Testing.Extensions.UnitTests/AzureDevOpsArtifactUploaderTests.cs covering mode wiring (incl. off no-op), tag emission (exactly once, even under concurrent message events), glob include/exclude, escaping, missing TF_BUILD, null/whitespace TestResultsDirectory, out-of-dir filtering, and validator rejection.
  • HelpInfoAllExtensionsTests expectations updated for new options (both --help and --info blocks, alphabetical order preserved).

Build status (local)

  • .\.dotnet\dotnet.exe build src\Platform\Microsoft.Testing.Extensions.AzureDevOpsReport\Microsoft.Testing.Extensions.AzureDevOpsReport.csproj -c Debug0 warnings, 0 errors.
  • .\.dotnet\dotnet.exe test test\UnitTests\Microsoft.Testing.Extensions.UnitTests\Microsoft.Testing.Extensions.UnitTests.csproj566/566 passed.
  • .\build.cmd -pack0 warnings, 0 errors.

Out of scope (deliberate)

  • Result-level attachments (attach a dump to its failing test in the AzDO Tests tab) — depends on the REST client introduced by the live-publishing PR.
  • Code-coverage attachment under the Coverage tab — same reason.

Both can land as small follow-ups once that PR is merged.

Checklist

  • Critical & Major review findings addressed
  • Localized via resx + xlf (regenerated with /t:UpdateXlf, not hand-edited)
  • Help/info acceptance test expectations updated
  • No new public API
  • .\build.cmd green (0 warnings, 0 errors)
  • PR feedback addressed

Refs #5951

Add opt-in Azure DevOps artifact upload support, help/info updates, and unit coverage for the new reporter behavior.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
CopilotAI review requested due to automatic review settings May 16, 2026 16:33

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds an opt-in pipeline-level artifact upload + build-tag feature to Microsoft.Testing.Extensions.AzureDevOpsReport. It introduces four new CLI options that, when running under Azure DevOps (TF_BUILD=true), emit ##vso[artifact.upload …] and ##vso[build.addbuildtag]… logging commands at session end based on consumed FileArtifact/TestNodeUpdateMessage events and a glob filter over the test results directory.

Changes:

  • New AzureDevOpsArtifactUploader (data consumer + session lifetime handler) plus supporting PathComparison / TargetFrameworkMonikerHelper helpers and AzureDevOpsArtifactUploadMode enum.
  • New CLI options (--report-azdo-upload-artifacts, --report-azdo-upload-artifact-{name,include,exclude}) with validator that rejects absolute globs, unknown modes, and artifact options when uploads are off; resx/xlf strings added; pre-existing Eanble→Enable typo fixed.
  • Help/info acceptance expectations updated and 12 new unit tests covering modes, tag emission, escaping, globbing, missing TF_BUILD, and validator rejection.
Show a summary per file
FileDescription
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsArtifactUploader.csNew uploader implementing the artifact-upload + build-tag logic.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsCommandLineOptions.csNew option/mode constants.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsCommandLineProvider.csRegisters and validates the new options.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsExtensions.csRegisters the uploader as data consumer + session lifetime handler.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsReporter.csSwitched TFM resolution to shared helper.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/PathComparison.csOS-aware path string comparer/comparison.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/TargetFrameworkMonikerHelper.csShared TFM resolution helper.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Microsoft.Testing.Extensions.AzureDevOpsReport.csprojAdds Microsoft.Extensions.FileSystemGlobbing reference.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/AzureDevOpsResources.resxNew strings + Eanble→Enable typo fix.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/*.xlfRegenerated translations for all locales.
test/UnitTests/Microsoft.Testing.Extensions.UnitTests/AzureDevOpsArtifactUploaderTests.cs12 new unit tests for uploader + validator.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/HelpInfoAllExtensionsTests.csAdds AzDO extension + new option entries to help/info expectations.

Copilot's findings

  • Files reviewed: 24/24 changed files
  • Comments generated: 1

@Evangelink

Copy link
Copy Markdown
MemberAuthor

/review

@github-actions

github-actionsBot commented May 17, 2026

Copy link
Copy Markdown
Contributor

Expert Code Review (command) completed successfully!

Expert-reviewer launched in background for PR #8295. It will post the review directly.

@Evangelink
Amaury Levé (Evangelink) marked this pull request as ready for review May 17, 2026 19:13
CopilotAI review requested due to automatic review settings May 18, 2026 07:13
@Evangelink

Copy link
Copy Markdown
MemberAuthor

Copilot address review comments

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot's findings

  • Files reviewed: 24/24 changed files
  • Comments generated: 0 new

Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com>

CopilotAI commented May 18, 2026

Copy link
Copy Markdown
Contributor

Copilot address review comments

Addressed in 79f309a. I updated AzureDevOpsArtifactUploaderTests to use OS-agnostic paths (Path.Combine-based helpers) and adjusted assertions/globs accordingly so the files-mode tests pass cross-platform.

Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com>
CopilotAI review requested due to automatic review settings May 18, 2026 07:21
@Evangelink
Amaury Levé (Evangelink) removed the request for review from CopilotMay 18, 2026 07:21
Resolve the AzureDevOpsReporter merge conflict and fix net462-compatible escaping in AzureDevOpsArtifactUploaderTests.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
CopilotAI review requested due to automatic review settings May 18, 2026 19:08

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot's findings

  • Files reviewed: 24/24 changed files
  • Comments generated: 0 new

@Evangelink

Copy link
Copy Markdown
MemberAuthor

Copilot resolve the merge conflicts in this pull request

…rtifact-upload
# Conflicts:
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsCommandLineProvider.cs
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsExtensions.cs
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsReporter.cs
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Microsoft.Testing.Extensions.AzureDevOpsReport.csproj
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/AzureDevOpsResources.resx
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.cs.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.de.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.es.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.fr.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.it.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.ja.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.ko.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.pl.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.pt-BR.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.ru.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.tr.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.zh-Hans.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.zh-Hant.xlf
#	test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/HelpInfoAllExtensionsTests.cs
Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com>

CopilotAI commented May 19, 2026

Copy link
Copy Markdown
Contributor

Copilot resolve the merge conflicts in this pull request

Merge conflicts are resolved in commit bb8d863 by merging origin/main and reconciling the Azure DevOps report, resources, and help-info expectations.

CopilotAI review requested due to automatic review settings May 19, 2026 12:42

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot's findings

  • Files reviewed: 11/11 changed files
  • Comments generated: 0 new

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@Evangelink

Copy link
Copy Markdown
MemberAuthor

Addressed in b5bd216:

  • Cross-platform paths in tests (the only inline review comment from Copilot): already fixed in earlier 79f309a / bbe254cAzureDevOpsArtifactUploaderTests uses Path.Combine-based InResults/OutsideResults helpers rooted at Path.GetTempPath(). Verified.
  • Merge fallout from ff7261f05 (broken state on origin):
    • AzureDevOpsExtensions.cs: restored using Microsoft.Testing.Platform.Extensions; (required for CompositeExtensionFactory<>).
    • AzureDevOpsReporter.cs: removed leftover dead GetTargetFrameworkMoniker method (call site now uses TargetFrameworkMonikerHelper).
    • Resources/xlf/AzureDevOpsResources.*.xlf: regenerated via /t:UpdateXlf so all 14 locales have the 7 new state="new" trans-units that the merge dropped.

Verification:

  • .\build.cmd — Build succeeded, 0 Warning(s), 0 Error(s).
  • All 23 AzureDevOpsArtifactUploaderTests pass (full extensions unit-test suite: 174 passed, 2 skipped, 0 failed).

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@Evangelink
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Auto-upload test artifacts and build tags to Azure DevOps - #8295

Merged
Amaury Levé (Evangelink) merged 9 commits into
mainfrom
dev/amauryleve/azdo-artifact-upload
May 19, 2026
Merged

Auto-upload test artifacts and build tags to Azure DevOps#8295
Amaury Levé (Evangelink) merged 9 commits into
mainfrom
dev/amauryleve/azdo-artifact-upload

Conversation

@Evangelink

Copy link
Copy Markdown
Member

Part 3 of the brainstorm in #5951 — adds opt-in pipeline-level artifact upload + build tags to Microsoft.Testing.Extensions.AzureDevOpsReport. Pure ##vso logging commands — no Azure DevOps authentication required.

This is one of three PRs derived from the #5951 brainstorm. The other two:

  • Live test-result publishing
  • Flaky-history annotations + quarantine awareness

See comment for the broader plan.

Why

Today HangDump, CrashDump, CodeCoverage, and TRX all drop files into TestResultsDirectory, but nothing teaches Azure DevOps to publish them as build artifacts — users have to add a separate PublishBuildArtifacts@1 task. Similarly, build queries can't easily filter "runs that produced a hang/crash dump" without grepping logs.

What

Four new opt-in CLI options on Microsoft.Testing.Extensions.AzureDevOpsReport:

OptionTypePurpose
--report-azdo-upload-artifacts <mode>off (default) | tags-only | files | allEnable build tags, file upload, or both.
--report-azdo-upload-artifact-name <name>stringOverride the artifact container name. Default: TestResults_<assemblyName>_<tfm>.
--report-azdo-upload-artifact-include <glob>repeatableFile globs (relative to TestResultsDirectory) to include. Default: **/*.
--report-azdo-upload-artifact-exclude <glob>repeatableExclusion globs. Default: none.

At session end (when TF_BUILD=true and the mode is not off), the new AzureDevOpsArtifactUploader:

  • Emits ##vso[build.addbuildtag]has-crashdump and/or has-hangdump based on the producer UID of consumed TestNodeFileArtifact events (CrashDumpProcessLifetimeHandler / HangDumpProcessLifetimeHandler — robust to localization).
  • Emits ##vso[build.addbuildtag]has-test-failures if the session contained any failed test.
  • Enumerates TestResultsDirectory via IFileSystem, applies include/exclude globs (case-sensitive on Linux, case-insensitive on Windows), and emits ##vso[artifact.upload containerfolder=…;artifactname=…]<path> for each match — properly escaping ;, \r, \n via the existing AzDoEscaper.

All output goes through IOutputDevice (no direct Console.WriteLine). All env/path/file access goes through IEnvironment / IConfiguration / IFileSystem. Exception isolation: callbacks never propagate non-cancellation exceptions out of the lifetime handler.

Highlights from the expert-reviewer round

The implementation went through one round of expert-reviewer. Critical/major issues addressed:

  • Failure isolation: all callbacks now try/catch everything except OperationCanceledException; warnings logged via ILogger.
  • Validator: rejects --report-azdo-upload-artifact-{include,exclude,name} when --report-azdo-upload-artifacts is absent or off.
  • Race-free flags: monotonic crash/hang/failure flags use Interlocked.Exchange + Volatile.Read.
  • Cross-platform paths: new PathComparison helper (OrdinalIgnoreCase on Windows, Ordinal elsewhere) applied to glob construction, prefix matching, and sort.
  • Dump detection: producer-UID-based only — no longer matches on localized DisplayName or path substring.
  • Out-of-dir filter: files resolved outside TestResultsDirectory (e.g. via symlinks) are skipped.
  • Lazy TFM resolution via new shared TargetFrameworkMonikerHelper (also adopted by the existing AzureDevOpsReporter).
  • Pre-existing typo EanbleEnable fixed in Resources/AzureDevOpsResources.resx.

Tests

  • 12 new unit tests in test/UnitTests/Microsoft.Testing.Extensions.UnitTests/AzureDevOpsArtifactUploaderTests.cs covering mode wiring (incl. off no-op), tag emission (exactly once, even under concurrent message events), glob include/exclude, escaping, missing TF_BUILD, null/whitespace TestResultsDirectory, out-of-dir filtering, and validator rejection.
  • HelpInfoAllExtensionsTests expectations updated for new options (both --help and --info blocks, alphabetical order preserved).

Build status (local)

  • .\.dotnet\dotnet.exe build src\Platform\Microsoft.Testing.Extensions.AzureDevOpsReport\Microsoft.Testing.Extensions.AzureDevOpsReport.csproj -c Debug0 warnings, 0 errors.
  • .\.dotnet\dotnet.exe test test\UnitTests\Microsoft.Testing.Extensions.UnitTests\Microsoft.Testing.Extensions.UnitTests.csproj566/566 passed.
  • .\build.cmd -pack0 warnings, 0 errors.

Out of scope (deliberate)

  • Result-level attachments (attach a dump to its failing test in the AzDO Tests tab) — depends on the REST client introduced by the live-publishing PR.
  • Code-coverage attachment under the Coverage tab — same reason.

Both can land as small follow-ups once that PR is merged.

Checklist

  • Critical & Major review findings addressed
  • Localized via resx + xlf (regenerated with /t:UpdateXlf, not hand-edited)
  • Help/info acceptance test expectations updated
  • No new public API
  • .\build.cmd green (0 warnings, 0 errors)
  • PR feedback addressed

Refs #5951

Add opt-in Azure DevOps artifact upload support, help/info updates, and unit coverage for the new reporter behavior.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
CopilotAI review requested due to automatic review settings May 16, 2026 16:33

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds an opt-in pipeline-level artifact upload + build-tag feature to Microsoft.Testing.Extensions.AzureDevOpsReport. It introduces four new CLI options that, when running under Azure DevOps (TF_BUILD=true), emit ##vso[artifact.upload …] and ##vso[build.addbuildtag]… logging commands at session end based on consumed FileArtifact/TestNodeUpdateMessage events and a glob filter over the test results directory.

Changes:

  • New AzureDevOpsArtifactUploader (data consumer + session lifetime handler) plus supporting PathComparison / TargetFrameworkMonikerHelper helpers and AzureDevOpsArtifactUploadMode enum.
  • New CLI options (--report-azdo-upload-artifacts, --report-azdo-upload-artifact-{name,include,exclude}) with validator that rejects absolute globs, unknown modes, and artifact options when uploads are off; resx/xlf strings added; pre-existing Eanble→Enable typo fixed.
  • Help/info acceptance expectations updated and 12 new unit tests covering modes, tag emission, escaping, globbing, missing TF_BUILD, and validator rejection.
Show a summary per file
FileDescription
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsArtifactUploader.csNew uploader implementing the artifact-upload + build-tag logic.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsCommandLineOptions.csNew option/mode constants.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsCommandLineProvider.csRegisters and validates the new options.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsExtensions.csRegisters the uploader as data consumer + session lifetime handler.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsReporter.csSwitched TFM resolution to shared helper.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/PathComparison.csOS-aware path string comparer/comparison.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/TargetFrameworkMonikerHelper.csShared TFM resolution helper.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Microsoft.Testing.Extensions.AzureDevOpsReport.csprojAdds Microsoft.Extensions.FileSystemGlobbing reference.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/AzureDevOpsResources.resxNew strings + Eanble→Enable typo fix.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/*.xlfRegenerated translations for all locales.
test/UnitTests/Microsoft.Testing.Extensions.UnitTests/AzureDevOpsArtifactUploaderTests.cs12 new unit tests for uploader + validator.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/HelpInfoAllExtensionsTests.csAdds AzDO extension + new option entries to help/info expectations.

Copilot's findings

  • Files reviewed: 24/24 changed files
  • Comments generated: 1

@Evangelink

Copy link
Copy Markdown
MemberAuthor

/review

@github-actions

github-actionsBot commented May 17, 2026

Copy link
Copy Markdown
Contributor

Expert Code Review (command) completed successfully!

Expert-reviewer launched in background for PR #8295. It will post the review directly.

@Evangelink
Amaury Levé (Evangelink) marked this pull request as ready for review May 17, 2026 19:13
CopilotAI review requested due to automatic review settings May 18, 2026 07:13
@Evangelink

Copy link
Copy Markdown
MemberAuthor

Copilot address review comments

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot's findings

  • Files reviewed: 24/24 changed files
  • Comments generated: 0 new

Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com>

CopilotAI commented May 18, 2026

Copy link
Copy Markdown
Contributor

Copilot address review comments

Addressed in 79f309a. I updated AzureDevOpsArtifactUploaderTests to use OS-agnostic paths (Path.Combine-based helpers) and adjusted assertions/globs accordingly so the files-mode tests pass cross-platform.

Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com>
CopilotAI review requested due to automatic review settings May 18, 2026 07:21
@Evangelink
Amaury Levé (Evangelink) removed the request for review from CopilotMay 18, 2026 07:21
Resolve the AzureDevOpsReporter merge conflict and fix net462-compatible escaping in AzureDevOpsArtifactUploaderTests.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
CopilotAI review requested due to automatic review settings May 18, 2026 19:08

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot's findings

  • Files reviewed: 24/24 changed files
  • Comments generated: 0 new

@Evangelink

Copy link
Copy Markdown
MemberAuthor

Copilot resolve the merge conflicts in this pull request

…rtifact-upload
# Conflicts:
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsCommandLineProvider.cs
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsExtensions.cs
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsReporter.cs
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Microsoft.Testing.Extensions.AzureDevOpsReport.csproj
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/AzureDevOpsResources.resx
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.cs.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.de.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.es.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.fr.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.it.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.ja.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.ko.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.pl.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.pt-BR.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.ru.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.tr.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.zh-Hans.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.zh-Hant.xlf
#	test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/HelpInfoAllExtensionsTests.cs
Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com>

CopilotAI commented May 19, 2026

Copy link
Copy Markdown
Contributor

Copilot resolve the merge conflicts in this pull request

Merge conflicts are resolved in commit bb8d863 by merging origin/main and reconciling the Azure DevOps report, resources, and help-info expectations.

CopilotAI review requested due to automatic review settings May 19, 2026 12:42

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot's findings

  • Files reviewed: 11/11 changed files
  • Comments generated: 0 new

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@Evangelink

Copy link
Copy Markdown
MemberAuthor

Addressed in b5bd216:

  • Cross-platform paths in tests (the only inline review comment from Copilot): already fixed in earlier 79f309a / bbe254cAzureDevOpsArtifactUploaderTests uses Path.Combine-based InResults/OutsideResults helpers rooted at Path.GetTempPath(). Verified.
  • Merge fallout from ff7261f05 (broken state on origin):
    • AzureDevOpsExtensions.cs: restored using Microsoft.Testing.Platform.Extensions; (required for CompositeExtensionFactory<>).
    • AzureDevOpsReporter.cs: removed leftover dead GetTargetFrameworkMoniker method (call site now uses TargetFrameworkMonikerHelper).
    • Resources/xlf/AzureDevOpsResources.*.xlf: regenerated via /t:UpdateXlf so all 14 locales have the 7 new state="new" trans-units that the merge dropped.

Verification:

  • .\build.cmd — Build succeeded, 0 Warning(s), 0 Error(s).
  • All 23 AzureDevOpsArtifactUploaderTests pass (full extensions unit-test suite: 174 passed, 2 skipped, 0 failed).

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@Evangelink
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Auto-upload test artifacts and build tags to Azure DevOps - #8295

Merged
Amaury Levé (Evangelink) merged 9 commits into
mainfrom
dev/amauryleve/azdo-artifact-upload
May 19, 2026
Merged

Auto-upload test artifacts and build tags to Azure DevOps#8295
Amaury Levé (Evangelink) merged 9 commits into
mainfrom
dev/amauryleve/azdo-artifact-upload

Conversation

@Evangelink

Copy link
Copy Markdown
Member

Part 3 of the brainstorm in #5951 — adds opt-in pipeline-level artifact upload + build tags to Microsoft.Testing.Extensions.AzureDevOpsReport. Pure ##vso logging commands — no Azure DevOps authentication required.

This is one of three PRs derived from the #5951 brainstorm. The other two:

  • Live test-result publishing
  • Flaky-history annotations + quarantine awareness

See comment for the broader plan.

Why

Today HangDump, CrashDump, CodeCoverage, and TRX all drop files into TestResultsDirectory, but nothing teaches Azure DevOps to publish them as build artifacts — users have to add a separate PublishBuildArtifacts@1 task. Similarly, build queries can't easily filter "runs that produced a hang/crash dump" without grepping logs.

What

Four new opt-in CLI options on Microsoft.Testing.Extensions.AzureDevOpsReport:

OptionTypePurpose
--report-azdo-upload-artifacts <mode>off (default) | tags-only | files | allEnable build tags, file upload, or both.
--report-azdo-upload-artifact-name <name>stringOverride the artifact container name. Default: TestResults_<assemblyName>_<tfm>.
--report-azdo-upload-artifact-include <glob>repeatableFile globs (relative to TestResultsDirectory) to include. Default: **/*.
--report-azdo-upload-artifact-exclude <glob>repeatableExclusion globs. Default: none.

At session end (when TF_BUILD=true and the mode is not off), the new AzureDevOpsArtifactUploader:

  • Emits ##vso[build.addbuildtag]has-crashdump and/or has-hangdump based on the producer UID of consumed TestNodeFileArtifact events (CrashDumpProcessLifetimeHandler / HangDumpProcessLifetimeHandler — robust to localization).
  • Emits ##vso[build.addbuildtag]has-test-failures if the session contained any failed test.
  • Enumerates TestResultsDirectory via IFileSystem, applies include/exclude globs (case-sensitive on Linux, case-insensitive on Windows), and emits ##vso[artifact.upload containerfolder=…;artifactname=…]<path> for each match — properly escaping ;, \r, \n via the existing AzDoEscaper.

All output goes through IOutputDevice (no direct Console.WriteLine). All env/path/file access goes through IEnvironment / IConfiguration / IFileSystem. Exception isolation: callbacks never propagate non-cancellation exceptions out of the lifetime handler.

Highlights from the expert-reviewer round

The implementation went through one round of expert-reviewer. Critical/major issues addressed:

  • Failure isolation: all callbacks now try/catch everything except OperationCanceledException; warnings logged via ILogger.
  • Validator: rejects --report-azdo-upload-artifact-{include,exclude,name} when --report-azdo-upload-artifacts is absent or off.
  • Race-free flags: monotonic crash/hang/failure flags use Interlocked.Exchange + Volatile.Read.
  • Cross-platform paths: new PathComparison helper (OrdinalIgnoreCase on Windows, Ordinal elsewhere) applied to glob construction, prefix matching, and sort.
  • Dump detection: producer-UID-based only — no longer matches on localized DisplayName or path substring.
  • Out-of-dir filter: files resolved outside TestResultsDirectory (e.g. via symlinks) are skipped.
  • Lazy TFM resolution via new shared TargetFrameworkMonikerHelper (also adopted by the existing AzureDevOpsReporter).
  • Pre-existing typo EanbleEnable fixed in Resources/AzureDevOpsResources.resx.

Tests

  • 12 new unit tests in test/UnitTests/Microsoft.Testing.Extensions.UnitTests/AzureDevOpsArtifactUploaderTests.cs covering mode wiring (incl. off no-op), tag emission (exactly once, even under concurrent message events), glob include/exclude, escaping, missing TF_BUILD, null/whitespace TestResultsDirectory, out-of-dir filtering, and validator rejection.
  • HelpInfoAllExtensionsTests expectations updated for new options (both --help and --info blocks, alphabetical order preserved).

Build status (local)

  • .\.dotnet\dotnet.exe build src\Platform\Microsoft.Testing.Extensions.AzureDevOpsReport\Microsoft.Testing.Extensions.AzureDevOpsReport.csproj -c Debug0 warnings, 0 errors.
  • .\.dotnet\dotnet.exe test test\UnitTests\Microsoft.Testing.Extensions.UnitTests\Microsoft.Testing.Extensions.UnitTests.csproj566/566 passed.
  • .\build.cmd -pack0 warnings, 0 errors.

Out of scope (deliberate)

  • Result-level attachments (attach a dump to its failing test in the AzDO Tests tab) — depends on the REST client introduced by the live-publishing PR.
  • Code-coverage attachment under the Coverage tab — same reason.

Both can land as small follow-ups once that PR is merged.

Checklist

  • Critical & Major review findings addressed
  • Localized via resx + xlf (regenerated with /t:UpdateXlf, not hand-edited)
  • Help/info acceptance test expectations updated
  • No new public API
  • .\build.cmd green (0 warnings, 0 errors)
  • PR feedback addressed

Refs #5951

Add opt-in Azure DevOps artifact upload support, help/info updates, and unit coverage for the new reporter behavior.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
CopilotAI review requested due to automatic review settings May 16, 2026 16:33

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds an opt-in pipeline-level artifact upload + build-tag feature to Microsoft.Testing.Extensions.AzureDevOpsReport. It introduces four new CLI options that, when running under Azure DevOps (TF_BUILD=true), emit ##vso[artifact.upload …] and ##vso[build.addbuildtag]… logging commands at session end based on consumed FileArtifact/TestNodeUpdateMessage events and a glob filter over the test results directory.

Changes:

  • New AzureDevOpsArtifactUploader (data consumer + session lifetime handler) plus supporting PathComparison / TargetFrameworkMonikerHelper helpers and AzureDevOpsArtifactUploadMode enum.
  • New CLI options (--report-azdo-upload-artifacts, --report-azdo-upload-artifact-{name,include,exclude}) with validator that rejects absolute globs, unknown modes, and artifact options when uploads are off; resx/xlf strings added; pre-existing Eanble→Enable typo fixed.
  • Help/info acceptance expectations updated and 12 new unit tests covering modes, tag emission, escaping, globbing, missing TF_BUILD, and validator rejection.
Show a summary per file
FileDescription
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsArtifactUploader.csNew uploader implementing the artifact-upload + build-tag logic.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsCommandLineOptions.csNew option/mode constants.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsCommandLineProvider.csRegisters and validates the new options.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsExtensions.csRegisters the uploader as data consumer + session lifetime handler.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsReporter.csSwitched TFM resolution to shared helper.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/PathComparison.csOS-aware path string comparer/comparison.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/TargetFrameworkMonikerHelper.csShared TFM resolution helper.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Microsoft.Testing.Extensions.AzureDevOpsReport.csprojAdds Microsoft.Extensions.FileSystemGlobbing reference.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/AzureDevOpsResources.resxNew strings + Eanble→Enable typo fix.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/*.xlfRegenerated translations for all locales.
test/UnitTests/Microsoft.Testing.Extensions.UnitTests/AzureDevOpsArtifactUploaderTests.cs12 new unit tests for uploader + validator.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/HelpInfoAllExtensionsTests.csAdds AzDO extension + new option entries to help/info expectations.

Copilot's findings

  • Files reviewed: 24/24 changed files
  • Comments generated: 1

@Evangelink

Copy link
Copy Markdown
MemberAuthor

/review

@github-actions

github-actionsBot commented May 17, 2026

Copy link
Copy Markdown
Contributor

Expert Code Review (command) completed successfully!

Expert-reviewer launched in background for PR #8295. It will post the review directly.

@Evangelink
Amaury Levé (Evangelink) marked this pull request as ready for review May 17, 2026 19:13
CopilotAI review requested due to automatic review settings May 18, 2026 07:13
@Evangelink

Copy link
Copy Markdown
MemberAuthor

Copilot address review comments

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot's findings

  • Files reviewed: 24/24 changed files
  • Comments generated: 0 new

Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com>

CopilotAI commented May 18, 2026

Copy link
Copy Markdown
Contributor

Copilot address review comments

Addressed in 79f309a. I updated AzureDevOpsArtifactUploaderTests to use OS-agnostic paths (Path.Combine-based helpers) and adjusted assertions/globs accordingly so the files-mode tests pass cross-platform.

Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com>
CopilotAI review requested due to automatic review settings May 18, 2026 07:21
@Evangelink
Amaury Levé (Evangelink) removed the request for review from CopilotMay 18, 2026 07:21
Resolve the AzureDevOpsReporter merge conflict and fix net462-compatible escaping in AzureDevOpsArtifactUploaderTests.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
CopilotAI review requested due to automatic review settings May 18, 2026 19:08

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot's findings

  • Files reviewed: 24/24 changed files
  • Comments generated: 0 new

@Evangelink

Copy link
Copy Markdown
MemberAuthor

Copilot resolve the merge conflicts in this pull request

…rtifact-upload
# Conflicts:
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsCommandLineProvider.cs
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsExtensions.cs
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsReporter.cs
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Microsoft.Testing.Extensions.AzureDevOpsReport.csproj
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/AzureDevOpsResources.resx
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.cs.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.de.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.es.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.fr.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.it.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.ja.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.ko.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.pl.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.pt-BR.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.ru.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.tr.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.zh-Hans.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.zh-Hant.xlf
#	test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/HelpInfoAllExtensionsTests.cs
Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com>

CopilotAI commented May 19, 2026

Copy link
Copy Markdown
Contributor

Copilot resolve the merge conflicts in this pull request

Merge conflicts are resolved in commit bb8d863 by merging origin/main and reconciling the Azure DevOps report, resources, and help-info expectations.

CopilotAI review requested due to automatic review settings May 19, 2026 12:42

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot's findings

  • Files reviewed: 11/11 changed files
  • Comments generated: 0 new

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@Evangelink

Copy link
Copy Markdown
MemberAuthor

Addressed in b5bd216:

  • Cross-platform paths in tests (the only inline review comment from Copilot): already fixed in earlier 79f309a / bbe254cAzureDevOpsArtifactUploaderTests uses Path.Combine-based InResults/OutsideResults helpers rooted at Path.GetTempPath(). Verified.
  • Merge fallout from ff7261f05 (broken state on origin):
    • AzureDevOpsExtensions.cs: restored using Microsoft.Testing.Platform.Extensions; (required for CompositeExtensionFactory<>).
    • AzureDevOpsReporter.cs: removed leftover dead GetTargetFrameworkMoniker method (call site now uses TargetFrameworkMonikerHelper).
    • Resources/xlf/AzureDevOpsResources.*.xlf: regenerated via /t:UpdateXlf so all 14 locales have the 7 new state="new" trans-units that the merge dropped.

Verification:

  • .\build.cmd — Build succeeded, 0 Warning(s), 0 Error(s).
  • All 23 AzureDevOpsArtifactUploaderTests pass (full extensions unit-test suite: 174 passed, 2 skipped, 0 failed).

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@Evangelink
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

Auto-upload test artifacts and build tags to Azure DevOps - #8295

Merged
Amaury Levé (Evangelink) merged 9 commits into
mainfrom
dev/amauryleve/azdo-artifact-upload
May 19, 2026
Merged

Auto-upload test artifacts and build tags to Azure DevOps#8295
Amaury Levé (Evangelink) merged 9 commits into
mainfrom
dev/amauryleve/azdo-artifact-upload

Conversation

@Evangelink

Copy link
Copy Markdown
Member

Part 3 of the brainstorm in #5951 — adds opt-in pipeline-level artifact upload + build tags to Microsoft.Testing.Extensions.AzureDevOpsReport. Pure ##vso logging commands — no Azure DevOps authentication required.

This is one of three PRs derived from the #5951 brainstorm. The other two:

  • Live test-result publishing
  • Flaky-history annotations + quarantine awareness

See comment for the broader plan.

Why

Today HangDump, CrashDump, CodeCoverage, and TRX all drop files into TestResultsDirectory, but nothing teaches Azure DevOps to publish them as build artifacts — users have to add a separate PublishBuildArtifacts@1 task. Similarly, build queries can't easily filter "runs that produced a hang/crash dump" without grepping logs.

What

Four new opt-in CLI options on Microsoft.Testing.Extensions.AzureDevOpsReport:

OptionTypePurpose
--report-azdo-upload-artifacts <mode>off (default) | tags-only | files | allEnable build tags, file upload, or both.
--report-azdo-upload-artifact-name <name>stringOverride the artifact container name. Default: TestResults_<assemblyName>_<tfm>.
--report-azdo-upload-artifact-include <glob>repeatableFile globs (relative to TestResultsDirectory) to include. Default: **/*.
--report-azdo-upload-artifact-exclude <glob>repeatableExclusion globs. Default: none.

At session end (when TF_BUILD=true and the mode is not off), the new AzureDevOpsArtifactUploader:

  • Emits ##vso[build.addbuildtag]has-crashdump and/or has-hangdump based on the producer UID of consumed TestNodeFileArtifact events (CrashDumpProcessLifetimeHandler / HangDumpProcessLifetimeHandler — robust to localization).
  • Emits ##vso[build.addbuildtag]has-test-failures if the session contained any failed test.
  • Enumerates TestResultsDirectory via IFileSystem, applies include/exclude globs (case-sensitive on Linux, case-insensitive on Windows), and emits ##vso[artifact.upload containerfolder=…;artifactname=…]<path> for each match — properly escaping ;, \r, \n via the existing AzDoEscaper.

All output goes through IOutputDevice (no direct Console.WriteLine). All env/path/file access goes through IEnvironment / IConfiguration / IFileSystem. Exception isolation: callbacks never propagate non-cancellation exceptions out of the lifetime handler.

Highlights from the expert-reviewer round

The implementation went through one round of expert-reviewer. Critical/major issues addressed:

  • Failure isolation: all callbacks now try/catch everything except OperationCanceledException; warnings logged via ILogger.
  • Validator: rejects --report-azdo-upload-artifact-{include,exclude,name} when --report-azdo-upload-artifacts is absent or off.
  • Race-free flags: monotonic crash/hang/failure flags use Interlocked.Exchange + Volatile.Read.
  • Cross-platform paths: new PathComparison helper (OrdinalIgnoreCase on Windows, Ordinal elsewhere) applied to glob construction, prefix matching, and sort.
  • Dump detection: producer-UID-based only — no longer matches on localized DisplayName or path substring.
  • Out-of-dir filter: files resolved outside TestResultsDirectory (e.g. via symlinks) are skipped.
  • Lazy TFM resolution via new shared TargetFrameworkMonikerHelper (also adopted by the existing AzureDevOpsReporter).
  • Pre-existing typo EanbleEnable fixed in Resources/AzureDevOpsResources.resx.

Tests

  • 12 new unit tests in test/UnitTests/Microsoft.Testing.Extensions.UnitTests/AzureDevOpsArtifactUploaderTests.cs covering mode wiring (incl. off no-op), tag emission (exactly once, even under concurrent message events), glob include/exclude, escaping, missing TF_BUILD, null/whitespace TestResultsDirectory, out-of-dir filtering, and validator rejection.
  • HelpInfoAllExtensionsTests expectations updated for new options (both --help and --info blocks, alphabetical order preserved).

Build status (local)

  • .\.dotnet\dotnet.exe build src\Platform\Microsoft.Testing.Extensions.AzureDevOpsReport\Microsoft.Testing.Extensions.AzureDevOpsReport.csproj -c Debug0 warnings, 0 errors.
  • .\.dotnet\dotnet.exe test test\UnitTests\Microsoft.Testing.Extensions.UnitTests\Microsoft.Testing.Extensions.UnitTests.csproj566/566 passed.
  • .\build.cmd -pack0 warnings, 0 errors.

Out of scope (deliberate)

  • Result-level attachments (attach a dump to its failing test in the AzDO Tests tab) — depends on the REST client introduced by the live-publishing PR.
  • Code-coverage attachment under the Coverage tab — same reason.

Both can land as small follow-ups once that PR is merged.

Checklist

  • Critical & Major review findings addressed
  • Localized via resx + xlf (regenerated with /t:UpdateXlf, not hand-edited)
  • Help/info acceptance test expectations updated
  • No new public API
  • .\build.cmd green (0 warnings, 0 errors)
  • PR feedback addressed

Refs #5951

Add opt-in Azure DevOps artifact upload support, help/info updates, and unit coverage for the new reporter behavior.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
CopilotAI review requested due to automatic review settings May 16, 2026 16:33

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds an opt-in pipeline-level artifact upload + build-tag feature to Microsoft.Testing.Extensions.AzureDevOpsReport. It introduces four new CLI options that, when running under Azure DevOps (TF_BUILD=true), emit ##vso[artifact.upload …] and ##vso[build.addbuildtag]… logging commands at session end based on consumed FileArtifact/TestNodeUpdateMessage events and a glob filter over the test results directory.

Changes:

  • New AzureDevOpsArtifactUploader (data consumer + session lifetime handler) plus supporting PathComparison / TargetFrameworkMonikerHelper helpers and AzureDevOpsArtifactUploadMode enum.
  • New CLI options (--report-azdo-upload-artifacts, --report-azdo-upload-artifact-{name,include,exclude}) with validator that rejects absolute globs, unknown modes, and artifact options when uploads are off; resx/xlf strings added; pre-existing Eanble→Enable typo fixed.
  • Help/info acceptance expectations updated and 12 new unit tests covering modes, tag emission, escaping, globbing, missing TF_BUILD, and validator rejection.
Show a summary per file
FileDescription
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsArtifactUploader.csNew uploader implementing the artifact-upload + build-tag logic.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsCommandLineOptions.csNew option/mode constants.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsCommandLineProvider.csRegisters and validates the new options.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsExtensions.csRegisters the uploader as data consumer + session lifetime handler.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsReporter.csSwitched TFM resolution to shared helper.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/PathComparison.csOS-aware path string comparer/comparison.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/TargetFrameworkMonikerHelper.csShared TFM resolution helper.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Microsoft.Testing.Extensions.AzureDevOpsReport.csprojAdds Microsoft.Extensions.FileSystemGlobbing reference.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/AzureDevOpsResources.resxNew strings + Eanble→Enable typo fix.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/*.xlfRegenerated translations for all locales.
test/UnitTests/Microsoft.Testing.Extensions.UnitTests/AzureDevOpsArtifactUploaderTests.cs12 new unit tests for uploader + validator.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/HelpInfoAllExtensionsTests.csAdds AzDO extension + new option entries to help/info expectations.

Copilot's findings

  • Files reviewed: 24/24 changed files
  • Comments generated: 1

@Evangelink

Copy link
Copy Markdown
MemberAuthor

/review

@github-actions

github-actionsBot commented May 17, 2026

Copy link
Copy Markdown
Contributor

Expert Code Review (command) completed successfully!

Expert-reviewer launched in background for PR #8295. It will post the review directly.

@Evangelink
Amaury Levé (Evangelink) marked this pull request as ready for review May 17, 2026 19:13
CopilotAI review requested due to automatic review settings May 18, 2026 07:13
@Evangelink

Copy link
Copy Markdown
MemberAuthor

Copilot address review comments

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot's findings

  • Files reviewed: 24/24 changed files
  • Comments generated: 0 new

Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com>

CopilotAI commented May 18, 2026

Copy link
Copy Markdown
Contributor

Copilot address review comments

Addressed in 79f309a. I updated AzureDevOpsArtifactUploaderTests to use OS-agnostic paths (Path.Combine-based helpers) and adjusted assertions/globs accordingly so the files-mode tests pass cross-platform.

Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com>
CopilotAI review requested due to automatic review settings May 18, 2026 07:21
@Evangelink
Amaury Levé (Evangelink) removed the request for review from CopilotMay 18, 2026 07:21
Resolve the AzureDevOpsReporter merge conflict and fix net462-compatible escaping in AzureDevOpsArtifactUploaderTests.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
CopilotAI review requested due to automatic review settings May 18, 2026 19:08

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot's findings

  • Files reviewed: 24/24 changed files
  • Comments generated: 0 new

@Evangelink

Copy link
Copy Markdown
MemberAuthor

Copilot resolve the merge conflicts in this pull request

…rtifact-upload
# Conflicts:
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsCommandLineProvider.cs
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsExtensions.cs
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsReporter.cs
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Microsoft.Testing.Extensions.AzureDevOpsReport.csproj
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/AzureDevOpsResources.resx
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.cs.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.de.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.es.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.fr.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.it.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.ja.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.ko.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.pl.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.pt-BR.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.ru.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.tr.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.zh-Hans.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.zh-Hant.xlf
#	test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/HelpInfoAllExtensionsTests.cs
Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com>

CopilotAI commented May 19, 2026

Copy link
Copy Markdown
Contributor

Copilot resolve the merge conflicts in this pull request

Merge conflicts are resolved in commit bb8d863 by merging origin/main and reconciling the Azure DevOps report, resources, and help-info expectations.

CopilotAI review requested due to automatic review settings May 19, 2026 12:42

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot's findings

  • Files reviewed: 11/11 changed files
  • Comments generated: 0 new

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@Evangelink

Copy link
Copy Markdown
MemberAuthor

Addressed in b5bd216:

  • Cross-platform paths in tests (the only inline review comment from Copilot): already fixed in earlier 79f309a / bbe254cAzureDevOpsArtifactUploaderTests uses Path.Combine-based InResults/OutsideResults helpers rooted at Path.GetTempPath(). Verified.
  • Merge fallout from ff7261f05 (broken state on origin):
    • AzureDevOpsExtensions.cs: restored using Microsoft.Testing.Platform.Extensions; (required for CompositeExtensionFactory<>).
    • AzureDevOpsReporter.cs: removed leftover dead GetTargetFrameworkMoniker method (call site now uses TargetFrameworkMonikerHelper).
    • Resources/xlf/AzureDevOpsResources.*.xlf: regenerated via /t:UpdateXlf so all 14 locales have the 7 new state="new" trans-units that the merge dropped.

Verification:

  • .\build.cmd — Build succeeded, 0 Warning(s), 0 Error(s).
  • All 23 AzureDevOpsArtifactUploaderTests pass (full extensions unit-test suite: 174 passed, 2 skipped, 0 failed).

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@Evangelink
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Auto-upload test artifacts and build tags to Azure DevOps - #8295

Merged
Amaury Levé (Evangelink) merged 9 commits into
mainfrom
dev/amauryleve/azdo-artifact-upload
May 19, 2026
Merged

Auto-upload test artifacts and build tags to Azure DevOps#8295
Amaury Levé (Evangelink) merged 9 commits into
mainfrom
dev/amauryleve/azdo-artifact-upload

Conversation

@Evangelink

Copy link
Copy Markdown
Member

Part 3 of the brainstorm in #5951 — adds opt-in pipeline-level artifact upload + build tags to Microsoft.Testing.Extensions.AzureDevOpsReport. Pure ##vso logging commands — no Azure DevOps authentication required.

This is one of three PRs derived from the #5951 brainstorm. The other two:

  • Live test-result publishing
  • Flaky-history annotations + quarantine awareness

See comment for the broader plan.

Why

Today HangDump, CrashDump, CodeCoverage, and TRX all drop files into TestResultsDirectory, but nothing teaches Azure DevOps to publish them as build artifacts — users have to add a separate PublishBuildArtifacts@1 task. Similarly, build queries can't easily filter "runs that produced a hang/crash dump" without grepping logs.

What

Four new opt-in CLI options on Microsoft.Testing.Extensions.AzureDevOpsReport:

OptionTypePurpose
--report-azdo-upload-artifacts <mode>off (default) | tags-only | files | allEnable build tags, file upload, or both.
--report-azdo-upload-artifact-name <name>stringOverride the artifact container name. Default: TestResults_<assemblyName>_<tfm>.
--report-azdo-upload-artifact-include <glob>repeatableFile globs (relative to TestResultsDirectory) to include. Default: **/*.
--report-azdo-upload-artifact-exclude <glob>repeatableExclusion globs. Default: none.

At session end (when TF_BUILD=true and the mode is not off), the new AzureDevOpsArtifactUploader:

  • Emits ##vso[build.addbuildtag]has-crashdump and/or has-hangdump based on the producer UID of consumed TestNodeFileArtifact events (CrashDumpProcessLifetimeHandler / HangDumpProcessLifetimeHandler — robust to localization).
  • Emits ##vso[build.addbuildtag]has-test-failures if the session contained any failed test.
  • Enumerates TestResultsDirectory via IFileSystem, applies include/exclude globs (case-sensitive on Linux, case-insensitive on Windows), and emits ##vso[artifact.upload containerfolder=…;artifactname=…]<path> for each match — properly escaping ;, \r, \n via the existing AzDoEscaper.

All output goes through IOutputDevice (no direct Console.WriteLine). All env/path/file access goes through IEnvironment / IConfiguration / IFileSystem. Exception isolation: callbacks never propagate non-cancellation exceptions out of the lifetime handler.

Highlights from the expert-reviewer round

The implementation went through one round of expert-reviewer. Critical/major issues addressed:

  • Failure isolation: all callbacks now try/catch everything except OperationCanceledException; warnings logged via ILogger.
  • Validator: rejects --report-azdo-upload-artifact-{include,exclude,name} when --report-azdo-upload-artifacts is absent or off.
  • Race-free flags: monotonic crash/hang/failure flags use Interlocked.Exchange + Volatile.Read.
  • Cross-platform paths: new PathComparison helper (OrdinalIgnoreCase on Windows, Ordinal elsewhere) applied to glob construction, prefix matching, and sort.
  • Dump detection: producer-UID-based only — no longer matches on localized DisplayName or path substring.
  • Out-of-dir filter: files resolved outside TestResultsDirectory (e.g. via symlinks) are skipped.
  • Lazy TFM resolution via new shared TargetFrameworkMonikerHelper (also adopted by the existing AzureDevOpsReporter).
  • Pre-existing typo EanbleEnable fixed in Resources/AzureDevOpsResources.resx.

Tests

  • 12 new unit tests in test/UnitTests/Microsoft.Testing.Extensions.UnitTests/AzureDevOpsArtifactUploaderTests.cs covering mode wiring (incl. off no-op), tag emission (exactly once, even under concurrent message events), glob include/exclude, escaping, missing TF_BUILD, null/whitespace TestResultsDirectory, out-of-dir filtering, and validator rejection.
  • HelpInfoAllExtensionsTests expectations updated for new options (both --help and --info blocks, alphabetical order preserved).

Build status (local)

  • .\.dotnet\dotnet.exe build src\Platform\Microsoft.Testing.Extensions.AzureDevOpsReport\Microsoft.Testing.Extensions.AzureDevOpsReport.csproj -c Debug0 warnings, 0 errors.
  • .\.dotnet\dotnet.exe test test\UnitTests\Microsoft.Testing.Extensions.UnitTests\Microsoft.Testing.Extensions.UnitTests.csproj566/566 passed.
  • .\build.cmd -pack0 warnings, 0 errors.

Out of scope (deliberate)

  • Result-level attachments (attach a dump to its failing test in the AzDO Tests tab) — depends on the REST client introduced by the live-publishing PR.
  • Code-coverage attachment under the Coverage tab — same reason.

Both can land as small follow-ups once that PR is merged.

Checklist

  • Critical & Major review findings addressed
  • Localized via resx + xlf (regenerated with /t:UpdateXlf, not hand-edited)
  • Help/info acceptance test expectations updated
  • No new public API
  • .\build.cmd green (0 warnings, 0 errors)
  • PR feedback addressed

Refs #5951

Add opt-in Azure DevOps artifact upload support, help/info updates, and unit coverage for the new reporter behavior.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
CopilotAI review requested due to automatic review settings May 16, 2026 16:33

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds an opt-in pipeline-level artifact upload + build-tag feature to Microsoft.Testing.Extensions.AzureDevOpsReport. It introduces four new CLI options that, when running under Azure DevOps (TF_BUILD=true), emit ##vso[artifact.upload …] and ##vso[build.addbuildtag]… logging commands at session end based on consumed FileArtifact/TestNodeUpdateMessage events and a glob filter over the test results directory.

Changes:

  • New AzureDevOpsArtifactUploader (data consumer + session lifetime handler) plus supporting PathComparison / TargetFrameworkMonikerHelper helpers and AzureDevOpsArtifactUploadMode enum.
  • New CLI options (--report-azdo-upload-artifacts, --report-azdo-upload-artifact-{name,include,exclude}) with validator that rejects absolute globs, unknown modes, and artifact options when uploads are off; resx/xlf strings added; pre-existing Eanble→Enable typo fixed.
  • Help/info acceptance expectations updated and 12 new unit tests covering modes, tag emission, escaping, globbing, missing TF_BUILD, and validator rejection.
Show a summary per file
FileDescription
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsArtifactUploader.csNew uploader implementing the artifact-upload + build-tag logic.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsCommandLineOptions.csNew option/mode constants.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsCommandLineProvider.csRegisters and validates the new options.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsExtensions.csRegisters the uploader as data consumer + session lifetime handler.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsReporter.csSwitched TFM resolution to shared helper.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/PathComparison.csOS-aware path string comparer/comparison.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/TargetFrameworkMonikerHelper.csShared TFM resolution helper.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Microsoft.Testing.Extensions.AzureDevOpsReport.csprojAdds Microsoft.Extensions.FileSystemGlobbing reference.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/AzureDevOpsResources.resxNew strings + Eanble→Enable typo fix.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/*.xlfRegenerated translations for all locales.
test/UnitTests/Microsoft.Testing.Extensions.UnitTests/AzureDevOpsArtifactUploaderTests.cs12 new unit tests for uploader + validator.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/HelpInfoAllExtensionsTests.csAdds AzDO extension + new option entries to help/info expectations.

Copilot's findings

  • Files reviewed: 24/24 changed files
  • Comments generated: 1

@Evangelink

Copy link
Copy Markdown
MemberAuthor

/review

@github-actions

github-actionsBot commented May 17, 2026

Copy link
Copy Markdown
Contributor

Expert Code Review (command) completed successfully!

Expert-reviewer launched in background for PR #8295. It will post the review directly.

@Evangelink
Amaury Levé (Evangelink) marked this pull request as ready for review May 17, 2026 19:13
CopilotAI review requested due to automatic review settings May 18, 2026 07:13
@Evangelink

Copy link
Copy Markdown
MemberAuthor

Copilot address review comments

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot's findings

  • Files reviewed: 24/24 changed files
  • Comments generated: 0 new

Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com>

CopilotAI commented May 18, 2026

Copy link
Copy Markdown
Contributor

Copilot address review comments

Addressed in 79f309a. I updated AzureDevOpsArtifactUploaderTests to use OS-agnostic paths (Path.Combine-based helpers) and adjusted assertions/globs accordingly so the files-mode tests pass cross-platform.

Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com>
CopilotAI review requested due to automatic review settings May 18, 2026 07:21
@Evangelink
Amaury Levé (Evangelink) removed the request for review from CopilotMay 18, 2026 07:21
Resolve the AzureDevOpsReporter merge conflict and fix net462-compatible escaping in AzureDevOpsArtifactUploaderTests.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
CopilotAI review requested due to automatic review settings May 18, 2026 19:08

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot's findings

  • Files reviewed: 24/24 changed files
  • Comments generated: 0 new

@Evangelink

Copy link
Copy Markdown
MemberAuthor

Copilot resolve the merge conflicts in this pull request

…rtifact-upload
# Conflicts:
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsCommandLineProvider.cs
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsExtensions.cs
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsReporter.cs
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Microsoft.Testing.Extensions.AzureDevOpsReport.csproj
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/AzureDevOpsResources.resx
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.cs.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.de.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.es.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.fr.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.it.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.ja.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.ko.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.pl.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.pt-BR.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.ru.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.tr.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.zh-Hans.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.zh-Hant.xlf
#	test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/HelpInfoAllExtensionsTests.cs
Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com>

CopilotAI commented May 19, 2026

Copy link
Copy Markdown
Contributor

Copilot resolve the merge conflicts in this pull request

Merge conflicts are resolved in commit bb8d863 by merging origin/main and reconciling the Azure DevOps report, resources, and help-info expectations.

CopilotAI review requested due to automatic review settings May 19, 2026 12:42

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot's findings

  • Files reviewed: 11/11 changed files
  • Comments generated: 0 new

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@Evangelink

Copy link
Copy Markdown
MemberAuthor

Addressed in b5bd216:

  • Cross-platform paths in tests (the only inline review comment from Copilot): already fixed in earlier 79f309a / bbe254cAzureDevOpsArtifactUploaderTests uses Path.Combine-based InResults/OutsideResults helpers rooted at Path.GetTempPath(). Verified.
  • Merge fallout from ff7261f05 (broken state on origin):
    • AzureDevOpsExtensions.cs: restored using Microsoft.Testing.Platform.Extensions; (required for CompositeExtensionFactory<>).
    • AzureDevOpsReporter.cs: removed leftover dead GetTargetFrameworkMoniker method (call site now uses TargetFrameworkMonikerHelper).
    • Resources/xlf/AzureDevOpsResources.*.xlf: regenerated via /t:UpdateXlf so all 14 locales have the 7 new state="new" trans-units that the merge dropped.

Verification:

  • .\build.cmd — Build succeeded, 0 Warning(s), 0 Error(s).
  • All 23 AzureDevOpsArtifactUploaderTests pass (full extensions unit-test suite: 174 passed, 2 skipped, 0 failed).

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@Evangelink
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Auto-upload test artifacts and build tags to Azure DevOps - #8295

Merged
Amaury Levé (Evangelink) merged 9 commits into
mainfrom
dev/amauryleve/azdo-artifact-upload
May 19, 2026
Merged

Auto-upload test artifacts and build tags to Azure DevOps#8295
Amaury Levé (Evangelink) merged 9 commits into
mainfrom
dev/amauryleve/azdo-artifact-upload

Conversation

@Evangelink

Copy link
Copy Markdown
Member

Part 3 of the brainstorm in #5951 — adds opt-in pipeline-level artifact upload + build tags to Microsoft.Testing.Extensions.AzureDevOpsReport. Pure ##vso logging commands — no Azure DevOps authentication required.

This is one of three PRs derived from the #5951 brainstorm. The other two:

  • Live test-result publishing
  • Flaky-history annotations + quarantine awareness

See comment for the broader plan.

Why

Today HangDump, CrashDump, CodeCoverage, and TRX all drop files into TestResultsDirectory, but nothing teaches Azure DevOps to publish them as build artifacts — users have to add a separate PublishBuildArtifacts@1 task. Similarly, build queries can't easily filter "runs that produced a hang/crash dump" without grepping logs.

What

Four new opt-in CLI options on Microsoft.Testing.Extensions.AzureDevOpsReport:

OptionTypePurpose
--report-azdo-upload-artifacts <mode>off (default) | tags-only | files | allEnable build tags, file upload, or both.
--report-azdo-upload-artifact-name <name>stringOverride the artifact container name. Default: TestResults_<assemblyName>_<tfm>.
--report-azdo-upload-artifact-include <glob>repeatableFile globs (relative to TestResultsDirectory) to include. Default: **/*.
--report-azdo-upload-artifact-exclude <glob>repeatableExclusion globs. Default: none.

At session end (when TF_BUILD=true and the mode is not off), the new AzureDevOpsArtifactUploader:

  • Emits ##vso[build.addbuildtag]has-crashdump and/or has-hangdump based on the producer UID of consumed TestNodeFileArtifact events (CrashDumpProcessLifetimeHandler / HangDumpProcessLifetimeHandler — robust to localization).
  • Emits ##vso[build.addbuildtag]has-test-failures if the session contained any failed test.
  • Enumerates TestResultsDirectory via IFileSystem, applies include/exclude globs (case-sensitive on Linux, case-insensitive on Windows), and emits ##vso[artifact.upload containerfolder=…;artifactname=…]<path> for each match — properly escaping ;, \r, \n via the existing AzDoEscaper.

All output goes through IOutputDevice (no direct Console.WriteLine). All env/path/file access goes through IEnvironment / IConfiguration / IFileSystem. Exception isolation: callbacks never propagate non-cancellation exceptions out of the lifetime handler.

Highlights from the expert-reviewer round

The implementation went through one round of expert-reviewer. Critical/major issues addressed:

  • Failure isolation: all callbacks now try/catch everything except OperationCanceledException; warnings logged via ILogger.
  • Validator: rejects --report-azdo-upload-artifact-{include,exclude,name} when --report-azdo-upload-artifacts is absent or off.
  • Race-free flags: monotonic crash/hang/failure flags use Interlocked.Exchange + Volatile.Read.
  • Cross-platform paths: new PathComparison helper (OrdinalIgnoreCase on Windows, Ordinal elsewhere) applied to glob construction, prefix matching, and sort.
  • Dump detection: producer-UID-based only — no longer matches on localized DisplayName or path substring.
  • Out-of-dir filter: files resolved outside TestResultsDirectory (e.g. via symlinks) are skipped.
  • Lazy TFM resolution via new shared TargetFrameworkMonikerHelper (also adopted by the existing AzureDevOpsReporter).
  • Pre-existing typo EanbleEnable fixed in Resources/AzureDevOpsResources.resx.

Tests

  • 12 new unit tests in test/UnitTests/Microsoft.Testing.Extensions.UnitTests/AzureDevOpsArtifactUploaderTests.cs covering mode wiring (incl. off no-op), tag emission (exactly once, even under concurrent message events), glob include/exclude, escaping, missing TF_BUILD, null/whitespace TestResultsDirectory, out-of-dir filtering, and validator rejection.
  • HelpInfoAllExtensionsTests expectations updated for new options (both --help and --info blocks, alphabetical order preserved).

Build status (local)

  • .\.dotnet\dotnet.exe build src\Platform\Microsoft.Testing.Extensions.AzureDevOpsReport\Microsoft.Testing.Extensions.AzureDevOpsReport.csproj -c Debug0 warnings, 0 errors.
  • .\.dotnet\dotnet.exe test test\UnitTests\Microsoft.Testing.Extensions.UnitTests\Microsoft.Testing.Extensions.UnitTests.csproj566/566 passed.
  • .\build.cmd -pack0 warnings, 0 errors.

Out of scope (deliberate)

  • Result-level attachments (attach a dump to its failing test in the AzDO Tests tab) — depends on the REST client introduced by the live-publishing PR.
  • Code-coverage attachment under the Coverage tab — same reason.

Both can land as small follow-ups once that PR is merged.

Checklist

  • Critical & Major review findings addressed
  • Localized via resx + xlf (regenerated with /t:UpdateXlf, not hand-edited)
  • Help/info acceptance test expectations updated
  • No new public API
  • .\build.cmd green (0 warnings, 0 errors)
  • PR feedback addressed

Refs #5951

Add opt-in Azure DevOps artifact upload support, help/info updates, and unit coverage for the new reporter behavior.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
CopilotAI review requested due to automatic review settings May 16, 2026 16:33

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds an opt-in pipeline-level artifact upload + build-tag feature to Microsoft.Testing.Extensions.AzureDevOpsReport. It introduces four new CLI options that, when running under Azure DevOps (TF_BUILD=true), emit ##vso[artifact.upload …] and ##vso[build.addbuildtag]… logging commands at session end based on consumed FileArtifact/TestNodeUpdateMessage events and a glob filter over the test results directory.

Changes:

  • New AzureDevOpsArtifactUploader (data consumer + session lifetime handler) plus supporting PathComparison / TargetFrameworkMonikerHelper helpers and AzureDevOpsArtifactUploadMode enum.
  • New CLI options (--report-azdo-upload-artifacts, --report-azdo-upload-artifact-{name,include,exclude}) with validator that rejects absolute globs, unknown modes, and artifact options when uploads are off; resx/xlf strings added; pre-existing Eanble→Enable typo fixed.
  • Help/info acceptance expectations updated and 12 new unit tests covering modes, tag emission, escaping, globbing, missing TF_BUILD, and validator rejection.
Show a summary per file
FileDescription
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsArtifactUploader.csNew uploader implementing the artifact-upload + build-tag logic.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsCommandLineOptions.csNew option/mode constants.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsCommandLineProvider.csRegisters and validates the new options.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsExtensions.csRegisters the uploader as data consumer + session lifetime handler.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsReporter.csSwitched TFM resolution to shared helper.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/PathComparison.csOS-aware path string comparer/comparison.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/TargetFrameworkMonikerHelper.csShared TFM resolution helper.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Microsoft.Testing.Extensions.AzureDevOpsReport.csprojAdds Microsoft.Extensions.FileSystemGlobbing reference.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/AzureDevOpsResources.resxNew strings + Eanble→Enable typo fix.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/*.xlfRegenerated translations for all locales.
test/UnitTests/Microsoft.Testing.Extensions.UnitTests/AzureDevOpsArtifactUploaderTests.cs12 new unit tests for uploader + validator.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/HelpInfoAllExtensionsTests.csAdds AzDO extension + new option entries to help/info expectations.

Copilot's findings

  • Files reviewed: 24/24 changed files
  • Comments generated: 1

@Evangelink

Copy link
Copy Markdown
MemberAuthor

/review

@github-actions

github-actionsBot commented May 17, 2026

Copy link
Copy Markdown
Contributor

Expert Code Review (command) completed successfully!

Expert-reviewer launched in background for PR #8295. It will post the review directly.

@Evangelink
Amaury Levé (Evangelink) marked this pull request as ready for review May 17, 2026 19:13
CopilotAI review requested due to automatic review settings May 18, 2026 07:13
@Evangelink

Copy link
Copy Markdown
MemberAuthor

Copilot address review comments

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot's findings

  • Files reviewed: 24/24 changed files
  • Comments generated: 0 new

Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com>

CopilotAI commented May 18, 2026

Copy link
Copy Markdown
Contributor

Copilot address review comments

Addressed in 79f309a. I updated AzureDevOpsArtifactUploaderTests to use OS-agnostic paths (Path.Combine-based helpers) and adjusted assertions/globs accordingly so the files-mode tests pass cross-platform.

Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com>
CopilotAI review requested due to automatic review settings May 18, 2026 07:21
@Evangelink
Amaury Levé (Evangelink) removed the request for review from CopilotMay 18, 2026 07:21
Resolve the AzureDevOpsReporter merge conflict and fix net462-compatible escaping in AzureDevOpsArtifactUploaderTests.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
CopilotAI review requested due to automatic review settings May 18, 2026 19:08

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot's findings

  • Files reviewed: 24/24 changed files
  • Comments generated: 0 new

@Evangelink

Copy link
Copy Markdown
MemberAuthor

Copilot resolve the merge conflicts in this pull request

…rtifact-upload
# Conflicts:
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsCommandLineProvider.cs
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsExtensions.cs
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsReporter.cs
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Microsoft.Testing.Extensions.AzureDevOpsReport.csproj
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/AzureDevOpsResources.resx
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.cs.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.de.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.es.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.fr.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.it.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.ja.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.ko.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.pl.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.pt-BR.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.ru.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.tr.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.zh-Hans.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.zh-Hant.xlf
#	test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/HelpInfoAllExtensionsTests.cs
Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com>

CopilotAI commented May 19, 2026

Copy link
Copy Markdown
Contributor

Copilot resolve the merge conflicts in this pull request

Merge conflicts are resolved in commit bb8d863 by merging origin/main and reconciling the Azure DevOps report, resources, and help-info expectations.

CopilotAI review requested due to automatic review settings May 19, 2026 12:42

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot's findings

  • Files reviewed: 11/11 changed files
  • Comments generated: 0 new

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@Evangelink

Copy link
Copy Markdown
MemberAuthor

Addressed in b5bd216:

  • Cross-platform paths in tests (the only inline review comment from Copilot): already fixed in earlier 79f309a / bbe254cAzureDevOpsArtifactUploaderTests uses Path.Combine-based InResults/OutsideResults helpers rooted at Path.GetTempPath(). Verified.
  • Merge fallout from ff7261f05 (broken state on origin):
    • AzureDevOpsExtensions.cs: restored using Microsoft.Testing.Platform.Extensions; (required for CompositeExtensionFactory<>).
    • AzureDevOpsReporter.cs: removed leftover dead GetTargetFrameworkMoniker method (call site now uses TargetFrameworkMonikerHelper).
    • Resources/xlf/AzureDevOpsResources.*.xlf: regenerated via /t:UpdateXlf so all 14 locales have the 7 new state="new" trans-units that the merge dropped.

Verification:

  • .\build.cmd — Build succeeded, 0 Warning(s), 0 Error(s).
  • All 23 AzureDevOpsArtifactUploaderTests pass (full extensions unit-test suite: 174 passed, 2 skipped, 0 failed).

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@Evangelink
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

Auto-upload test artifacts and build tags to Azure DevOps - #8295

Merged
Amaury Levé (Evangelink) merged 9 commits into
mainfrom
dev/amauryleve/azdo-artifact-upload
May 19, 2026
Merged

Auto-upload test artifacts and build tags to Azure DevOps#8295
Amaury Levé (Evangelink) merged 9 commits into
mainfrom
dev/amauryleve/azdo-artifact-upload

Conversation

@Evangelink

Copy link
Copy Markdown
Member

Part 3 of the brainstorm in #5951 — adds opt-in pipeline-level artifact upload + build tags to Microsoft.Testing.Extensions.AzureDevOpsReport. Pure ##vso logging commands — no Azure DevOps authentication required.

This is one of three PRs derived from the #5951 brainstorm. The other two:

  • Live test-result publishing
  • Flaky-history annotations + quarantine awareness

See comment for the broader plan.

Why

Today HangDump, CrashDump, CodeCoverage, and TRX all drop files into TestResultsDirectory, but nothing teaches Azure DevOps to publish them as build artifacts — users have to add a separate PublishBuildArtifacts@1 task. Similarly, build queries can't easily filter "runs that produced a hang/crash dump" without grepping logs.

What

Four new opt-in CLI options on Microsoft.Testing.Extensions.AzureDevOpsReport:

OptionTypePurpose
--report-azdo-upload-artifacts <mode>off (default) | tags-only | files | allEnable build tags, file upload, or both.
--report-azdo-upload-artifact-name <name>stringOverride the artifact container name. Default: TestResults_<assemblyName>_<tfm>.
--report-azdo-upload-artifact-include <glob>repeatableFile globs (relative to TestResultsDirectory) to include. Default: **/*.
--report-azdo-upload-artifact-exclude <glob>repeatableExclusion globs. Default: none.

At session end (when TF_BUILD=true and the mode is not off), the new AzureDevOpsArtifactUploader:

  • Emits ##vso[build.addbuildtag]has-crashdump and/or has-hangdump based on the producer UID of consumed TestNodeFileArtifact events (CrashDumpProcessLifetimeHandler / HangDumpProcessLifetimeHandler — robust to localization).
  • Emits ##vso[build.addbuildtag]has-test-failures if the session contained any failed test.
  • Enumerates TestResultsDirectory via IFileSystem, applies include/exclude globs (case-sensitive on Linux, case-insensitive on Windows), and emits ##vso[artifact.upload containerfolder=…;artifactname=…]<path> for each match — properly escaping ;, \r, \n via the existing AzDoEscaper.

All output goes through IOutputDevice (no direct Console.WriteLine). All env/path/file access goes through IEnvironment / IConfiguration / IFileSystem. Exception isolation: callbacks never propagate non-cancellation exceptions out of the lifetime handler.

Highlights from the expert-reviewer round

The implementation went through one round of expert-reviewer. Critical/major issues addressed:

  • Failure isolation: all callbacks now try/catch everything except OperationCanceledException; warnings logged via ILogger.
  • Validator: rejects --report-azdo-upload-artifact-{include,exclude,name} when --report-azdo-upload-artifacts is absent or off.
  • Race-free flags: monotonic crash/hang/failure flags use Interlocked.Exchange + Volatile.Read.
  • Cross-platform paths: new PathComparison helper (OrdinalIgnoreCase on Windows, Ordinal elsewhere) applied to glob construction, prefix matching, and sort.
  • Dump detection: producer-UID-based only — no longer matches on localized DisplayName or path substring.
  • Out-of-dir filter: files resolved outside TestResultsDirectory (e.g. via symlinks) are skipped.
  • Lazy TFM resolution via new shared TargetFrameworkMonikerHelper (also adopted by the existing AzureDevOpsReporter).
  • Pre-existing typo EanbleEnable fixed in Resources/AzureDevOpsResources.resx.

Tests

  • 12 new unit tests in test/UnitTests/Microsoft.Testing.Extensions.UnitTests/AzureDevOpsArtifactUploaderTests.cs covering mode wiring (incl. off no-op), tag emission (exactly once, even under concurrent message events), glob include/exclude, escaping, missing TF_BUILD, null/whitespace TestResultsDirectory, out-of-dir filtering, and validator rejection.
  • HelpInfoAllExtensionsTests expectations updated for new options (both --help and --info blocks, alphabetical order preserved).

Build status (local)

  • .\.dotnet\dotnet.exe build src\Platform\Microsoft.Testing.Extensions.AzureDevOpsReport\Microsoft.Testing.Extensions.AzureDevOpsReport.csproj -c Debug0 warnings, 0 errors.
  • .\.dotnet\dotnet.exe test test\UnitTests\Microsoft.Testing.Extensions.UnitTests\Microsoft.Testing.Extensions.UnitTests.csproj566/566 passed.
  • .\build.cmd -pack0 warnings, 0 errors.

Out of scope (deliberate)

  • Result-level attachments (attach a dump to its failing test in the AzDO Tests tab) — depends on the REST client introduced by the live-publishing PR.
  • Code-coverage attachment under the Coverage tab — same reason.

Both can land as small follow-ups once that PR is merged.

Checklist

  • Critical & Major review findings addressed
  • Localized via resx + xlf (regenerated with /t:UpdateXlf, not hand-edited)
  • Help/info acceptance test expectations updated
  • No new public API
  • .\build.cmd green (0 warnings, 0 errors)
  • PR feedback addressed

Refs #5951

Add opt-in Azure DevOps artifact upload support, help/info updates, and unit coverage for the new reporter behavior.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
CopilotAI review requested due to automatic review settings May 16, 2026 16:33

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds an opt-in pipeline-level artifact upload + build-tag feature to Microsoft.Testing.Extensions.AzureDevOpsReport. It introduces four new CLI options that, when running under Azure DevOps (TF_BUILD=true), emit ##vso[artifact.upload …] and ##vso[build.addbuildtag]… logging commands at session end based on consumed FileArtifact/TestNodeUpdateMessage events and a glob filter over the test results directory.

Changes:

  • New AzureDevOpsArtifactUploader (data consumer + session lifetime handler) plus supporting PathComparison / TargetFrameworkMonikerHelper helpers and AzureDevOpsArtifactUploadMode enum.
  • New CLI options (--report-azdo-upload-artifacts, --report-azdo-upload-artifact-{name,include,exclude}) with validator that rejects absolute globs, unknown modes, and artifact options when uploads are off; resx/xlf strings added; pre-existing Eanble→Enable typo fixed.
  • Help/info acceptance expectations updated and 12 new unit tests covering modes, tag emission, escaping, globbing, missing TF_BUILD, and validator rejection.
Show a summary per file
FileDescription
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsArtifactUploader.csNew uploader implementing the artifact-upload + build-tag logic.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsCommandLineOptions.csNew option/mode constants.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsCommandLineProvider.csRegisters and validates the new options.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsExtensions.csRegisters the uploader as data consumer + session lifetime handler.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsReporter.csSwitched TFM resolution to shared helper.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/PathComparison.csOS-aware path string comparer/comparison.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/TargetFrameworkMonikerHelper.csShared TFM resolution helper.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Microsoft.Testing.Extensions.AzureDevOpsReport.csprojAdds Microsoft.Extensions.FileSystemGlobbing reference.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/AzureDevOpsResources.resxNew strings + Eanble→Enable typo fix.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/*.xlfRegenerated translations for all locales.
test/UnitTests/Microsoft.Testing.Extensions.UnitTests/AzureDevOpsArtifactUploaderTests.cs12 new unit tests for uploader + validator.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/HelpInfoAllExtensionsTests.csAdds AzDO extension + new option entries to help/info expectations.

Copilot's findings

  • Files reviewed: 24/24 changed files
  • Comments generated: 1

@Evangelink

Copy link
Copy Markdown
MemberAuthor

/review

@github-actions

github-actionsBot commented May 17, 2026

Copy link
Copy Markdown
Contributor

Expert Code Review (command) completed successfully!

Expert-reviewer launched in background for PR #8295. It will post the review directly.

@Evangelink
Amaury Levé (Evangelink) marked this pull request as ready for review May 17, 2026 19:13
CopilotAI review requested due to automatic review settings May 18, 2026 07:13
@Evangelink

Copy link
Copy Markdown
MemberAuthor

Copilot address review comments

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot's findings

  • Files reviewed: 24/24 changed files
  • Comments generated: 0 new

Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com>

CopilotAI commented May 18, 2026

Copy link
Copy Markdown
Contributor

Copilot address review comments

Addressed in 79f309a. I updated AzureDevOpsArtifactUploaderTests to use OS-agnostic paths (Path.Combine-based helpers) and adjusted assertions/globs accordingly so the files-mode tests pass cross-platform.

Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com>
CopilotAI review requested due to automatic review settings May 18, 2026 07:21
@Evangelink
Amaury Levé (Evangelink) removed the request for review from CopilotMay 18, 2026 07:21
Resolve the AzureDevOpsReporter merge conflict and fix net462-compatible escaping in AzureDevOpsArtifactUploaderTests.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
CopilotAI review requested due to automatic review settings May 18, 2026 19:08

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot's findings

  • Files reviewed: 24/24 changed files
  • Comments generated: 0 new

@Evangelink

Copy link
Copy Markdown
MemberAuthor

Copilot resolve the merge conflicts in this pull request

…rtifact-upload
# Conflicts:
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsCommandLineProvider.cs
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsExtensions.cs
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsReporter.cs
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Microsoft.Testing.Extensions.AzureDevOpsReport.csproj
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/AzureDevOpsResources.resx
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.cs.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.de.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.es.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.fr.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.it.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.ja.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.ko.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.pl.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.pt-BR.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.ru.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.tr.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.zh-Hans.xlf
#	src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/Resources/xlf/AzureDevOpsResources.zh-Hant.xlf
#	test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/HelpInfoAllExtensionsTests.cs
Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com>

CopilotAI commented May 19, 2026

Copy link
Copy Markdown
Contributor

Copilot resolve the merge conflicts in this pull request

Merge conflicts are resolved in commit bb8d863 by merging origin/main and reconciling the Azure DevOps report, resources, and help-info expectations.

CopilotAI review requested due to automatic review settings May 19, 2026 12:42

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot's findings

  • Files reviewed: 11/11 changed files
  • Comments generated: 0 new

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@Evangelink

Copy link
Copy Markdown
MemberAuthor

Addressed in b5bd216:

  • Cross-platform paths in tests (the only inline review comment from Copilot): already fixed in earlier 79f309a / bbe254cAzureDevOpsArtifactUploaderTests uses Path.Combine-based InResults/OutsideResults helpers rooted at Path.GetTempPath(). Verified.
  • Merge fallout from ff7261f05 (broken state on origin):
    • AzureDevOpsExtensions.cs: restored using Microsoft.Testing.Platform.Extensions; (required for CompositeExtensionFactory<>).
    • AzureDevOpsReporter.cs: removed leftover dead GetTargetFrameworkMoniker method (call site now uses TargetFrameworkMonikerHelper).
    • Resources/xlf/AzureDevOpsResources.*.xlf: regenerated via /t:UpdateXlf so all 14 locales have the 7 new state="new" trans-units that the merge dropped.

Verification:

  • .\build.cmd — Build succeeded, 0 Warning(s), 0 Error(s).
  • All 23 AzureDevOpsArtifactUploaderTests pass (full extensions unit-test suite: 174 passed, 2 skipped, 0 failed).

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@Evangelink