Reimplement source assembly-reference check without ObjectModel (Phase 6e-4c2) - #9631

Merged
Amaury Levé (Evangelink) merged 1 commit into
dev/amauryleve/vstest-decoupling-sourcehostfrom
dev/amauryleve/vstest-decoupling-sourcehandler
Jul 5, 2026
Merged

Reimplement source assembly-reference check without ObjectModel (Phase 6e-4c2)#9631
Amaury Levé (Evangelink) merged 1 commit into
dev/amauryleve/vstest-decoupling-sourcehostfrom
dev/amauryleve/vstest-decoupling-sourcehandler

Conversation

@Evangelink

Copy link
Copy Markdown
Member

Phase 6e-4c2 — reimplement the source assembly-reference check without ObjectModel

Part of the initiative to make Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices platform-agnostic by removing its dependency on Microsoft.TestPlatform.ObjectModel. VSTest coupling moves up into MSTest.TestAdapter; the platform-services engine becomes neutral. Strict byte-for-byte, no behavior change.

What this changes

TestSourceHandler.IsAssemblyReferenced (netfx-only) decided whether a source assembly references the test framework — used to skip discovery on sources that don't reference MSTest — by calling AssemblyHelper.DoesReferencesAssembly from Microsoft.VisualStudio.TestPlatform.ObjectModel.Utilities.

This replaces that call with a local neutral DoesSourceReferenceAssembly helper that reproduces the exact observable behavior of the VSTest implementation:

  • Assembly.ReflectionOnlyLoadFrom(source)GetReferencedAssemblies().
  • Match a referenced assembly by simple name (OrdinalIgnoreCase) plus public-key-token bytes; version is ignored — identical to AssemblyLoadWorker.CheckAssemblyReference.
  • Name match + public-key-token length/byte mismatch keeps scanning further references (mirrors the original continue), rather than short-circuiting.
  • Null/empty source or null reference assembly → null (undeterminable).
  • Any exception → null, so discovery proceeds conservatively.

The IsAssemblyReferenced decision line (return !utfReference.HasValue || utfReference.Value; — null-or-true ⇒ proceed, false ⇒ skip) is unchanged.

Fidelity note (dead child-AppDomain)

VSTest's DoesReferencesAssembly created a child AppDomain and an AssemblyLoadWorker instance, but then called the staticAssemblyLoadWorker.CheckAssemblyReference(...) — the worker instance is assigned and never used, and the ReflectionOnlyLoadFrom actually runs in the current domain. The child domain is therefore dead code for the result, so omitting it is behavior-preserving for every input where AppDomain.CreateDomain would have succeeded (the only theoretical divergence — a machine where domain creation throws but reflection-only load succeeds — is unreachable in practice, and the conservative null-return there would only proceed with discovery either way).

Removes the last real ObjectModel dependency from TestSourceHandler (it now carries only string-literal well-known-assembly names, which don't reference the package).

Verification

  • All real TFMs build 0-warning (UWP builds via full msbuild in CI, unaffected — change is #if NETFRAMEWORK-guarded).
  • MSTestAdapter.PlatformServices.UnitTests: 935/935 (net462), 897/897 (net8.0).
  • DesktopTestSourceTests — the direct IsAssemblyReferenced net — 7/7, covering: assembly referenced (true), not referenced (false), null name (true), null source (true).
  • PlatformServices.Desktop.IntegrationTests: 15/15 (net462).
  • Expert-reviewer pass.

Stacking

Stacks on #9630 (Phase 6e-4c1); base branch dev/amauryleve/vstest-decoupling-sourcehost. Review/merge after the earlier PRs in the chain reach the base. Do not squash-rebase the base.

…Phase 6e-4c2)
TestSourceHandler.IsAssemblyReferenced (netfx) used
AssemblyHelper.DoesReferencesAssembly from
Microsoft.VisualStudio.TestPlatform.ObjectModel.Utilities to decide whether a
source assembly references the test framework before running discovery.
Replace that call with a local neutral DoesSourceReferenceAssembly helper that
reproduces the exact observable behavior of the VSTest implementation:
- ReflectionOnlyLoadFrom(source), then GetReferencedAssemblies().
- Match a referenced assembly by simple name (OrdinalIgnoreCase) plus public
key token bytes; version is ignored -- identical to
AssemblyLoadWorker.CheckAssemblyReference.
- Null/empty source or null reference assembly returns null (undeterminable).
- Any exception returns null so discovery proceeds conservatively.
Fidelity note: the VSTest DoesReferencesAssembly created a child AppDomain and an
AssemblyLoadWorker instance, but then called the *static*
AssemblyLoadWorker.CheckAssemblyReference -- the worker instance is never used and
the ReflectionOnlyLoadFrom actually runs in the current domain. The child domain
is therefore dead code for the result, so omitting it is behavior-preserving for
every input where AppDomain creation would have succeeded.
Removes the last real ObjectModel dependency from TestSourceHandler.
Verified: PlatformServices.UnitTests 935 (net462) / 897 (net8.0);
DesktopTestSourceTests IsAssemblyReferenced branches 7/7 (referenced,
not-referenced, null-name, null-source); PlatformServices.Desktop.IntegrationTests
15/15. All real TFMs build 0-warning.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@Evangelink
Amaury Levé (Evangelink) marked this pull request as ready for review July 5, 2026 19:24
@Evangelink
Amaury Levé (Evangelink) merged commit c3c9653 into dev/amauryleve/vstest-decoupling-sourcehostJul 5, 2026
20 checks passed
@Evangelink
Amaury Levé (Evangelink) deleted the dev/amauryleve/vstest-decoupling-sourcehandler branch July 5, 2026 19:24

@github-actionsgithub-actionsBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

🤖 Automated review by GitHub Copilot. Posted via a maintainer's GitHub token, so it appears under their account — the account owner did not write or approve this content personally. Generated by the Expert Code Review workflow. To request a follow-up action, reply by tagging @copilot directly.

Review Summary — Phase 6e-4c2 (TestSourceHandler ObjectModel removal)

The architectural goal is correct and well-executed. The decoupling is clean, the #if NETFRAMEWORK guard is properly applied, and the fidelity note in the PR description is accurate about the dead child-AppDomain. One MAJOR null-safety bug in the helper method needs a follow-up fix; two MINOR polish items are noted.


Verdict: NEEDS WORK

#DimensionStatusSeverityNote
1Algorithmic Correctness⚠️ MinorMAJORLogic is correct for signed assemblies (the production case). For unsigned assemblies (null PKT), ArePublicKeyTokensEqual throws NRE → caught → null → proceed. Accidentally correct outcome, wrong code path.
2Threading & Concurrency✅ CleanStatic method, local variables only, no shared state.
3Security & IPC Contract Safety✅ CleanReflectionOnlyLoadFrom is read-only; no code execution; source path supplied by caller.
4Public API & Binary Compatibility✅ CleanNo public API changes; removed using for ObjectModel is a positive dependency reduction.
5Memory Management & Resource Leaks✅ CleanReflection-only load context limitations are pre-existing; behaviour unchanged from original call.
6Error Handling⚠️ MinorMINORBare catch absorbs all exceptions silently; no diagnostic trace makes debugging difficult (see inline comment on line 135).
7Naming & Code Style✅ CleanDoesSourceReferenceAssembly, ArePublicKeyTokensEqual are clear verb-led names; file-scoped namespace, no abbreviations.
8Documentation & Comments⚠️ MinorMINORXML doc on DoesSourceReferenceAssembly is accurate. Stale comment on line 78 ("different app domain") is now inaccurate (see inline comment).
9Test Coverage✅ Clean7/7 DesktopTestSourceTests reported; edge cases (null name, null source, referenced/not-referenced) are covered per PR description. Null-PKT path not explicitly tested but the outer catch provides the safety net.
10Performance✅ CleanStringComparison.OrdinalIgnoreCase used; no unnecessary allocations in the loop.
11Cross-TFM Correctness✅ CleanAll new code is inside #if NETFRAMEWORK; Assembly.ReflectionOnlyLoadFrom is .NET Framework-only.
12LocalizationN/ANo user-facing strings added.
13Dependency Management✅ CleanPositive change: removes the ObjectModel using directive from PlatformServices.
14Telemetry & ObservabilityN/ANo new user-visible behaviour; existing discovery flow preserved.
15CI/CD & Build System✅ CleanNo build script changes; guarded by #if NETFRAMEWORK.
16Backward Compatibility✅ CleanIsAssemblyReferenced return logic is unchanged; callers see identical results for all signed-assembly inputs.
17Null Safety❌ IssueMAJORGetPublicKeyToken() returns null on .NET Framework for unsigned assemblies. ArePublicKeyTokensEqual(byte[] left, byte[] right) has no null guards; left.Length throws NullReferenceException. referenceAssemblyPublicKeyToken declared byte[] rather than byte[]?, hiding the nullable concern. See inline comment on line 144.
18Exception Safety✅ CleanNo partial-state mutations; catch is for observation, not rollback.
19Immutability & Value Semantics✅ CleanPure static methods operating on immutable inputs.
20Design Patterns & Architecture✅ CleanAligns perfectly with the stated goal; dependency removed at the correct layer.
21Logging Correctness⚠️ MinorMINORNo diagnostic output when the catch fires; a Debug.WriteLine would cost nothing and aid debugging.
22Scope & PR Hygiene✅ CleanSingle concern; no unrelated changes; no dead code introduced.

Required follow-up

ArePublicKeyTokensEqual — add null guards (line 144, inline comment posted):

privatestaticboolArePublicKeyTokensEqual(byte[]?left,byte[]?right){if(leftisnull&&rightisnull)returntrue;if(leftisnull||rightisnull)returnfalse;if(left.Length!=right.Length)returnfalse;for(inti=0;i<left.Length;++i){if(left[i]!=right[i])returnfalse;}returntrue;}

Also fix the assignment on line 113 to byte[]? referenceAssemblyPublicKeyToken = referenceAssembly.GetPublicKeyToken();.

In practice MSTest is always shipped signed so null PKTs will not occur in the field, but the code is objectively wrong and should be fixed before being built upon in the next phase of this decoupling series.


private static bool ArePublicKeyTokensEqual(byte[] left, byte[] right)
{
if (left.Length != right.Length)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MAJOR — Null Safety]GetPublicKeyToken() returns null on .NET Framework for unsigned assemblies (those with no public key token). Both left (from referencedAssembly.GetPublicKeyToken()) and right (from referenceAssembly.GetPublicKeyToken()) can therefore be null, so left.Length on this line throws NullReferenceException. The outer catch absorbs that and returns null (→ proceed with discovery), so the process doesn't crash; but there are two problems:

  1. For two assemblies that both carry no public-key token, the semantically correct result is true (name match + neither is signed = same unsigned identity). Instead the caller gets a swallowed exception and falls through to "proceed" via null, which is accidentally correct today but for the wrong reason.
  2. referenceAssemblyPublicKeyToken on line 113 is declared byte[] (non-nullable), masking the actual nullable return of GetPublicKeyToken(). It should be byte[]?.

Suggested fix:

privatestaticboolArePublicKeyTokensEqual(byte[]?left,byte[]?right){if(leftisnull&&rightisnull)returntrue;if(leftisnull||rightisnull)returnfalse;if(left.Length!=right.Length)returnfalse;for(inti=0;i<left.Length;++i){if(left[i]!=right[i])returnfalse;}returntrue;}

Also update the assignment on line 113:

byte[]?referenceAssemblyPublicKeyToken=referenceAssembly.GetPublicKeyToken();


return false;
}
catch

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MINOR — Bare Catch / No Diagnostics] The catch block silently swallows every exception — file-not-found, BadImageFormatException, security exceptions, and the null-deref discussed on ArePublicKeyTokensEqual all land here invisibly. The conservative null return is intentional and correct, but without any trace output it is very hard to tell the difference between "file legitimately couldn't be opened" and "code bug".

Suggestion — at minimum emit a Debug.WriteLine (no user-visible surface, zero overhead in production):

catch(Exceptionex){Debug.WriteLine($"[MSTest] DoesSourceReferenceAssembly could not inspect '{source}': {ex.Message}");returnnull;}

@@ -79,7 +76,7 @@ public bool IsAssemblyReferenced(AssemblyName assemblyName, string source)
{
#if NETFRAMEWORK
// This loads the dll in a different app domain. We can optimize this to load in the current domain since this code could be run in a new app domain anyway.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MINOR — Stale Comment] The comment says "This loads the dll in a different app domain", but the new implementation (DoesSourceReferenceAssembly) uses Assembly.ReflectionOnlyLoadFrom in the current AppDomain — no child domain is created. Both sentences are now inaccurate (the second was an optimisation note that is now moot since the domain split was intentionally dropped).

Suggested replacement:

// Reflection-only loads the assembly to inspect its references without executing any code from it.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs/author-feedbackWaiting on the original author.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@Evangelink
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

Reimplement source assembly-reference check without ObjectModel (Phase 6e-4c2) - #9631

Merged
Amaury Levé (Evangelink) merged 1 commit into
dev/amauryleve/vstest-decoupling-sourcehostfrom
dev/amauryleve/vstest-decoupling-sourcehandler
Jul 5, 2026
Merged

Reimplement source assembly-reference check without ObjectModel (Phase 6e-4c2)#9631
Amaury Levé (Evangelink) merged 1 commit into
dev/amauryleve/vstest-decoupling-sourcehostfrom
dev/amauryleve/vstest-decoupling-sourcehandler

Conversation

@Evangelink

Copy link
Copy Markdown
Member

Phase 6e-4c2 — reimplement the source assembly-reference check without ObjectModel

Part of the initiative to make Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices platform-agnostic by removing its dependency on Microsoft.TestPlatform.ObjectModel. VSTest coupling moves up into MSTest.TestAdapter; the platform-services engine becomes neutral. Strict byte-for-byte, no behavior change.

What this changes

TestSourceHandler.IsAssemblyReferenced (netfx-only) decided whether a source assembly references the test framework — used to skip discovery on sources that don't reference MSTest — by calling AssemblyHelper.DoesReferencesAssembly from Microsoft.VisualStudio.TestPlatform.ObjectModel.Utilities.

This replaces that call with a local neutral DoesSourceReferenceAssembly helper that reproduces the exact observable behavior of the VSTest implementation:

  • Assembly.ReflectionOnlyLoadFrom(source)GetReferencedAssemblies().
  • Match a referenced assembly by simple name (OrdinalIgnoreCase) plus public-key-token bytes; version is ignored — identical to AssemblyLoadWorker.CheckAssemblyReference.
  • Name match + public-key-token length/byte mismatch keeps scanning further references (mirrors the original continue), rather than short-circuiting.
  • Null/empty source or null reference assembly → null (undeterminable).
  • Any exception → null, so discovery proceeds conservatively.

The IsAssemblyReferenced decision line (return !utfReference.HasValue || utfReference.Value; — null-or-true ⇒ proceed, false ⇒ skip) is unchanged.

Fidelity note (dead child-AppDomain)

VSTest's DoesReferencesAssembly created a child AppDomain and an AssemblyLoadWorker instance, but then called the staticAssemblyLoadWorker.CheckAssemblyReference(...) — the worker instance is assigned and never used, and the ReflectionOnlyLoadFrom actually runs in the current domain. The child domain is therefore dead code for the result, so omitting it is behavior-preserving for every input where AppDomain.CreateDomain would have succeeded (the only theoretical divergence — a machine where domain creation throws but reflection-only load succeeds — is unreachable in practice, and the conservative null-return there would only proceed with discovery either way).

Removes the last real ObjectModel dependency from TestSourceHandler (it now carries only string-literal well-known-assembly names, which don't reference the package).

Verification

  • All real TFMs build 0-warning (UWP builds via full msbuild in CI, unaffected — change is #if NETFRAMEWORK-guarded).
  • MSTestAdapter.PlatformServices.UnitTests: 935/935 (net462), 897/897 (net8.0).
  • DesktopTestSourceTests — the direct IsAssemblyReferenced net — 7/7, covering: assembly referenced (true), not referenced (false), null name (true), null source (true).
  • PlatformServices.Desktop.IntegrationTests: 15/15 (net462).
  • Expert-reviewer pass.

Stacking

Stacks on #9630 (Phase 6e-4c1); base branch dev/amauryleve/vstest-decoupling-sourcehost. Review/merge after the earlier PRs in the chain reach the base. Do not squash-rebase the base.

…Phase 6e-4c2)
TestSourceHandler.IsAssemblyReferenced (netfx) used
AssemblyHelper.DoesReferencesAssembly from
Microsoft.VisualStudio.TestPlatform.ObjectModel.Utilities to decide whether a
source assembly references the test framework before running discovery.
Replace that call with a local neutral DoesSourceReferenceAssembly helper that
reproduces the exact observable behavior of the VSTest implementation:
- ReflectionOnlyLoadFrom(source), then GetReferencedAssemblies().
- Match a referenced assembly by simple name (OrdinalIgnoreCase) plus public
key token bytes; version is ignored -- identical to
AssemblyLoadWorker.CheckAssemblyReference.
- Null/empty source or null reference assembly returns null (undeterminable).
- Any exception returns null so discovery proceeds conservatively.
Fidelity note: the VSTest DoesReferencesAssembly created a child AppDomain and an
AssemblyLoadWorker instance, but then called the *static*
AssemblyLoadWorker.CheckAssemblyReference -- the worker instance is never used and
the ReflectionOnlyLoadFrom actually runs in the current domain. The child domain
is therefore dead code for the result, so omitting it is behavior-preserving for
every input where AppDomain creation would have succeeded.
Removes the last real ObjectModel dependency from TestSourceHandler.
Verified: PlatformServices.UnitTests 935 (net462) / 897 (net8.0);
DesktopTestSourceTests IsAssemblyReferenced branches 7/7 (referenced,
not-referenced, null-name, null-source); PlatformServices.Desktop.IntegrationTests
15/15. All real TFMs build 0-warning.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@Evangelink
Amaury Levé (Evangelink) marked this pull request as ready for review July 5, 2026 19:24
@Evangelink
Amaury Levé (Evangelink) merged commit c3c9653 into dev/amauryleve/vstest-decoupling-sourcehostJul 5, 2026
20 checks passed
@Evangelink
Amaury Levé (Evangelink) deleted the dev/amauryleve/vstest-decoupling-sourcehandler branch July 5, 2026 19:24

@github-actionsgithub-actionsBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

🤖 Automated review by GitHub Copilot. Posted via a maintainer's GitHub token, so it appears under their account — the account owner did not write or approve this content personally. Generated by the Expert Code Review workflow. To request a follow-up action, reply by tagging @copilot directly.

Review Summary — Phase 6e-4c2 (TestSourceHandler ObjectModel removal)

The architectural goal is correct and well-executed. The decoupling is clean, the #if NETFRAMEWORK guard is properly applied, and the fidelity note in the PR description is accurate about the dead child-AppDomain. One MAJOR null-safety bug in the helper method needs a follow-up fix; two MINOR polish items are noted.


Verdict: NEEDS WORK

#DimensionStatusSeverityNote
1Algorithmic Correctness⚠️ MinorMAJORLogic is correct for signed assemblies (the production case). For unsigned assemblies (null PKT), ArePublicKeyTokensEqual throws NRE → caught → null → proceed. Accidentally correct outcome, wrong code path.
2Threading & Concurrency✅ CleanStatic method, local variables only, no shared state.
3Security & IPC Contract Safety✅ CleanReflectionOnlyLoadFrom is read-only; no code execution; source path supplied by caller.
4Public API & Binary Compatibility✅ CleanNo public API changes; removed using for ObjectModel is a positive dependency reduction.
5Memory Management & Resource Leaks✅ CleanReflection-only load context limitations are pre-existing; behaviour unchanged from original call.
6Error Handling⚠️ MinorMINORBare catch absorbs all exceptions silently; no diagnostic trace makes debugging difficult (see inline comment on line 135).
7Naming & Code Style✅ CleanDoesSourceReferenceAssembly, ArePublicKeyTokensEqual are clear verb-led names; file-scoped namespace, no abbreviations.
8Documentation & Comments⚠️ MinorMINORXML doc on DoesSourceReferenceAssembly is accurate. Stale comment on line 78 ("different app domain") is now inaccurate (see inline comment).
9Test Coverage✅ Clean7/7 DesktopTestSourceTests reported; edge cases (null name, null source, referenced/not-referenced) are covered per PR description. Null-PKT path not explicitly tested but the outer catch provides the safety net.
10Performance✅ CleanStringComparison.OrdinalIgnoreCase used; no unnecessary allocations in the loop.
11Cross-TFM Correctness✅ CleanAll new code is inside #if NETFRAMEWORK; Assembly.ReflectionOnlyLoadFrom is .NET Framework-only.
12LocalizationN/ANo user-facing strings added.
13Dependency Management✅ CleanPositive change: removes the ObjectModel using directive from PlatformServices.
14Telemetry & ObservabilityN/ANo new user-visible behaviour; existing discovery flow preserved.
15CI/CD & Build System✅ CleanNo build script changes; guarded by #if NETFRAMEWORK.
16Backward Compatibility✅ CleanIsAssemblyReferenced return logic is unchanged; callers see identical results for all signed-assembly inputs.
17Null Safety❌ IssueMAJORGetPublicKeyToken() returns null on .NET Framework for unsigned assemblies. ArePublicKeyTokensEqual(byte[] left, byte[] right) has no null guards; left.Length throws NullReferenceException. referenceAssemblyPublicKeyToken declared byte[] rather than byte[]?, hiding the nullable concern. See inline comment on line 144.
18Exception Safety✅ CleanNo partial-state mutations; catch is for observation, not rollback.
19Immutability & Value Semantics✅ CleanPure static methods operating on immutable inputs.
20Design Patterns & Architecture✅ CleanAligns perfectly with the stated goal; dependency removed at the correct layer.
21Logging Correctness⚠️ MinorMINORNo diagnostic output when the catch fires; a Debug.WriteLine would cost nothing and aid debugging.
22Scope & PR Hygiene✅ CleanSingle concern; no unrelated changes; no dead code introduced.

Required follow-up

ArePublicKeyTokensEqual — add null guards (line 144, inline comment posted):

privatestaticboolArePublicKeyTokensEqual(byte[]?left,byte[]?right){if(leftisnull&&rightisnull)returntrue;if(leftisnull||rightisnull)returnfalse;if(left.Length!=right.Length)returnfalse;for(inti=0;i<left.Length;++i){if(left[i]!=right[i])returnfalse;}returntrue;}

Also fix the assignment on line 113 to byte[]? referenceAssemblyPublicKeyToken = referenceAssembly.GetPublicKeyToken();.

In practice MSTest is always shipped signed so null PKTs will not occur in the field, but the code is objectively wrong and should be fixed before being built upon in the next phase of this decoupling series.


private static bool ArePublicKeyTokensEqual(byte[] left, byte[] right)
{
if (left.Length != right.Length)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MAJOR — Null Safety]GetPublicKeyToken() returns null on .NET Framework for unsigned assemblies (those with no public key token). Both left (from referencedAssembly.GetPublicKeyToken()) and right (from referenceAssembly.GetPublicKeyToken()) can therefore be null, so left.Length on this line throws NullReferenceException. The outer catch absorbs that and returns null (→ proceed with discovery), so the process doesn't crash; but there are two problems:

  1. For two assemblies that both carry no public-key token, the semantically correct result is true (name match + neither is signed = same unsigned identity). Instead the caller gets a swallowed exception and falls through to "proceed" via null, which is accidentally correct today but for the wrong reason.
  2. referenceAssemblyPublicKeyToken on line 113 is declared byte[] (non-nullable), masking the actual nullable return of GetPublicKeyToken(). It should be byte[]?.

Suggested fix:

privatestaticboolArePublicKeyTokensEqual(byte[]?left,byte[]?right){if(leftisnull&&rightisnull)returntrue;if(leftisnull||rightisnull)returnfalse;if(left.Length!=right.Length)returnfalse;for(inti=0;i<left.Length;++i){if(left[i]!=right[i])returnfalse;}returntrue;}

Also update the assignment on line 113:

byte[]?referenceAssemblyPublicKeyToken=referenceAssembly.GetPublicKeyToken();


return false;
}
catch

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MINOR — Bare Catch / No Diagnostics] The catch block silently swallows every exception — file-not-found, BadImageFormatException, security exceptions, and the null-deref discussed on ArePublicKeyTokensEqual all land here invisibly. The conservative null return is intentional and correct, but without any trace output it is very hard to tell the difference between "file legitimately couldn't be opened" and "code bug".

Suggestion — at minimum emit a Debug.WriteLine (no user-visible surface, zero overhead in production):

catch(Exceptionex){Debug.WriteLine($"[MSTest] DoesSourceReferenceAssembly could not inspect '{source}': {ex.Message}");returnnull;}

@@ -79,7 +76,7 @@ public bool IsAssemblyReferenced(AssemblyName assemblyName, string source)
{
#if NETFRAMEWORK
// This loads the dll in a different app domain. We can optimize this to load in the current domain since this code could be run in a new app domain anyway.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MINOR — Stale Comment] The comment says "This loads the dll in a different app domain", but the new implementation (DoesSourceReferenceAssembly) uses Assembly.ReflectionOnlyLoadFrom in the current AppDomain — no child domain is created. Both sentences are now inaccurate (the second was an optimisation note that is now moot since the domain split was intentionally dropped).

Suggested replacement:

// Reflection-only loads the assembly to inspect its references without executing any code from it.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs/author-feedbackWaiting on the original author.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@Evangelink
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Reimplement source assembly-reference check without ObjectModel (Phase 6e-4c2) - #9631

Merged
Amaury Levé (Evangelink) merged 1 commit into
dev/amauryleve/vstest-decoupling-sourcehostfrom
dev/amauryleve/vstest-decoupling-sourcehandler
Jul 5, 2026
Merged

Reimplement source assembly-reference check without ObjectModel (Phase 6e-4c2)#9631
Amaury Levé (Evangelink) merged 1 commit into
dev/amauryleve/vstest-decoupling-sourcehostfrom
dev/amauryleve/vstest-decoupling-sourcehandler

Conversation

@Evangelink

Copy link
Copy Markdown
Member

Phase 6e-4c2 — reimplement the source assembly-reference check without ObjectModel

Part of the initiative to make Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices platform-agnostic by removing its dependency on Microsoft.TestPlatform.ObjectModel. VSTest coupling moves up into MSTest.TestAdapter; the platform-services engine becomes neutral. Strict byte-for-byte, no behavior change.

What this changes

TestSourceHandler.IsAssemblyReferenced (netfx-only) decided whether a source assembly references the test framework — used to skip discovery on sources that don't reference MSTest — by calling AssemblyHelper.DoesReferencesAssembly from Microsoft.VisualStudio.TestPlatform.ObjectModel.Utilities.

This replaces that call with a local neutral DoesSourceReferenceAssembly helper that reproduces the exact observable behavior of the VSTest implementation:

  • Assembly.ReflectionOnlyLoadFrom(source)GetReferencedAssemblies().
  • Match a referenced assembly by simple name (OrdinalIgnoreCase) plus public-key-token bytes; version is ignored — identical to AssemblyLoadWorker.CheckAssemblyReference.
  • Name match + public-key-token length/byte mismatch keeps scanning further references (mirrors the original continue), rather than short-circuiting.
  • Null/empty source or null reference assembly → null (undeterminable).
  • Any exception → null, so discovery proceeds conservatively.

The IsAssemblyReferenced decision line (return !utfReference.HasValue || utfReference.Value; — null-or-true ⇒ proceed, false ⇒ skip) is unchanged.

Fidelity note (dead child-AppDomain)

VSTest's DoesReferencesAssembly created a child AppDomain and an AssemblyLoadWorker instance, but then called the staticAssemblyLoadWorker.CheckAssemblyReference(...) — the worker instance is assigned and never used, and the ReflectionOnlyLoadFrom actually runs in the current domain. The child domain is therefore dead code for the result, so omitting it is behavior-preserving for every input where AppDomain.CreateDomain would have succeeded (the only theoretical divergence — a machine where domain creation throws but reflection-only load succeeds — is unreachable in practice, and the conservative null-return there would only proceed with discovery either way).

Removes the last real ObjectModel dependency from TestSourceHandler (it now carries only string-literal well-known-assembly names, which don't reference the package).

Verification

  • All real TFMs build 0-warning (UWP builds via full msbuild in CI, unaffected — change is #if NETFRAMEWORK-guarded).
  • MSTestAdapter.PlatformServices.UnitTests: 935/935 (net462), 897/897 (net8.0).
  • DesktopTestSourceTests — the direct IsAssemblyReferenced net — 7/7, covering: assembly referenced (true), not referenced (false), null name (true), null source (true).
  • PlatformServices.Desktop.IntegrationTests: 15/15 (net462).
  • Expert-reviewer pass.

Stacking

Stacks on #9630 (Phase 6e-4c1); base branch dev/amauryleve/vstest-decoupling-sourcehost. Review/merge after the earlier PRs in the chain reach the base. Do not squash-rebase the base.

…Phase 6e-4c2)
TestSourceHandler.IsAssemblyReferenced (netfx) used
AssemblyHelper.DoesReferencesAssembly from
Microsoft.VisualStudio.TestPlatform.ObjectModel.Utilities to decide whether a
source assembly references the test framework before running discovery.
Replace that call with a local neutral DoesSourceReferenceAssembly helper that
reproduces the exact observable behavior of the VSTest implementation:
- ReflectionOnlyLoadFrom(source), then GetReferencedAssemblies().
- Match a referenced assembly by simple name (OrdinalIgnoreCase) plus public
key token bytes; version is ignored -- identical to
AssemblyLoadWorker.CheckAssemblyReference.
- Null/empty source or null reference assembly returns null (undeterminable).
- Any exception returns null so discovery proceeds conservatively.
Fidelity note: the VSTest DoesReferencesAssembly created a child AppDomain and an
AssemblyLoadWorker instance, but then called the *static*
AssemblyLoadWorker.CheckAssemblyReference -- the worker instance is never used and
the ReflectionOnlyLoadFrom actually runs in the current domain. The child domain
is therefore dead code for the result, so omitting it is behavior-preserving for
every input where AppDomain creation would have succeeded.
Removes the last real ObjectModel dependency from TestSourceHandler.
Verified: PlatformServices.UnitTests 935 (net462) / 897 (net8.0);
DesktopTestSourceTests IsAssemblyReferenced branches 7/7 (referenced,
not-referenced, null-name, null-source); PlatformServices.Desktop.IntegrationTests
15/15. All real TFMs build 0-warning.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@Evangelink
Amaury Levé (Evangelink) marked this pull request as ready for review July 5, 2026 19:24
@Evangelink
Amaury Levé (Evangelink) merged commit c3c9653 into dev/amauryleve/vstest-decoupling-sourcehostJul 5, 2026
20 checks passed
@Evangelink
Amaury Levé (Evangelink) deleted the dev/amauryleve/vstest-decoupling-sourcehandler branch July 5, 2026 19:24

@github-actionsgithub-actionsBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

🤖 Automated review by GitHub Copilot. Posted via a maintainer's GitHub token, so it appears under their account — the account owner did not write or approve this content personally. Generated by the Expert Code Review workflow. To request a follow-up action, reply by tagging @copilot directly.

Review Summary — Phase 6e-4c2 (TestSourceHandler ObjectModel removal)

The architectural goal is correct and well-executed. The decoupling is clean, the #if NETFRAMEWORK guard is properly applied, and the fidelity note in the PR description is accurate about the dead child-AppDomain. One MAJOR null-safety bug in the helper method needs a follow-up fix; two MINOR polish items are noted.


Verdict: NEEDS WORK

#DimensionStatusSeverityNote
1Algorithmic Correctness⚠️ MinorMAJORLogic is correct for signed assemblies (the production case). For unsigned assemblies (null PKT), ArePublicKeyTokensEqual throws NRE → caught → null → proceed. Accidentally correct outcome, wrong code path.
2Threading & Concurrency✅ CleanStatic method, local variables only, no shared state.
3Security & IPC Contract Safety✅ CleanReflectionOnlyLoadFrom is read-only; no code execution; source path supplied by caller.
4Public API & Binary Compatibility✅ CleanNo public API changes; removed using for ObjectModel is a positive dependency reduction.
5Memory Management & Resource Leaks✅ CleanReflection-only load context limitations are pre-existing; behaviour unchanged from original call.
6Error Handling⚠️ MinorMINORBare catch absorbs all exceptions silently; no diagnostic trace makes debugging difficult (see inline comment on line 135).
7Naming & Code Style✅ CleanDoesSourceReferenceAssembly, ArePublicKeyTokensEqual are clear verb-led names; file-scoped namespace, no abbreviations.
8Documentation & Comments⚠️ MinorMINORXML doc on DoesSourceReferenceAssembly is accurate. Stale comment on line 78 ("different app domain") is now inaccurate (see inline comment).
9Test Coverage✅ Clean7/7 DesktopTestSourceTests reported; edge cases (null name, null source, referenced/not-referenced) are covered per PR description. Null-PKT path not explicitly tested but the outer catch provides the safety net.
10Performance✅ CleanStringComparison.OrdinalIgnoreCase used; no unnecessary allocations in the loop.
11Cross-TFM Correctness✅ CleanAll new code is inside #if NETFRAMEWORK; Assembly.ReflectionOnlyLoadFrom is .NET Framework-only.
12LocalizationN/ANo user-facing strings added.
13Dependency Management✅ CleanPositive change: removes the ObjectModel using directive from PlatformServices.
14Telemetry & ObservabilityN/ANo new user-visible behaviour; existing discovery flow preserved.
15CI/CD & Build System✅ CleanNo build script changes; guarded by #if NETFRAMEWORK.
16Backward Compatibility✅ CleanIsAssemblyReferenced return logic is unchanged; callers see identical results for all signed-assembly inputs.
17Null Safety❌ IssueMAJORGetPublicKeyToken() returns null on .NET Framework for unsigned assemblies. ArePublicKeyTokensEqual(byte[] left, byte[] right) has no null guards; left.Length throws NullReferenceException. referenceAssemblyPublicKeyToken declared byte[] rather than byte[]?, hiding the nullable concern. See inline comment on line 144.
18Exception Safety✅ CleanNo partial-state mutations; catch is for observation, not rollback.
19Immutability & Value Semantics✅ CleanPure static methods operating on immutable inputs.
20Design Patterns & Architecture✅ CleanAligns perfectly with the stated goal; dependency removed at the correct layer.
21Logging Correctness⚠️ MinorMINORNo diagnostic output when the catch fires; a Debug.WriteLine would cost nothing and aid debugging.
22Scope & PR Hygiene✅ CleanSingle concern; no unrelated changes; no dead code introduced.

Required follow-up

ArePublicKeyTokensEqual — add null guards (line 144, inline comment posted):

privatestaticboolArePublicKeyTokensEqual(byte[]?left,byte[]?right){if(leftisnull&&rightisnull)returntrue;if(leftisnull||rightisnull)returnfalse;if(left.Length!=right.Length)returnfalse;for(inti=0;i<left.Length;++i){if(left[i]!=right[i])returnfalse;}returntrue;}

Also fix the assignment on line 113 to byte[]? referenceAssemblyPublicKeyToken = referenceAssembly.GetPublicKeyToken();.

In practice MSTest is always shipped signed so null PKTs will not occur in the field, but the code is objectively wrong and should be fixed before being built upon in the next phase of this decoupling series.


private static bool ArePublicKeyTokensEqual(byte[] left, byte[] right)
{
if (left.Length != right.Length)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MAJOR — Null Safety]GetPublicKeyToken() returns null on .NET Framework for unsigned assemblies (those with no public key token). Both left (from referencedAssembly.GetPublicKeyToken()) and right (from referenceAssembly.GetPublicKeyToken()) can therefore be null, so left.Length on this line throws NullReferenceException. The outer catch absorbs that and returns null (→ proceed with discovery), so the process doesn't crash; but there are two problems:

  1. For two assemblies that both carry no public-key token, the semantically correct result is true (name match + neither is signed = same unsigned identity). Instead the caller gets a swallowed exception and falls through to "proceed" via null, which is accidentally correct today but for the wrong reason.
  2. referenceAssemblyPublicKeyToken on line 113 is declared byte[] (non-nullable), masking the actual nullable return of GetPublicKeyToken(). It should be byte[]?.

Suggested fix:

privatestaticboolArePublicKeyTokensEqual(byte[]?left,byte[]?right){if(leftisnull&&rightisnull)returntrue;if(leftisnull||rightisnull)returnfalse;if(left.Length!=right.Length)returnfalse;for(inti=0;i<left.Length;++i){if(left[i]!=right[i])returnfalse;}returntrue;}

Also update the assignment on line 113:

byte[]?referenceAssemblyPublicKeyToken=referenceAssembly.GetPublicKeyToken();


return false;
}
catch

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MINOR — Bare Catch / No Diagnostics] The catch block silently swallows every exception — file-not-found, BadImageFormatException, security exceptions, and the null-deref discussed on ArePublicKeyTokensEqual all land here invisibly. The conservative null return is intentional and correct, but without any trace output it is very hard to tell the difference between "file legitimately couldn't be opened" and "code bug".

Suggestion — at minimum emit a Debug.WriteLine (no user-visible surface, zero overhead in production):

catch(Exceptionex){Debug.WriteLine($"[MSTest] DoesSourceReferenceAssembly could not inspect '{source}': {ex.Message}");returnnull;}

@@ -79,7 +76,7 @@ public bool IsAssemblyReferenced(AssemblyName assemblyName, string source)
{
#if NETFRAMEWORK
// This loads the dll in a different app domain. We can optimize this to load in the current domain since this code could be run in a new app domain anyway.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MINOR — Stale Comment] The comment says "This loads the dll in a different app domain", but the new implementation (DoesSourceReferenceAssembly) uses Assembly.ReflectionOnlyLoadFrom in the current AppDomain — no child domain is created. Both sentences are now inaccurate (the second was an optimisation note that is now moot since the domain split was intentionally dropped).

Suggested replacement:

// Reflection-only loads the assembly to inspect its references without executing any code from it.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs/author-feedbackWaiting on the original author.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@Evangelink
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Reimplement source assembly-reference check without ObjectModel (Phase 6e-4c2) - #9631

Merged
Amaury Levé (Evangelink) merged 1 commit into
dev/amauryleve/vstest-decoupling-sourcehostfrom
dev/amauryleve/vstest-decoupling-sourcehandler
Jul 5, 2026
Merged

Reimplement source assembly-reference check without ObjectModel (Phase 6e-4c2)#9631
Amaury Levé (Evangelink) merged 1 commit into
dev/amauryleve/vstest-decoupling-sourcehostfrom
dev/amauryleve/vstest-decoupling-sourcehandler

Conversation

@Evangelink

Copy link
Copy Markdown
Member

Phase 6e-4c2 — reimplement the source assembly-reference check without ObjectModel

Part of the initiative to make Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices platform-agnostic by removing its dependency on Microsoft.TestPlatform.ObjectModel. VSTest coupling moves up into MSTest.TestAdapter; the platform-services engine becomes neutral. Strict byte-for-byte, no behavior change.

What this changes

TestSourceHandler.IsAssemblyReferenced (netfx-only) decided whether a source assembly references the test framework — used to skip discovery on sources that don't reference MSTest — by calling AssemblyHelper.DoesReferencesAssembly from Microsoft.VisualStudio.TestPlatform.ObjectModel.Utilities.

This replaces that call with a local neutral DoesSourceReferenceAssembly helper that reproduces the exact observable behavior of the VSTest implementation:

  • Assembly.ReflectionOnlyLoadFrom(source)GetReferencedAssemblies().
  • Match a referenced assembly by simple name (OrdinalIgnoreCase) plus public-key-token bytes; version is ignored — identical to AssemblyLoadWorker.CheckAssemblyReference.
  • Name match + public-key-token length/byte mismatch keeps scanning further references (mirrors the original continue), rather than short-circuiting.
  • Null/empty source or null reference assembly → null (undeterminable).
  • Any exception → null, so discovery proceeds conservatively.

The IsAssemblyReferenced decision line (return !utfReference.HasValue || utfReference.Value; — null-or-true ⇒ proceed, false ⇒ skip) is unchanged.

Fidelity note (dead child-AppDomain)

VSTest's DoesReferencesAssembly created a child AppDomain and an AssemblyLoadWorker instance, but then called the staticAssemblyLoadWorker.CheckAssemblyReference(...) — the worker instance is assigned and never used, and the ReflectionOnlyLoadFrom actually runs in the current domain. The child domain is therefore dead code for the result, so omitting it is behavior-preserving for every input where AppDomain.CreateDomain would have succeeded (the only theoretical divergence — a machine where domain creation throws but reflection-only load succeeds — is unreachable in practice, and the conservative null-return there would only proceed with discovery either way).

Removes the last real ObjectModel dependency from TestSourceHandler (it now carries only string-literal well-known-assembly names, which don't reference the package).

Verification

  • All real TFMs build 0-warning (UWP builds via full msbuild in CI, unaffected — change is #if NETFRAMEWORK-guarded).
  • MSTestAdapter.PlatformServices.UnitTests: 935/935 (net462), 897/897 (net8.0).
  • DesktopTestSourceTests — the direct IsAssemblyReferenced net — 7/7, covering: assembly referenced (true), not referenced (false), null name (true), null source (true).
  • PlatformServices.Desktop.IntegrationTests: 15/15 (net462).
  • Expert-reviewer pass.

Stacking

Stacks on #9630 (Phase 6e-4c1); base branch dev/amauryleve/vstest-decoupling-sourcehost. Review/merge after the earlier PRs in the chain reach the base. Do not squash-rebase the base.

…Phase 6e-4c2)
TestSourceHandler.IsAssemblyReferenced (netfx) used
AssemblyHelper.DoesReferencesAssembly from
Microsoft.VisualStudio.TestPlatform.ObjectModel.Utilities to decide whether a
source assembly references the test framework before running discovery.
Replace that call with a local neutral DoesSourceReferenceAssembly helper that
reproduces the exact observable behavior of the VSTest implementation:
- ReflectionOnlyLoadFrom(source), then GetReferencedAssemblies().
- Match a referenced assembly by simple name (OrdinalIgnoreCase) plus public
key token bytes; version is ignored -- identical to
AssemblyLoadWorker.CheckAssemblyReference.
- Null/empty source or null reference assembly returns null (undeterminable).
- Any exception returns null so discovery proceeds conservatively.
Fidelity note: the VSTest DoesReferencesAssembly created a child AppDomain and an
AssemblyLoadWorker instance, but then called the *static*
AssemblyLoadWorker.CheckAssemblyReference -- the worker instance is never used and
the ReflectionOnlyLoadFrom actually runs in the current domain. The child domain
is therefore dead code for the result, so omitting it is behavior-preserving for
every input where AppDomain creation would have succeeded.
Removes the last real ObjectModel dependency from TestSourceHandler.
Verified: PlatformServices.UnitTests 935 (net462) / 897 (net8.0);
DesktopTestSourceTests IsAssemblyReferenced branches 7/7 (referenced,
not-referenced, null-name, null-source); PlatformServices.Desktop.IntegrationTests
15/15. All real TFMs build 0-warning.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@Evangelink
Amaury Levé (Evangelink) marked this pull request as ready for review July 5, 2026 19:24
@Evangelink
Amaury Levé (Evangelink) merged commit c3c9653 into dev/amauryleve/vstest-decoupling-sourcehostJul 5, 2026
20 checks passed
@Evangelink
Amaury Levé (Evangelink) deleted the dev/amauryleve/vstest-decoupling-sourcehandler branch July 5, 2026 19:24

@github-actionsgithub-actionsBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

🤖 Automated review by GitHub Copilot. Posted via a maintainer's GitHub token, so it appears under their account — the account owner did not write or approve this content personally. Generated by the Expert Code Review workflow. To request a follow-up action, reply by tagging @copilot directly.

Review Summary — Phase 6e-4c2 (TestSourceHandler ObjectModel removal)

The architectural goal is correct and well-executed. The decoupling is clean, the #if NETFRAMEWORK guard is properly applied, and the fidelity note in the PR description is accurate about the dead child-AppDomain. One MAJOR null-safety bug in the helper method needs a follow-up fix; two MINOR polish items are noted.


Verdict: NEEDS WORK

#DimensionStatusSeverityNote
1Algorithmic Correctness⚠️ MinorMAJORLogic is correct for signed assemblies (the production case). For unsigned assemblies (null PKT), ArePublicKeyTokensEqual throws NRE → caught → null → proceed. Accidentally correct outcome, wrong code path.
2Threading & Concurrency✅ CleanStatic method, local variables only, no shared state.
3Security & IPC Contract Safety✅ CleanReflectionOnlyLoadFrom is read-only; no code execution; source path supplied by caller.
4Public API & Binary Compatibility✅ CleanNo public API changes; removed using for ObjectModel is a positive dependency reduction.
5Memory Management & Resource Leaks✅ CleanReflection-only load context limitations are pre-existing; behaviour unchanged from original call.
6Error Handling⚠️ MinorMINORBare catch absorbs all exceptions silently; no diagnostic trace makes debugging difficult (see inline comment on line 135).
7Naming & Code Style✅ CleanDoesSourceReferenceAssembly, ArePublicKeyTokensEqual are clear verb-led names; file-scoped namespace, no abbreviations.
8Documentation & Comments⚠️ MinorMINORXML doc on DoesSourceReferenceAssembly is accurate. Stale comment on line 78 ("different app domain") is now inaccurate (see inline comment).
9Test Coverage✅ Clean7/7 DesktopTestSourceTests reported; edge cases (null name, null source, referenced/not-referenced) are covered per PR description. Null-PKT path not explicitly tested but the outer catch provides the safety net.
10Performance✅ CleanStringComparison.OrdinalIgnoreCase used; no unnecessary allocations in the loop.
11Cross-TFM Correctness✅ CleanAll new code is inside #if NETFRAMEWORK; Assembly.ReflectionOnlyLoadFrom is .NET Framework-only.
12LocalizationN/ANo user-facing strings added.
13Dependency Management✅ CleanPositive change: removes the ObjectModel using directive from PlatformServices.
14Telemetry & ObservabilityN/ANo new user-visible behaviour; existing discovery flow preserved.
15CI/CD & Build System✅ CleanNo build script changes; guarded by #if NETFRAMEWORK.
16Backward Compatibility✅ CleanIsAssemblyReferenced return logic is unchanged; callers see identical results for all signed-assembly inputs.
17Null Safety❌ IssueMAJORGetPublicKeyToken() returns null on .NET Framework for unsigned assemblies. ArePublicKeyTokensEqual(byte[] left, byte[] right) has no null guards; left.Length throws NullReferenceException. referenceAssemblyPublicKeyToken declared byte[] rather than byte[]?, hiding the nullable concern. See inline comment on line 144.
18Exception Safety✅ CleanNo partial-state mutations; catch is for observation, not rollback.
19Immutability & Value Semantics✅ CleanPure static methods operating on immutable inputs.
20Design Patterns & Architecture✅ CleanAligns perfectly with the stated goal; dependency removed at the correct layer.
21Logging Correctness⚠️ MinorMINORNo diagnostic output when the catch fires; a Debug.WriteLine would cost nothing and aid debugging.
22Scope & PR Hygiene✅ CleanSingle concern; no unrelated changes; no dead code introduced.

Required follow-up

ArePublicKeyTokensEqual — add null guards (line 144, inline comment posted):

privatestaticboolArePublicKeyTokensEqual(byte[]?left,byte[]?right){if(leftisnull&&rightisnull)returntrue;if(leftisnull||rightisnull)returnfalse;if(left.Length!=right.Length)returnfalse;for(inti=0;i<left.Length;++i){if(left[i]!=right[i])returnfalse;}returntrue;}

Also fix the assignment on line 113 to byte[]? referenceAssemblyPublicKeyToken = referenceAssembly.GetPublicKeyToken();.

In practice MSTest is always shipped signed so null PKTs will not occur in the field, but the code is objectively wrong and should be fixed before being built upon in the next phase of this decoupling series.


private static bool ArePublicKeyTokensEqual(byte[] left, byte[] right)
{
if (left.Length != right.Length)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MAJOR — Null Safety]GetPublicKeyToken() returns null on .NET Framework for unsigned assemblies (those with no public key token). Both left (from referencedAssembly.GetPublicKeyToken()) and right (from referenceAssembly.GetPublicKeyToken()) can therefore be null, so left.Length on this line throws NullReferenceException. The outer catch absorbs that and returns null (→ proceed with discovery), so the process doesn't crash; but there are two problems:

  1. For two assemblies that both carry no public-key token, the semantically correct result is true (name match + neither is signed = same unsigned identity). Instead the caller gets a swallowed exception and falls through to "proceed" via null, which is accidentally correct today but for the wrong reason.
  2. referenceAssemblyPublicKeyToken on line 113 is declared byte[] (non-nullable), masking the actual nullable return of GetPublicKeyToken(). It should be byte[]?.

Suggested fix:

privatestaticboolArePublicKeyTokensEqual(byte[]?left,byte[]?right){if(leftisnull&&rightisnull)returntrue;if(leftisnull||rightisnull)returnfalse;if(left.Length!=right.Length)returnfalse;for(inti=0;i<left.Length;++i){if(left[i]!=right[i])returnfalse;}returntrue;}

Also update the assignment on line 113:

byte[]?referenceAssemblyPublicKeyToken=referenceAssembly.GetPublicKeyToken();


return false;
}
catch

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MINOR — Bare Catch / No Diagnostics] The catch block silently swallows every exception — file-not-found, BadImageFormatException, security exceptions, and the null-deref discussed on ArePublicKeyTokensEqual all land here invisibly. The conservative null return is intentional and correct, but without any trace output it is very hard to tell the difference between "file legitimately couldn't be opened" and "code bug".

Suggestion — at minimum emit a Debug.WriteLine (no user-visible surface, zero overhead in production):

catch(Exceptionex){Debug.WriteLine($"[MSTest] DoesSourceReferenceAssembly could not inspect '{source}': {ex.Message}");returnnull;}

@@ -79,7 +76,7 @@ public bool IsAssemblyReferenced(AssemblyName assemblyName, string source)
{
#if NETFRAMEWORK
// This loads the dll in a different app domain. We can optimize this to load in the current domain since this code could be run in a new app domain anyway.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MINOR — Stale Comment] The comment says "This loads the dll in a different app domain", but the new implementation (DoesSourceReferenceAssembly) uses Assembly.ReflectionOnlyLoadFrom in the current AppDomain — no child domain is created. Both sentences are now inaccurate (the second was an optimisation note that is now moot since the domain split was intentionally dropped).

Suggested replacement:

// Reflection-only loads the assembly to inspect its references without executing any code from it.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs/author-feedbackWaiting on the original author.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@Evangelink
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

Reimplement source assembly-reference check without ObjectModel (Phase 6e-4c2) - #9631

Merged
Amaury Levé (Evangelink) merged 1 commit into
dev/amauryleve/vstest-decoupling-sourcehostfrom
dev/amauryleve/vstest-decoupling-sourcehandler
Jul 5, 2026
Merged

Reimplement source assembly-reference check without ObjectModel (Phase 6e-4c2)#9631
Amaury Levé (Evangelink) merged 1 commit into
dev/amauryleve/vstest-decoupling-sourcehostfrom
dev/amauryleve/vstest-decoupling-sourcehandler

Conversation

@Evangelink

Copy link
Copy Markdown
Member

Phase 6e-4c2 — reimplement the source assembly-reference check without ObjectModel

Part of the initiative to make Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices platform-agnostic by removing its dependency on Microsoft.TestPlatform.ObjectModel. VSTest coupling moves up into MSTest.TestAdapter; the platform-services engine becomes neutral. Strict byte-for-byte, no behavior change.

What this changes

TestSourceHandler.IsAssemblyReferenced (netfx-only) decided whether a source assembly references the test framework — used to skip discovery on sources that don't reference MSTest — by calling AssemblyHelper.DoesReferencesAssembly from Microsoft.VisualStudio.TestPlatform.ObjectModel.Utilities.

This replaces that call with a local neutral DoesSourceReferenceAssembly helper that reproduces the exact observable behavior of the VSTest implementation:

  • Assembly.ReflectionOnlyLoadFrom(source)GetReferencedAssemblies().
  • Match a referenced assembly by simple name (OrdinalIgnoreCase) plus public-key-token bytes; version is ignored — identical to AssemblyLoadWorker.CheckAssemblyReference.
  • Name match + public-key-token length/byte mismatch keeps scanning further references (mirrors the original continue), rather than short-circuiting.
  • Null/empty source or null reference assembly → null (undeterminable).
  • Any exception → null, so discovery proceeds conservatively.

The IsAssemblyReferenced decision line (return !utfReference.HasValue || utfReference.Value; — null-or-true ⇒ proceed, false ⇒ skip) is unchanged.

Fidelity note (dead child-AppDomain)

VSTest's DoesReferencesAssembly created a child AppDomain and an AssemblyLoadWorker instance, but then called the staticAssemblyLoadWorker.CheckAssemblyReference(...) — the worker instance is assigned and never used, and the ReflectionOnlyLoadFrom actually runs in the current domain. The child domain is therefore dead code for the result, so omitting it is behavior-preserving for every input where AppDomain.CreateDomain would have succeeded (the only theoretical divergence — a machine where domain creation throws but reflection-only load succeeds — is unreachable in practice, and the conservative null-return there would only proceed with discovery either way).

Removes the last real ObjectModel dependency from TestSourceHandler (it now carries only string-literal well-known-assembly names, which don't reference the package).

Verification

  • All real TFMs build 0-warning (UWP builds via full msbuild in CI, unaffected — change is #if NETFRAMEWORK-guarded).
  • MSTestAdapter.PlatformServices.UnitTests: 935/935 (net462), 897/897 (net8.0).
  • DesktopTestSourceTests — the direct IsAssemblyReferenced net — 7/7, covering: assembly referenced (true), not referenced (false), null name (true), null source (true).
  • PlatformServices.Desktop.IntegrationTests: 15/15 (net462).
  • Expert-reviewer pass.

Stacking

Stacks on #9630 (Phase 6e-4c1); base branch dev/amauryleve/vstest-decoupling-sourcehost. Review/merge after the earlier PRs in the chain reach the base. Do not squash-rebase the base.

…Phase 6e-4c2)
TestSourceHandler.IsAssemblyReferenced (netfx) used
AssemblyHelper.DoesReferencesAssembly from
Microsoft.VisualStudio.TestPlatform.ObjectModel.Utilities to decide whether a
source assembly references the test framework before running discovery.
Replace that call with a local neutral DoesSourceReferenceAssembly helper that
reproduces the exact observable behavior of the VSTest implementation:
- ReflectionOnlyLoadFrom(source), then GetReferencedAssemblies().
- Match a referenced assembly by simple name (OrdinalIgnoreCase) plus public
key token bytes; version is ignored -- identical to
AssemblyLoadWorker.CheckAssemblyReference.
- Null/empty source or null reference assembly returns null (undeterminable).
- Any exception returns null so discovery proceeds conservatively.
Fidelity note: the VSTest DoesReferencesAssembly created a child AppDomain and an
AssemblyLoadWorker instance, but then called the *static*
AssemblyLoadWorker.CheckAssemblyReference -- the worker instance is never used and
the ReflectionOnlyLoadFrom actually runs in the current domain. The child domain
is therefore dead code for the result, so omitting it is behavior-preserving for
every input where AppDomain creation would have succeeded.
Removes the last real ObjectModel dependency from TestSourceHandler.
Verified: PlatformServices.UnitTests 935 (net462) / 897 (net8.0);
DesktopTestSourceTests IsAssemblyReferenced branches 7/7 (referenced,
not-referenced, null-name, null-source); PlatformServices.Desktop.IntegrationTests
15/15. All real TFMs build 0-warning.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@Evangelink
Amaury Levé (Evangelink) marked this pull request as ready for review July 5, 2026 19:24
@Evangelink
Amaury Levé (Evangelink) merged commit c3c9653 into dev/amauryleve/vstest-decoupling-sourcehostJul 5, 2026
20 checks passed
@Evangelink
Amaury Levé (Evangelink) deleted the dev/amauryleve/vstest-decoupling-sourcehandler branch July 5, 2026 19:24

@github-actionsgithub-actionsBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

🤖 Automated review by GitHub Copilot. Posted via a maintainer's GitHub token, so it appears under their account — the account owner did not write or approve this content personally. Generated by the Expert Code Review workflow. To request a follow-up action, reply by tagging @copilot directly.

Review Summary — Phase 6e-4c2 (TestSourceHandler ObjectModel removal)

The architectural goal is correct and well-executed. The decoupling is clean, the #if NETFRAMEWORK guard is properly applied, and the fidelity note in the PR description is accurate about the dead child-AppDomain. One MAJOR null-safety bug in the helper method needs a follow-up fix; two MINOR polish items are noted.


Verdict: NEEDS WORK

#DimensionStatusSeverityNote
1Algorithmic Correctness⚠️ MinorMAJORLogic is correct for signed assemblies (the production case). For unsigned assemblies (null PKT), ArePublicKeyTokensEqual throws NRE → caught → null → proceed. Accidentally correct outcome, wrong code path.
2Threading & Concurrency✅ CleanStatic method, local variables only, no shared state.
3Security & IPC Contract Safety✅ CleanReflectionOnlyLoadFrom is read-only; no code execution; source path supplied by caller.
4Public API & Binary Compatibility✅ CleanNo public API changes; removed using for ObjectModel is a positive dependency reduction.
5Memory Management & Resource Leaks✅ CleanReflection-only load context limitations are pre-existing; behaviour unchanged from original call.
6Error Handling⚠️ MinorMINORBare catch absorbs all exceptions silently; no diagnostic trace makes debugging difficult (see inline comment on line 135).
7Naming & Code Style✅ CleanDoesSourceReferenceAssembly, ArePublicKeyTokensEqual are clear verb-led names; file-scoped namespace, no abbreviations.
8Documentation & Comments⚠️ MinorMINORXML doc on DoesSourceReferenceAssembly is accurate. Stale comment on line 78 ("different app domain") is now inaccurate (see inline comment).
9Test Coverage✅ Clean7/7 DesktopTestSourceTests reported; edge cases (null name, null source, referenced/not-referenced) are covered per PR description. Null-PKT path not explicitly tested but the outer catch provides the safety net.
10Performance✅ CleanStringComparison.OrdinalIgnoreCase used; no unnecessary allocations in the loop.
11Cross-TFM Correctness✅ CleanAll new code is inside #if NETFRAMEWORK; Assembly.ReflectionOnlyLoadFrom is .NET Framework-only.
12LocalizationN/ANo user-facing strings added.
13Dependency Management✅ CleanPositive change: removes the ObjectModel using directive from PlatformServices.
14Telemetry & ObservabilityN/ANo new user-visible behaviour; existing discovery flow preserved.
15CI/CD & Build System✅ CleanNo build script changes; guarded by #if NETFRAMEWORK.
16Backward Compatibility✅ CleanIsAssemblyReferenced return logic is unchanged; callers see identical results for all signed-assembly inputs.
17Null Safety❌ IssueMAJORGetPublicKeyToken() returns null on .NET Framework for unsigned assemblies. ArePublicKeyTokensEqual(byte[] left, byte[] right) has no null guards; left.Length throws NullReferenceException. referenceAssemblyPublicKeyToken declared byte[] rather than byte[]?, hiding the nullable concern. See inline comment on line 144.
18Exception Safety✅ CleanNo partial-state mutations; catch is for observation, not rollback.
19Immutability & Value Semantics✅ CleanPure static methods operating on immutable inputs.
20Design Patterns & Architecture✅ CleanAligns perfectly with the stated goal; dependency removed at the correct layer.
21Logging Correctness⚠️ MinorMINORNo diagnostic output when the catch fires; a Debug.WriteLine would cost nothing and aid debugging.
22Scope & PR Hygiene✅ CleanSingle concern; no unrelated changes; no dead code introduced.

Required follow-up

ArePublicKeyTokensEqual — add null guards (line 144, inline comment posted):

privatestaticboolArePublicKeyTokensEqual(byte[]?left,byte[]?right){if(leftisnull&&rightisnull)returntrue;if(leftisnull||rightisnull)returnfalse;if(left.Length!=right.Length)returnfalse;for(inti=0;i<left.Length;++i){if(left[i]!=right[i])returnfalse;}returntrue;}

Also fix the assignment on line 113 to byte[]? referenceAssemblyPublicKeyToken = referenceAssembly.GetPublicKeyToken();.

In practice MSTest is always shipped signed so null PKTs will not occur in the field, but the code is objectively wrong and should be fixed before being built upon in the next phase of this decoupling series.


private static bool ArePublicKeyTokensEqual(byte[] left, byte[] right)
{
if (left.Length != right.Length)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MAJOR — Null Safety]GetPublicKeyToken() returns null on .NET Framework for unsigned assemblies (those with no public key token). Both left (from referencedAssembly.GetPublicKeyToken()) and right (from referenceAssembly.GetPublicKeyToken()) can therefore be null, so left.Length on this line throws NullReferenceException. The outer catch absorbs that and returns null (→ proceed with discovery), so the process doesn't crash; but there are two problems:

  1. For two assemblies that both carry no public-key token, the semantically correct result is true (name match + neither is signed = same unsigned identity). Instead the caller gets a swallowed exception and falls through to "proceed" via null, which is accidentally correct today but for the wrong reason.
  2. referenceAssemblyPublicKeyToken on line 113 is declared byte[] (non-nullable), masking the actual nullable return of GetPublicKeyToken(). It should be byte[]?.

Suggested fix:

privatestaticboolArePublicKeyTokensEqual(byte[]?left,byte[]?right){if(leftisnull&&rightisnull)returntrue;if(leftisnull||rightisnull)returnfalse;if(left.Length!=right.Length)returnfalse;for(inti=0;i<left.Length;++i){if(left[i]!=right[i])returnfalse;}returntrue;}

Also update the assignment on line 113:

byte[]?referenceAssemblyPublicKeyToken=referenceAssembly.GetPublicKeyToken();


return false;
}
catch

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MINOR — Bare Catch / No Diagnostics] The catch block silently swallows every exception — file-not-found, BadImageFormatException, security exceptions, and the null-deref discussed on ArePublicKeyTokensEqual all land here invisibly. The conservative null return is intentional and correct, but without any trace output it is very hard to tell the difference between "file legitimately couldn't be opened" and "code bug".

Suggestion — at minimum emit a Debug.WriteLine (no user-visible surface, zero overhead in production):

catch(Exceptionex){Debug.WriteLine($"[MSTest] DoesSourceReferenceAssembly could not inspect '{source}': {ex.Message}");returnnull;}

@@ -79,7 +76,7 @@ public bool IsAssemblyReferenced(AssemblyName assemblyName, string source)
{
#if NETFRAMEWORK
// This loads the dll in a different app domain. We can optimize this to load in the current domain since this code could be run in a new app domain anyway.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MINOR — Stale Comment] The comment says "This loads the dll in a different app domain", but the new implementation (DoesSourceReferenceAssembly) uses Assembly.ReflectionOnlyLoadFrom in the current AppDomain — no child domain is created. Both sentences are now inaccurate (the second was an optimisation note that is now moot since the domain split was intentionally dropped).

Suggested replacement:

// Reflection-only loads the assembly to inspect its references without executing any code from it.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs/author-feedbackWaiting on the original author.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@Evangelink
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Reimplement source assembly-reference check without ObjectModel (Phase 6e-4c2) - #9631

Merged
Amaury Levé (Evangelink) merged 1 commit into
dev/amauryleve/vstest-decoupling-sourcehostfrom
dev/amauryleve/vstest-decoupling-sourcehandler
Jul 5, 2026
Merged

Reimplement source assembly-reference check without ObjectModel (Phase 6e-4c2)#9631
Amaury Levé (Evangelink) merged 1 commit into
dev/amauryleve/vstest-decoupling-sourcehostfrom
dev/amauryleve/vstest-decoupling-sourcehandler

Conversation

@Evangelink

Copy link
Copy Markdown
Member

Phase 6e-4c2 — reimplement the source assembly-reference check without ObjectModel

Part of the initiative to make Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices platform-agnostic by removing its dependency on Microsoft.TestPlatform.ObjectModel. VSTest coupling moves up into MSTest.TestAdapter; the platform-services engine becomes neutral. Strict byte-for-byte, no behavior change.

What this changes

TestSourceHandler.IsAssemblyReferenced (netfx-only) decided whether a source assembly references the test framework — used to skip discovery on sources that don't reference MSTest — by calling AssemblyHelper.DoesReferencesAssembly from Microsoft.VisualStudio.TestPlatform.ObjectModel.Utilities.

This replaces that call with a local neutral DoesSourceReferenceAssembly helper that reproduces the exact observable behavior of the VSTest implementation:

  • Assembly.ReflectionOnlyLoadFrom(source)GetReferencedAssemblies().
  • Match a referenced assembly by simple name (OrdinalIgnoreCase) plus public-key-token bytes; version is ignored — identical to AssemblyLoadWorker.CheckAssemblyReference.
  • Name match + public-key-token length/byte mismatch keeps scanning further references (mirrors the original continue), rather than short-circuiting.
  • Null/empty source or null reference assembly → null (undeterminable).
  • Any exception → null, so discovery proceeds conservatively.

The IsAssemblyReferenced decision line (return !utfReference.HasValue || utfReference.Value; — null-or-true ⇒ proceed, false ⇒ skip) is unchanged.

Fidelity note (dead child-AppDomain)

VSTest's DoesReferencesAssembly created a child AppDomain and an AssemblyLoadWorker instance, but then called the staticAssemblyLoadWorker.CheckAssemblyReference(...) — the worker instance is assigned and never used, and the ReflectionOnlyLoadFrom actually runs in the current domain. The child domain is therefore dead code for the result, so omitting it is behavior-preserving for every input where AppDomain.CreateDomain would have succeeded (the only theoretical divergence — a machine where domain creation throws but reflection-only load succeeds — is unreachable in practice, and the conservative null-return there would only proceed with discovery either way).

Removes the last real ObjectModel dependency from TestSourceHandler (it now carries only string-literal well-known-assembly names, which don't reference the package).

Verification

  • All real TFMs build 0-warning (UWP builds via full msbuild in CI, unaffected — change is #if NETFRAMEWORK-guarded).
  • MSTestAdapter.PlatformServices.UnitTests: 935/935 (net462), 897/897 (net8.0).
  • DesktopTestSourceTests — the direct IsAssemblyReferenced net — 7/7, covering: assembly referenced (true), not referenced (false), null name (true), null source (true).
  • PlatformServices.Desktop.IntegrationTests: 15/15 (net462).
  • Expert-reviewer pass.

Stacking

Stacks on #9630 (Phase 6e-4c1); base branch dev/amauryleve/vstest-decoupling-sourcehost. Review/merge after the earlier PRs in the chain reach the base. Do not squash-rebase the base.

…Phase 6e-4c2)
TestSourceHandler.IsAssemblyReferenced (netfx) used
AssemblyHelper.DoesReferencesAssembly from
Microsoft.VisualStudio.TestPlatform.ObjectModel.Utilities to decide whether a
source assembly references the test framework before running discovery.
Replace that call with a local neutral DoesSourceReferenceAssembly helper that
reproduces the exact observable behavior of the VSTest implementation:
- ReflectionOnlyLoadFrom(source), then GetReferencedAssemblies().
- Match a referenced assembly by simple name (OrdinalIgnoreCase) plus public
key token bytes; version is ignored -- identical to
AssemblyLoadWorker.CheckAssemblyReference.
- Null/empty source or null reference assembly returns null (undeterminable).
- Any exception returns null so discovery proceeds conservatively.
Fidelity note: the VSTest DoesReferencesAssembly created a child AppDomain and an
AssemblyLoadWorker instance, but then called the *static*
AssemblyLoadWorker.CheckAssemblyReference -- the worker instance is never used and
the ReflectionOnlyLoadFrom actually runs in the current domain. The child domain
is therefore dead code for the result, so omitting it is behavior-preserving for
every input where AppDomain creation would have succeeded.
Removes the last real ObjectModel dependency from TestSourceHandler.
Verified: PlatformServices.UnitTests 935 (net462) / 897 (net8.0);
DesktopTestSourceTests IsAssemblyReferenced branches 7/7 (referenced,
not-referenced, null-name, null-source); PlatformServices.Desktop.IntegrationTests
15/15. All real TFMs build 0-warning.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@Evangelink
Amaury Levé (Evangelink) marked this pull request as ready for review July 5, 2026 19:24
@Evangelink
Amaury Levé (Evangelink) merged commit c3c9653 into dev/amauryleve/vstest-decoupling-sourcehostJul 5, 2026
20 checks passed
@Evangelink
Amaury Levé (Evangelink) deleted the dev/amauryleve/vstest-decoupling-sourcehandler branch July 5, 2026 19:24

@github-actionsgithub-actionsBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

🤖 Automated review by GitHub Copilot. Posted via a maintainer's GitHub token, so it appears under their account — the account owner did not write or approve this content personally. Generated by the Expert Code Review workflow. To request a follow-up action, reply by tagging @copilot directly.

Review Summary — Phase 6e-4c2 (TestSourceHandler ObjectModel removal)

The architectural goal is correct and well-executed. The decoupling is clean, the #if NETFRAMEWORK guard is properly applied, and the fidelity note in the PR description is accurate about the dead child-AppDomain. One MAJOR null-safety bug in the helper method needs a follow-up fix; two MINOR polish items are noted.


Verdict: NEEDS WORK

#DimensionStatusSeverityNote
1Algorithmic Correctness⚠️ MinorMAJORLogic is correct for signed assemblies (the production case). For unsigned assemblies (null PKT), ArePublicKeyTokensEqual throws NRE → caught → null → proceed. Accidentally correct outcome, wrong code path.
2Threading & Concurrency✅ CleanStatic method, local variables only, no shared state.
3Security & IPC Contract Safety✅ CleanReflectionOnlyLoadFrom is read-only; no code execution; source path supplied by caller.
4Public API & Binary Compatibility✅ CleanNo public API changes; removed using for ObjectModel is a positive dependency reduction.
5Memory Management & Resource Leaks✅ CleanReflection-only load context limitations are pre-existing; behaviour unchanged from original call.
6Error Handling⚠️ MinorMINORBare catch absorbs all exceptions silently; no diagnostic trace makes debugging difficult (see inline comment on line 135).
7Naming & Code Style✅ CleanDoesSourceReferenceAssembly, ArePublicKeyTokensEqual are clear verb-led names; file-scoped namespace, no abbreviations.
8Documentation & Comments⚠️ MinorMINORXML doc on DoesSourceReferenceAssembly is accurate. Stale comment on line 78 ("different app domain") is now inaccurate (see inline comment).
9Test Coverage✅ Clean7/7 DesktopTestSourceTests reported; edge cases (null name, null source, referenced/not-referenced) are covered per PR description. Null-PKT path not explicitly tested but the outer catch provides the safety net.
10Performance✅ CleanStringComparison.OrdinalIgnoreCase used; no unnecessary allocations in the loop.
11Cross-TFM Correctness✅ CleanAll new code is inside #if NETFRAMEWORK; Assembly.ReflectionOnlyLoadFrom is .NET Framework-only.
12LocalizationN/ANo user-facing strings added.
13Dependency Management✅ CleanPositive change: removes the ObjectModel using directive from PlatformServices.
14Telemetry & ObservabilityN/ANo new user-visible behaviour; existing discovery flow preserved.
15CI/CD & Build System✅ CleanNo build script changes; guarded by #if NETFRAMEWORK.
16Backward Compatibility✅ CleanIsAssemblyReferenced return logic is unchanged; callers see identical results for all signed-assembly inputs.
17Null Safety❌ IssueMAJORGetPublicKeyToken() returns null on .NET Framework for unsigned assemblies. ArePublicKeyTokensEqual(byte[] left, byte[] right) has no null guards; left.Length throws NullReferenceException. referenceAssemblyPublicKeyToken declared byte[] rather than byte[]?, hiding the nullable concern. See inline comment on line 144.
18Exception Safety✅ CleanNo partial-state mutations; catch is for observation, not rollback.
19Immutability & Value Semantics✅ CleanPure static methods operating on immutable inputs.
20Design Patterns & Architecture✅ CleanAligns perfectly with the stated goal; dependency removed at the correct layer.
21Logging Correctness⚠️ MinorMINORNo diagnostic output when the catch fires; a Debug.WriteLine would cost nothing and aid debugging.
22Scope & PR Hygiene✅ CleanSingle concern; no unrelated changes; no dead code introduced.

Required follow-up

ArePublicKeyTokensEqual — add null guards (line 144, inline comment posted):

privatestaticboolArePublicKeyTokensEqual(byte[]?left,byte[]?right){if(leftisnull&&rightisnull)returntrue;if(leftisnull||rightisnull)returnfalse;if(left.Length!=right.Length)returnfalse;for(inti=0;i<left.Length;++i){if(left[i]!=right[i])returnfalse;}returntrue;}

Also fix the assignment on line 113 to byte[]? referenceAssemblyPublicKeyToken = referenceAssembly.GetPublicKeyToken();.

In practice MSTest is always shipped signed so null PKTs will not occur in the field, but the code is objectively wrong and should be fixed before being built upon in the next phase of this decoupling series.


private static bool ArePublicKeyTokensEqual(byte[] left, byte[] right)
{
if (left.Length != right.Length)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MAJOR — Null Safety]GetPublicKeyToken() returns null on .NET Framework for unsigned assemblies (those with no public key token). Both left (from referencedAssembly.GetPublicKeyToken()) and right (from referenceAssembly.GetPublicKeyToken()) can therefore be null, so left.Length on this line throws NullReferenceException. The outer catch absorbs that and returns null (→ proceed with discovery), so the process doesn't crash; but there are two problems:

  1. For two assemblies that both carry no public-key token, the semantically correct result is true (name match + neither is signed = same unsigned identity). Instead the caller gets a swallowed exception and falls through to "proceed" via null, which is accidentally correct today but for the wrong reason.
  2. referenceAssemblyPublicKeyToken on line 113 is declared byte[] (non-nullable), masking the actual nullable return of GetPublicKeyToken(). It should be byte[]?.

Suggested fix:

privatestaticboolArePublicKeyTokensEqual(byte[]?left,byte[]?right){if(leftisnull&&rightisnull)returntrue;if(leftisnull||rightisnull)returnfalse;if(left.Length!=right.Length)returnfalse;for(inti=0;i<left.Length;++i){if(left[i]!=right[i])returnfalse;}returntrue;}

Also update the assignment on line 113:

byte[]?referenceAssemblyPublicKeyToken=referenceAssembly.GetPublicKeyToken();


return false;
}
catch

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MINOR — Bare Catch / No Diagnostics] The catch block silently swallows every exception — file-not-found, BadImageFormatException, security exceptions, and the null-deref discussed on ArePublicKeyTokensEqual all land here invisibly. The conservative null return is intentional and correct, but without any trace output it is very hard to tell the difference between "file legitimately couldn't be opened" and "code bug".

Suggestion — at minimum emit a Debug.WriteLine (no user-visible surface, zero overhead in production):

catch(Exceptionex){Debug.WriteLine($"[MSTest] DoesSourceReferenceAssembly could not inspect '{source}': {ex.Message}");returnnull;}

@@ -79,7 +76,7 @@ public bool IsAssemblyReferenced(AssemblyName assemblyName, string source)
{
#if NETFRAMEWORK
// This loads the dll in a different app domain. We can optimize this to load in the current domain since this code could be run in a new app domain anyway.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MINOR — Stale Comment] The comment says "This loads the dll in a different app domain", but the new implementation (DoesSourceReferenceAssembly) uses Assembly.ReflectionOnlyLoadFrom in the current AppDomain — no child domain is created. Both sentences are now inaccurate (the second was an optimisation note that is now moot since the domain split was intentionally dropped).

Suggested replacement:

// Reflection-only loads the assembly to inspect its references without executing any code from it.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs/author-feedbackWaiting on the original author.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@Evangelink
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Reimplement source assembly-reference check without ObjectModel (Phase 6e-4c2) - #9631

Merged
Amaury Levé (Evangelink) merged 1 commit into
dev/amauryleve/vstest-decoupling-sourcehostfrom
dev/amauryleve/vstest-decoupling-sourcehandler
Jul 5, 2026
Merged

Reimplement source assembly-reference check without ObjectModel (Phase 6e-4c2)#9631
Amaury Levé (Evangelink) merged 1 commit into
dev/amauryleve/vstest-decoupling-sourcehostfrom
dev/amauryleve/vstest-decoupling-sourcehandler

Conversation

@Evangelink

Copy link
Copy Markdown
Member

Phase 6e-4c2 — reimplement the source assembly-reference check without ObjectModel

Part of the initiative to make Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices platform-agnostic by removing its dependency on Microsoft.TestPlatform.ObjectModel. VSTest coupling moves up into MSTest.TestAdapter; the platform-services engine becomes neutral. Strict byte-for-byte, no behavior change.

What this changes

TestSourceHandler.IsAssemblyReferenced (netfx-only) decided whether a source assembly references the test framework — used to skip discovery on sources that don't reference MSTest — by calling AssemblyHelper.DoesReferencesAssembly from Microsoft.VisualStudio.TestPlatform.ObjectModel.Utilities.

This replaces that call with a local neutral DoesSourceReferenceAssembly helper that reproduces the exact observable behavior of the VSTest implementation:

  • Assembly.ReflectionOnlyLoadFrom(source)GetReferencedAssemblies().
  • Match a referenced assembly by simple name (OrdinalIgnoreCase) plus public-key-token bytes; version is ignored — identical to AssemblyLoadWorker.CheckAssemblyReference.
  • Name match + public-key-token length/byte mismatch keeps scanning further references (mirrors the original continue), rather than short-circuiting.
  • Null/empty source or null reference assembly → null (undeterminable).
  • Any exception → null, so discovery proceeds conservatively.

The IsAssemblyReferenced decision line (return !utfReference.HasValue || utfReference.Value; — null-or-true ⇒ proceed, false ⇒ skip) is unchanged.

Fidelity note (dead child-AppDomain)

VSTest's DoesReferencesAssembly created a child AppDomain and an AssemblyLoadWorker instance, but then called the staticAssemblyLoadWorker.CheckAssemblyReference(...) — the worker instance is assigned and never used, and the ReflectionOnlyLoadFrom actually runs in the current domain. The child domain is therefore dead code for the result, so omitting it is behavior-preserving for every input where AppDomain.CreateDomain would have succeeded (the only theoretical divergence — a machine where domain creation throws but reflection-only load succeeds — is unreachable in practice, and the conservative null-return there would only proceed with discovery either way).

Removes the last real ObjectModel dependency from TestSourceHandler (it now carries only string-literal well-known-assembly names, which don't reference the package).

Verification

  • All real TFMs build 0-warning (UWP builds via full msbuild in CI, unaffected — change is #if NETFRAMEWORK-guarded).
  • MSTestAdapter.PlatformServices.UnitTests: 935/935 (net462), 897/897 (net8.0).
  • DesktopTestSourceTests — the direct IsAssemblyReferenced net — 7/7, covering: assembly referenced (true), not referenced (false), null name (true), null source (true).
  • PlatformServices.Desktop.IntegrationTests: 15/15 (net462).
  • Expert-reviewer pass.

Stacking

Stacks on #9630 (Phase 6e-4c1); base branch dev/amauryleve/vstest-decoupling-sourcehost. Review/merge after the earlier PRs in the chain reach the base. Do not squash-rebase the base.

…Phase 6e-4c2)
TestSourceHandler.IsAssemblyReferenced (netfx) used
AssemblyHelper.DoesReferencesAssembly from
Microsoft.VisualStudio.TestPlatform.ObjectModel.Utilities to decide whether a
source assembly references the test framework before running discovery.
Replace that call with a local neutral DoesSourceReferenceAssembly helper that
reproduces the exact observable behavior of the VSTest implementation:
- ReflectionOnlyLoadFrom(source), then GetReferencedAssemblies().
- Match a referenced assembly by simple name (OrdinalIgnoreCase) plus public
key token bytes; version is ignored -- identical to
AssemblyLoadWorker.CheckAssemblyReference.
- Null/empty source or null reference assembly returns null (undeterminable).
- Any exception returns null so discovery proceeds conservatively.
Fidelity note: the VSTest DoesReferencesAssembly created a child AppDomain and an
AssemblyLoadWorker instance, but then called the *static*
AssemblyLoadWorker.CheckAssemblyReference -- the worker instance is never used and
the ReflectionOnlyLoadFrom actually runs in the current domain. The child domain
is therefore dead code for the result, so omitting it is behavior-preserving for
every input where AppDomain creation would have succeeded.
Removes the last real ObjectModel dependency from TestSourceHandler.
Verified: PlatformServices.UnitTests 935 (net462) / 897 (net8.0);
DesktopTestSourceTests IsAssemblyReferenced branches 7/7 (referenced,
not-referenced, null-name, null-source); PlatformServices.Desktop.IntegrationTests
15/15. All real TFMs build 0-warning.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@Evangelink
Amaury Levé (Evangelink) marked this pull request as ready for review July 5, 2026 19:24
@Evangelink
Amaury Levé (Evangelink) merged commit c3c9653 into dev/amauryleve/vstest-decoupling-sourcehostJul 5, 2026
20 checks passed
@Evangelink
Amaury Levé (Evangelink) deleted the dev/amauryleve/vstest-decoupling-sourcehandler branch July 5, 2026 19:24

@github-actionsgithub-actionsBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

🤖 Automated review by GitHub Copilot. Posted via a maintainer's GitHub token, so it appears under their account — the account owner did not write or approve this content personally. Generated by the Expert Code Review workflow. To request a follow-up action, reply by tagging @copilot directly.

Review Summary — Phase 6e-4c2 (TestSourceHandler ObjectModel removal)

The architectural goal is correct and well-executed. The decoupling is clean, the #if NETFRAMEWORK guard is properly applied, and the fidelity note in the PR description is accurate about the dead child-AppDomain. One MAJOR null-safety bug in the helper method needs a follow-up fix; two MINOR polish items are noted.


Verdict: NEEDS WORK

#DimensionStatusSeverityNote
1Algorithmic Correctness⚠️ MinorMAJORLogic is correct for signed assemblies (the production case). For unsigned assemblies (null PKT), ArePublicKeyTokensEqual throws NRE → caught → null → proceed. Accidentally correct outcome, wrong code path.
2Threading & Concurrency✅ CleanStatic method, local variables only, no shared state.
3Security & IPC Contract Safety✅ CleanReflectionOnlyLoadFrom is read-only; no code execution; source path supplied by caller.
4Public API & Binary Compatibility✅ CleanNo public API changes; removed using for ObjectModel is a positive dependency reduction.
5Memory Management & Resource Leaks✅ CleanReflection-only load context limitations are pre-existing; behaviour unchanged from original call.
6Error Handling⚠️ MinorMINORBare catch absorbs all exceptions silently; no diagnostic trace makes debugging difficult (see inline comment on line 135).
7Naming & Code Style✅ CleanDoesSourceReferenceAssembly, ArePublicKeyTokensEqual are clear verb-led names; file-scoped namespace, no abbreviations.
8Documentation & Comments⚠️ MinorMINORXML doc on DoesSourceReferenceAssembly is accurate. Stale comment on line 78 ("different app domain") is now inaccurate (see inline comment).
9Test Coverage✅ Clean7/7 DesktopTestSourceTests reported; edge cases (null name, null source, referenced/not-referenced) are covered per PR description. Null-PKT path not explicitly tested but the outer catch provides the safety net.
10Performance✅ CleanStringComparison.OrdinalIgnoreCase used; no unnecessary allocations in the loop.
11Cross-TFM Correctness✅ CleanAll new code is inside #if NETFRAMEWORK; Assembly.ReflectionOnlyLoadFrom is .NET Framework-only.
12LocalizationN/ANo user-facing strings added.
13Dependency Management✅ CleanPositive change: removes the ObjectModel using directive from PlatformServices.
14Telemetry & ObservabilityN/ANo new user-visible behaviour; existing discovery flow preserved.
15CI/CD & Build System✅ CleanNo build script changes; guarded by #if NETFRAMEWORK.
16Backward Compatibility✅ CleanIsAssemblyReferenced return logic is unchanged; callers see identical results for all signed-assembly inputs.
17Null Safety❌ IssueMAJORGetPublicKeyToken() returns null on .NET Framework for unsigned assemblies. ArePublicKeyTokensEqual(byte[] left, byte[] right) has no null guards; left.Length throws NullReferenceException. referenceAssemblyPublicKeyToken declared byte[] rather than byte[]?, hiding the nullable concern. See inline comment on line 144.
18Exception Safety✅ CleanNo partial-state mutations; catch is for observation, not rollback.
19Immutability & Value Semantics✅ CleanPure static methods operating on immutable inputs.
20Design Patterns & Architecture✅ CleanAligns perfectly with the stated goal; dependency removed at the correct layer.
21Logging Correctness⚠️ MinorMINORNo diagnostic output when the catch fires; a Debug.WriteLine would cost nothing and aid debugging.
22Scope & PR Hygiene✅ CleanSingle concern; no unrelated changes; no dead code introduced.

Required follow-up

ArePublicKeyTokensEqual — add null guards (line 144, inline comment posted):

privatestaticboolArePublicKeyTokensEqual(byte[]?left,byte[]?right){if(leftisnull&&rightisnull)returntrue;if(leftisnull||rightisnull)returnfalse;if(left.Length!=right.Length)returnfalse;for(inti=0;i<left.Length;++i){if(left[i]!=right[i])returnfalse;}returntrue;}

Also fix the assignment on line 113 to byte[]? referenceAssemblyPublicKeyToken = referenceAssembly.GetPublicKeyToken();.

In practice MSTest is always shipped signed so null PKTs will not occur in the field, but the code is objectively wrong and should be fixed before being built upon in the next phase of this decoupling series.


private static bool ArePublicKeyTokensEqual(byte[] left, byte[] right)
{
if (left.Length != right.Length)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MAJOR — Null Safety]GetPublicKeyToken() returns null on .NET Framework for unsigned assemblies (those with no public key token). Both left (from referencedAssembly.GetPublicKeyToken()) and right (from referenceAssembly.GetPublicKeyToken()) can therefore be null, so left.Length on this line throws NullReferenceException. The outer catch absorbs that and returns null (→ proceed with discovery), so the process doesn't crash; but there are two problems:

  1. For two assemblies that both carry no public-key token, the semantically correct result is true (name match + neither is signed = same unsigned identity). Instead the caller gets a swallowed exception and falls through to "proceed" via null, which is accidentally correct today but for the wrong reason.
  2. referenceAssemblyPublicKeyToken on line 113 is declared byte[] (non-nullable), masking the actual nullable return of GetPublicKeyToken(). It should be byte[]?.

Suggested fix:

privatestaticboolArePublicKeyTokensEqual(byte[]?left,byte[]?right){if(leftisnull&&rightisnull)returntrue;if(leftisnull||rightisnull)returnfalse;if(left.Length!=right.Length)returnfalse;for(inti=0;i<left.Length;++i){if(left[i]!=right[i])returnfalse;}returntrue;}

Also update the assignment on line 113:

byte[]?referenceAssemblyPublicKeyToken=referenceAssembly.GetPublicKeyToken();


return false;
}
catch

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MINOR — Bare Catch / No Diagnostics] The catch block silently swallows every exception — file-not-found, BadImageFormatException, security exceptions, and the null-deref discussed on ArePublicKeyTokensEqual all land here invisibly. The conservative null return is intentional and correct, but without any trace output it is very hard to tell the difference between "file legitimately couldn't be opened" and "code bug".

Suggestion — at minimum emit a Debug.WriteLine (no user-visible surface, zero overhead in production):

catch(Exceptionex){Debug.WriteLine($"[MSTest] DoesSourceReferenceAssembly could not inspect '{source}': {ex.Message}");returnnull;}

@@ -79,7 +76,7 @@ public bool IsAssemblyReferenced(AssemblyName assemblyName, string source)
{
#if NETFRAMEWORK
// This loads the dll in a different app domain. We can optimize this to load in the current domain since this code could be run in a new app domain anyway.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MINOR — Stale Comment] The comment says "This loads the dll in a different app domain", but the new implementation (DoesSourceReferenceAssembly) uses Assembly.ReflectionOnlyLoadFrom in the current AppDomain — no child domain is created. Both sentences are now inaccurate (the second was an optimisation note that is now moot since the domain split was intentionally dropped).

Suggested replacement:

// Reflection-only loads the assembly to inspect its references without executing any code from it.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs/author-feedbackWaiting on the original author.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@Evangelink
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

Reimplement source assembly-reference check without ObjectModel (Phase 6e-4c2) - #9631

Merged
Amaury Levé (Evangelink) merged 1 commit into
dev/amauryleve/vstest-decoupling-sourcehostfrom
dev/amauryleve/vstest-decoupling-sourcehandler
Jul 5, 2026
Merged

Reimplement source assembly-reference check without ObjectModel (Phase 6e-4c2)#9631
Amaury Levé (Evangelink) merged 1 commit into
dev/amauryleve/vstest-decoupling-sourcehostfrom
dev/amauryleve/vstest-decoupling-sourcehandler

Conversation

@Evangelink

Copy link
Copy Markdown
Member

Phase 6e-4c2 — reimplement the source assembly-reference check without ObjectModel

Part of the initiative to make Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices platform-agnostic by removing its dependency on Microsoft.TestPlatform.ObjectModel. VSTest coupling moves up into MSTest.TestAdapter; the platform-services engine becomes neutral. Strict byte-for-byte, no behavior change.

What this changes

TestSourceHandler.IsAssemblyReferenced (netfx-only) decided whether a source assembly references the test framework — used to skip discovery on sources that don't reference MSTest — by calling AssemblyHelper.DoesReferencesAssembly from Microsoft.VisualStudio.TestPlatform.ObjectModel.Utilities.

This replaces that call with a local neutral DoesSourceReferenceAssembly helper that reproduces the exact observable behavior of the VSTest implementation:

  • Assembly.ReflectionOnlyLoadFrom(source)GetReferencedAssemblies().
  • Match a referenced assembly by simple name (OrdinalIgnoreCase) plus public-key-token bytes; version is ignored — identical to AssemblyLoadWorker.CheckAssemblyReference.
  • Name match + public-key-token length/byte mismatch keeps scanning further references (mirrors the original continue), rather than short-circuiting.
  • Null/empty source or null reference assembly → null (undeterminable).
  • Any exception → null, so discovery proceeds conservatively.

The IsAssemblyReferenced decision line (return !utfReference.HasValue || utfReference.Value; — null-or-true ⇒ proceed, false ⇒ skip) is unchanged.

Fidelity note (dead child-AppDomain)

VSTest's DoesReferencesAssembly created a child AppDomain and an AssemblyLoadWorker instance, but then called the staticAssemblyLoadWorker.CheckAssemblyReference(...) — the worker instance is assigned and never used, and the ReflectionOnlyLoadFrom actually runs in the current domain. The child domain is therefore dead code for the result, so omitting it is behavior-preserving for every input where AppDomain.CreateDomain would have succeeded (the only theoretical divergence — a machine where domain creation throws but reflection-only load succeeds — is unreachable in practice, and the conservative null-return there would only proceed with discovery either way).

Removes the last real ObjectModel dependency from TestSourceHandler (it now carries only string-literal well-known-assembly names, which don't reference the package).

Verification

  • All real TFMs build 0-warning (UWP builds via full msbuild in CI, unaffected — change is #if NETFRAMEWORK-guarded).
  • MSTestAdapter.PlatformServices.UnitTests: 935/935 (net462), 897/897 (net8.0).
  • DesktopTestSourceTests — the direct IsAssemblyReferenced net — 7/7, covering: assembly referenced (true), not referenced (false), null name (true), null source (true).
  • PlatformServices.Desktop.IntegrationTests: 15/15 (net462).
  • Expert-reviewer pass.

Stacking

Stacks on #9630 (Phase 6e-4c1); base branch dev/amauryleve/vstest-decoupling-sourcehost. Review/merge after the earlier PRs in the chain reach the base. Do not squash-rebase the base.

…Phase 6e-4c2)
TestSourceHandler.IsAssemblyReferenced (netfx) used
AssemblyHelper.DoesReferencesAssembly from
Microsoft.VisualStudio.TestPlatform.ObjectModel.Utilities to decide whether a
source assembly references the test framework before running discovery.
Replace that call with a local neutral DoesSourceReferenceAssembly helper that
reproduces the exact observable behavior of the VSTest implementation:
- ReflectionOnlyLoadFrom(source), then GetReferencedAssemblies().
- Match a referenced assembly by simple name (OrdinalIgnoreCase) plus public
key token bytes; version is ignored -- identical to
AssemblyLoadWorker.CheckAssemblyReference.
- Null/empty source or null reference assembly returns null (undeterminable).
- Any exception returns null so discovery proceeds conservatively.
Fidelity note: the VSTest DoesReferencesAssembly created a child AppDomain and an
AssemblyLoadWorker instance, but then called the *static*
AssemblyLoadWorker.CheckAssemblyReference -- the worker instance is never used and
the ReflectionOnlyLoadFrom actually runs in the current domain. The child domain
is therefore dead code for the result, so omitting it is behavior-preserving for
every input where AppDomain creation would have succeeded.
Removes the last real ObjectModel dependency from TestSourceHandler.
Verified: PlatformServices.UnitTests 935 (net462) / 897 (net8.0);
DesktopTestSourceTests IsAssemblyReferenced branches 7/7 (referenced,
not-referenced, null-name, null-source); PlatformServices.Desktop.IntegrationTests
15/15. All real TFMs build 0-warning.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@Evangelink
Amaury Levé (Evangelink) marked this pull request as ready for review July 5, 2026 19:24
@Evangelink
Amaury Levé (Evangelink) merged commit c3c9653 into dev/amauryleve/vstest-decoupling-sourcehostJul 5, 2026
20 checks passed
@Evangelink
Amaury Levé (Evangelink) deleted the dev/amauryleve/vstest-decoupling-sourcehandler branch July 5, 2026 19:24

@github-actionsgithub-actionsBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

🤖 Automated review by GitHub Copilot. Posted via a maintainer's GitHub token, so it appears under their account — the account owner did not write or approve this content personally. Generated by the Expert Code Review workflow. To request a follow-up action, reply by tagging @copilot directly.

Review Summary — Phase 6e-4c2 (TestSourceHandler ObjectModel removal)

The architectural goal is correct and well-executed. The decoupling is clean, the #if NETFRAMEWORK guard is properly applied, and the fidelity note in the PR description is accurate about the dead child-AppDomain. One MAJOR null-safety bug in the helper method needs a follow-up fix; two MINOR polish items are noted.


Verdict: NEEDS WORK

#DimensionStatusSeverityNote
1Algorithmic Correctness⚠️ MinorMAJORLogic is correct for signed assemblies (the production case). For unsigned assemblies (null PKT), ArePublicKeyTokensEqual throws NRE → caught → null → proceed. Accidentally correct outcome, wrong code path.
2Threading & Concurrency✅ CleanStatic method, local variables only, no shared state.
3Security & IPC Contract Safety✅ CleanReflectionOnlyLoadFrom is read-only; no code execution; source path supplied by caller.
4Public API & Binary Compatibility✅ CleanNo public API changes; removed using for ObjectModel is a positive dependency reduction.
5Memory Management & Resource Leaks✅ CleanReflection-only load context limitations are pre-existing; behaviour unchanged from original call.
6Error Handling⚠️ MinorMINORBare catch absorbs all exceptions silently; no diagnostic trace makes debugging difficult (see inline comment on line 135).
7Naming & Code Style✅ CleanDoesSourceReferenceAssembly, ArePublicKeyTokensEqual are clear verb-led names; file-scoped namespace, no abbreviations.
8Documentation & Comments⚠️ MinorMINORXML doc on DoesSourceReferenceAssembly is accurate. Stale comment on line 78 ("different app domain") is now inaccurate (see inline comment).
9Test Coverage✅ Clean7/7 DesktopTestSourceTests reported; edge cases (null name, null source, referenced/not-referenced) are covered per PR description. Null-PKT path not explicitly tested but the outer catch provides the safety net.
10Performance✅ CleanStringComparison.OrdinalIgnoreCase used; no unnecessary allocations in the loop.
11Cross-TFM Correctness✅ CleanAll new code is inside #if NETFRAMEWORK; Assembly.ReflectionOnlyLoadFrom is .NET Framework-only.
12LocalizationN/ANo user-facing strings added.
13Dependency Management✅ CleanPositive change: removes the ObjectModel using directive from PlatformServices.
14Telemetry & ObservabilityN/ANo new user-visible behaviour; existing discovery flow preserved.
15CI/CD & Build System✅ CleanNo build script changes; guarded by #if NETFRAMEWORK.
16Backward Compatibility✅ CleanIsAssemblyReferenced return logic is unchanged; callers see identical results for all signed-assembly inputs.
17Null Safety❌ IssueMAJORGetPublicKeyToken() returns null on .NET Framework for unsigned assemblies. ArePublicKeyTokensEqual(byte[] left, byte[] right) has no null guards; left.Length throws NullReferenceException. referenceAssemblyPublicKeyToken declared byte[] rather than byte[]?, hiding the nullable concern. See inline comment on line 144.
18Exception Safety✅ CleanNo partial-state mutations; catch is for observation, not rollback.
19Immutability & Value Semantics✅ CleanPure static methods operating on immutable inputs.
20Design Patterns & Architecture✅ CleanAligns perfectly with the stated goal; dependency removed at the correct layer.
21Logging Correctness⚠️ MinorMINORNo diagnostic output when the catch fires; a Debug.WriteLine would cost nothing and aid debugging.
22Scope & PR Hygiene✅ CleanSingle concern; no unrelated changes; no dead code introduced.

Required follow-up

ArePublicKeyTokensEqual — add null guards (line 144, inline comment posted):

privatestaticboolArePublicKeyTokensEqual(byte[]?left,byte[]?right){if(leftisnull&&rightisnull)returntrue;if(leftisnull||rightisnull)returnfalse;if(left.Length!=right.Length)returnfalse;for(inti=0;i<left.Length;++i){if(left[i]!=right[i])returnfalse;}returntrue;}

Also fix the assignment on line 113 to byte[]? referenceAssemblyPublicKeyToken = referenceAssembly.GetPublicKeyToken();.

In practice MSTest is always shipped signed so null PKTs will not occur in the field, but the code is objectively wrong and should be fixed before being built upon in the next phase of this decoupling series.


private static bool ArePublicKeyTokensEqual(byte[] left, byte[] right)
{
if (left.Length != right.Length)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MAJOR — Null Safety]GetPublicKeyToken() returns null on .NET Framework for unsigned assemblies (those with no public key token). Both left (from referencedAssembly.GetPublicKeyToken()) and right (from referenceAssembly.GetPublicKeyToken()) can therefore be null, so left.Length on this line throws NullReferenceException. The outer catch absorbs that and returns null (→ proceed with discovery), so the process doesn't crash; but there are two problems:

  1. For two assemblies that both carry no public-key token, the semantically correct result is true (name match + neither is signed = same unsigned identity). Instead the caller gets a swallowed exception and falls through to "proceed" via null, which is accidentally correct today but for the wrong reason.
  2. referenceAssemblyPublicKeyToken on line 113 is declared byte[] (non-nullable), masking the actual nullable return of GetPublicKeyToken(). It should be byte[]?.

Suggested fix:

privatestaticboolArePublicKeyTokensEqual(byte[]?left,byte[]?right){if(leftisnull&&rightisnull)returntrue;if(leftisnull||rightisnull)returnfalse;if(left.Length!=right.Length)returnfalse;for(inti=0;i<left.Length;++i){if(left[i]!=right[i])returnfalse;}returntrue;}

Also update the assignment on line 113:

byte[]?referenceAssemblyPublicKeyToken=referenceAssembly.GetPublicKeyToken();


return false;
}
catch

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MINOR — Bare Catch / No Diagnostics] The catch block silently swallows every exception — file-not-found, BadImageFormatException, security exceptions, and the null-deref discussed on ArePublicKeyTokensEqual all land here invisibly. The conservative null return is intentional and correct, but without any trace output it is very hard to tell the difference between "file legitimately couldn't be opened" and "code bug".

Suggestion — at minimum emit a Debug.WriteLine (no user-visible surface, zero overhead in production):

catch(Exceptionex){Debug.WriteLine($"[MSTest] DoesSourceReferenceAssembly could not inspect '{source}': {ex.Message}");returnnull;}

@@ -79,7 +76,7 @@ public bool IsAssemblyReferenced(AssemblyName assemblyName, string source)
{
#if NETFRAMEWORK
// This loads the dll in a different app domain. We can optimize this to load in the current domain since this code could be run in a new app domain anyway.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MINOR — Stale Comment] The comment says "This loads the dll in a different app domain", but the new implementation (DoesSourceReferenceAssembly) uses Assembly.ReflectionOnlyLoadFrom in the current AppDomain — no child domain is created. Both sentences are now inaccurate (the second was an optimisation note that is now moot since the domain split was intentionally dropped).

Suggested replacement:

// Reflection-only loads the assembly to inspect its references without executing any code from it.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs/author-feedbackWaiting on the original author.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@Evangelink