Skip to content

feat(mstest): add Compono.MSTest package (ADR-0057) - #120

Merged
ncipollina merged 5 commits into
mainfrom
docs/mstest-research-and-adr-0057
Sep 2, 2026
Merged

feat(mstest): add Compono.MSTest package (ADR-0057)#120
ncipollina merged 5 commits into
mainfrom
docs/mstest-research-and-adr-0057

Conversation

@ncipollina

Copy link
Copy Markdown
Contributor

🚀 Pull Request

📋 Summary

Adds Compono.MSTest, the third test-framework integration package alongside Compono.XunitV3/Compono.TUnit: [TestMethod] + [Compose]/[Compose<TProfile>]/[Compose<TProfile, TConfig>] as ITestDataSource attributes, [Shared] parameter sharing, and generator discovery — the same Compono-facing attribute family and semantics as the existing packages, adapted to MSTest's own synchronous, non-owning ITestDataSource contract. Implements ADR-0057 / PLAN-0057, both now Accepted/Done.

Two real findings during implementation forced ADR amendments rather than being papered over: MSTest.TestFramework 3.x and 4.x ship under different, binary-incompatible assembly identities, so the supported floor is 4.0.0, not 3.0.0 (ADR-0057 Amendment 1); and GetDisplayName is a discovery/listing-time MSTest hook only, never called during ordinary test execution under either runner (ADR-0057 Amendment 2). A related, pre-existing Native AOT/trimming gap was found incidentally in Compono.XunitV3's own ConfigProfileBinder — tracked separately as #119, explicitly out of scope here (no Compono.XunitV3 source touched).


📝 Changes

Compono.MSTest package (src/Compono.MSTest)

  • ComposeAttribute : Attribute, ITestDataSource, ComposeAttribute<TProfile>, ComposeAttribute<TProfile, TConfig>, SharedAttribute — the frozen public API from ADR-0057 §6.
  • Package-local Binding/ (BindingPlan, ParameterBindingPlan, PositionalArgumentBinder, ConfigProfileBinder, RowInvokers), ported from Compono.XunitV3's pattern (MSTest hands the same MethodInfo/ParameterInfo shape xUnit v3 does). RowInvokers dispatches through the existing, unchanged core RowInvokerRegistry/CompositionRow from its first commit — no throwaway reflection-based dispatch.
  • Seed/display-name bridge: since GetData/GetDisplayName share no MSTest-provided context object, a ConditionalWeakTable<object?[], object> keyed by the exact row-array instance carries the seed across the two calls — verified correct under both runners' discovery/listing output.
  • Depends on MSTest.TestFramework only (not the MSTest umbrella package), floor 4.0.0 (ADR-0057 Amendment 1).
  • Native AOT-safe: ConfigProfileBinder's TConfig/TProfile construction needed the same DynamicallyAccessedMembers(PublicConstructors) fix Compono.TUnit already required (ADR-0041 Amendment 1) — found via a real dotnet publish -p:PublishAot=true failure, fixed, and re-verified with zero Compono.MSTest-attributable trim warnings.

Generator discovery (src/Compono.Generators)

  • Three new metadata-name registrations (Compono.MSTest.ComposeAttribute/ `1/ `2) feeding the existing, already attribute-family-agnostic ComposeMethodDiscovery — additive only, 299/299 existing generator tests unaffected. New snapshot test proves a type reachable only through a Compono.MSTest-attributed parameter gets a generated plan + RowInvokerRegistry registration.

Tests

  • test/Compono.MSTest.Tests — binding/signature validation, inline values, [Shared], profile/config binding, negative-seed ordering, [DataRow]and[DynamicData] + [Compose] independent-row coexistence, a real-runner row-identity proof (RealRunnerRowIdentityTests), a reflection-pattern source guard, and the public-API-surface lock.
  • test/Compono.MSTest.SampleTests — the full attribute family through the real packaged Compono.MSTest → Compono dependency chain (never ProjectReference), including Compono.NSubstitute integration.
  • test/Compono.MSTest.AotSmokeTest — real dotnet publish -p:PublishAot=true + run against the packaged dependency chain.

Docs/skill/eval — new docs/packages/compono-mstest.md, skills/compono/references/mstest.md; README.md/docs/index.md/docs/packages/index.md/docs/public-api.md/docs/concepts/shared-values.md/docs/getting-started/installation.md/docs/roadmap/future-packages.md synced (also caught docs/index.md already missing Compono.Http/Compono.Logging, fixed alongside); regenerated docs/reference/api/Compono.MSTest/; two new evals + a scored benchmark in skills/compono-evals/.

CI/packagingdocs.yml, package-validation.yaml, inspect-packed-nupkgs.sh, generate-api-reference.sh all updated and exercised for real (a fresh pack of all ten publishable packages passed every inspect-packed-nupkgs.sh assertion for Compono.MSTest, including confirming no embedded Compono.Generators.dll).

External packaged-consumer validation — a purpose-built, disposable external fixture (not a real application, not retained) validated via scripts/dogfood-validate.sh with no script changes needed. This is explicitly consumer validation, not genuine dogfooding — no real MSTest consumer exists in any LayeredCraft/ncipollina repo yet. Full reproducible spec (exact layout, csproj, command, resolved versions, result) is recorded in PLAN-0057's Notes.


🧪 Validation

  • Whole-repo: dotnet build Compono.slnx — 0 errors. dotnet test Compono.slnx964/964 passing, zero regressions in any existing package.
  • MSTest version/runner matrix, actual resolved versions confirmed by direct project.assets.json inspection (not requested versions):
    MTPClassic VSTest
    MSTest family 4.0.0 (floor)35/35 (unit) · 5/5 (sample)35/35 (unit) · 5/5 (sample)
    MSTest family 4.3.3 (current)35/35 (unit) · 5/5 (sample)35/35 (unit) · 5/5 (sample)
  • Native AOT: real dotnet publish -c Release -p:PublishAot=true -r osx-arm64 --self-contained true + run, both [Compose] and [Compose<TProfile, TConfig>], zero Compono.MSTest-attributable trim warnings (-p:TrimmerSingleWarn=false confirmed).
  • Generator regression: Compono.Generators.Tests 299/299 (298 existing + 1 new snapshot).
  • Packaged-consumer: test/Compono.MSTest.SampleTests 5/5 through the real NuGet dependency chain, both runners.
  • External packaged-consumer validation: scripts/dogfood-validate.sh — PASS, 9/9, exit 0, fixture git tree byte-identical before/after.
  • Independent adversarial review surfaced one HIGH (the version matrix above was originally proven only against 4.3.3-resolved dependencies, not a genuine 4.0.0 leg) and three MEDIUM findings, all resolved with real re-verification (see PLAN-0057 Notes' "Adversarial review response" entry for the full account) rather than narrowed claims.

🧩 Related Issues

Relates to #119 (Compono.XunitV3.ConfigProfileBinder's own Native AOT gap, found incidentally, tracked and scoped separately — no Compono.XunitV3 source changed here).


📦 Release Notes

Adds Compono.MSTest (prerelease, MSTest.TestFramework 4.0.0+ required) alongside Compono.XunitV3/Compono.TUnit.


💬 Notes for Reviewers

  • ADR-0057 carries two amendments from real implementation evidence (assembly-identity incompatibility forcing the 4.0.0 floor; GetDisplayName discovery-only timing) — both are architectural corrections, not just doc tweaks, and are the most load-bearing part of this diff to review.
  • ADR-0041 Amendment 4 + issue [Bug]: Compono.XunitV3's ConfigProfileBinder likely has the same unannotated Native AOT/trimming gap fixed in Compono.MSTest #119 are informational-only in this PR — no Compono.XunitV3 behavior changes.
  • test/Compono.MSTest.SampleTests/test/Compono.MSTest.AotSmokeTest are deliberately not in Compono.slnx, matching the existing Compono.TUnit.SampleTests/Compono.TUnit.AotSmokeTest convention (manual/local-feed-driven proofs).

ncipollinaand others added 2 commits August 29, 2026 07:00
…est; accept Compono.MSTest (ADR-0057)
- RESEARCH-0015: disposal/ownership semantics — Outcome C, post-1.0 additive
feature, no 1.0 API change required.
- RESEARCH-0016: async composition viability — Outcome C, fully additive
post-1.0 capability.
- RESEARCH-0017: MSTest integration viability — Outcome A, add
Compono.MSTest before 1.0 (includes VSTest/MTP discovery-execution spike
findings and MSTest version-floor capability matrix).
- ADR-0057: Compono.MSTest package design, Accepted — ITestDataSource-based
[Compose] integration reusing CompositionRow/RowInvokerRegistry, no new
composition engine.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Adds the Compono.MSTest integration package: [TestMethod] + [Compose]/
[Compose<TProfile>]/[Compose<TProfile, TConfig>] as ITestDataSource
attributes, [Shared] parameter sharing, and generator discovery support,
matching Compono.XunitV3/Compono.TUnit's existing attribute family and
semantics.
Implementation evidence forced two ADR-0057 amendments: MSTest.TestFramework
3.x and 4.x ship under different, binary-incompatible assembly identities,
so the supported floor is 4.0.0, not 3.0.0 (Amendment 1); and
GetDisplayName is a discovery/listing-time MSTest hook, not visible during
ordinary test execution under either MTP or the classic VSTest adapter
(Amendment 2). A related Native AOT/trimming gap in Compono.XunitV3's own
ConfigProfileBinder was found incidentally (ADR-0041 Amendment 4) and is
tracked separately as #119, out of scope here.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GMQEjFrUVGYkctE3ECfRuA

@chatgpt-codex-connectorchatgpt-codex-connectorBot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit:bed19c48ba

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment threaddocs/packages/compono-mstest.md Outdated
Comment threaddocs/adr/0057-compono-mstest-package-design.md
…and-adr-0057
# Conflicts:
#	skills/compono-evals/evals.json
@github-actionsgithub-actionsBot added the type: feat New feature label Sep 2, 2026
@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Swish!

Reviewed commit:b44fc6912d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@ncipollinancipollina left a comment

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR #120 review — ADR/PLAN-0057 Compono.MSTest

Reviewed PR head b44fc6912dc48d4e17dee3cb34a01e1e524f1c0c against base 481854f704287547dd6be4cedcd7634ef3cc58b6 using the engineering-workflow PR review process.

Verdict

Request changes: no remaining product/architecture blockers found in the MSTest implementation itself, but the PR's current GitHub package-validation check is failing and must be green before merge.

Findings

BLOCKER

None.

HIGH

  1. Failing package-validation CI at the current PR head
    • Evidence: gh pr checks 120 reports package-validation failed for run 33651896016, job 100320715892.
    • Failed step: Inspect packed .nupkg contents.
    • Log excerpt: FAIL: Compono.TUnit's .nuspec dependency on TUnit.Core is '[1.65.63, 2.0.0)', expected the intended tested range '[1.65.38, 2.0.0)'.
    • Impact: The package-readiness gate is red. Even though the failing package is Compono.TUnit, the current PR head cannot be accepted/merged with package validation failing.
    • Recommended correction: Either make the packed Compono.TUnit nuspec emit the intended [1.65.38, 2.0.0) range again, or update .github/scripts/inspect-packed-nupkgs.sh/the relevant ADR-plan documentation if [1.65.63, 2.0.0) is now the true tested lower bound. Re-run package-validation to green on this PR head.

MEDIUM

None found in the current PR head. Previously identified gaps for MSTest 4.0.0 floor evidence, GetDisplayName documentation, [DynamicData] executable coverage, and external packaged-consumer validation evidence appear addressed in the current plan/docs/tests.

LOW

  1. Stale test comment references removed GetData logging hook
    • Location: test/Compono.MSTest.Tests/RealRunnerRowIdentityTests.cs, class cleanup comment.
    • Evidence: The comment says GetData counts were captured via COMPONO_MSTEST_GETDATA_LOG and says “see ComposeAttribute.cs”; current src/Compono.MSTest/ComposeAttribute.cs only increments GetDataCallCount and has no environment-variable logging hook.
    • Impact: Documentation-only drift in a test comment; it can mislead the next reviewer/debugger but does not affect runtime behavior.
    • Recommended correction: Reword the comment to say the log hook was temporary/implementation-time evidence recorded in PLAN-0057, or remove the see ComposeAttribute.cs claim.

Local verification performed

  • git status --short before review-file creation: clean.
  • dotnet build --no-restore: passed locally, 0 warnings/0 errors.
  • dotnet test --no-build: started; the harness timed it out after MSTest and many other projects passed, so I do not count this as a full-suite pass.
  • dotnet test test/Compono.MSTest.SampleTests/Compono.MSTest.SampleTests.csproj --no-restore --no-build -f net10.0: passed, 5/5.
  • GitHub checks inspected with gh pr checks 120; package-validation failed as above.

Notes

  • No implementation files were modified as part of this review.
  • This summary file is the requested local durable review artifact.

@ncipollina

Copy link
Copy Markdown
ContributorAuthor

CI update after my review comment: build / build has now also completed as failed at the current PR head (33651896567 / job 100320717421). The job metadata shows Build Code succeeded and Test Code was cancelled, but I could not retrieve detailed step logs from this environment. This doesn’t change the substantive finding: current PR-head CI is red (package-validation plus now build / build), so please get the checks back to green before merge.

…tch Directory.Packages.props
Dependabot PR #121 (merged to main after this branch's last merge) bumped
TUnit.Core/TUnit from 1.65.38 to 1.65.63 in Directory.Packages.props. This
script hardcodes the expected packed-nuspec dependency range as a literal
string per package, which dependabot has no way to know to update -
main itself independently fails this same assertion as of #121, unrelated
to PLAN-0057's own scope. Smallest correct fix: update the literal to match
the now-current floor.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GMQEjFrUVGYkctE3ECfRuA
@ncipollina

Copy link
Copy Markdown
ContributorAuthor

Re: #120 (comment) — Root cause confirmed for both:

package-validation (the HIGH finding): not caused by PR #120 or its conflict-resolution merge. Dependabot PR #121 landed on mainafter this branch's last merge, bumping TUnit.Core/TUnit from 1.65.38 to 1.65.63 in Directory.Packages.props. .github/scripts/inspect-packed-nupkgs.sh hardcodes the expected packed-nuspec dependency range as a literal string per package — dependabot has no way to know to update it, so main itself independently fails this exact assertion as of #121 (verified directly: packed Compono.TUnit from a clean origin/main checkout also produces TUnit.Core [1.65.63, 2.0.0), not the script's stale [1.65.38, 2.0.0) expectation). Fixed in 5cfe446 with the smallest correct change — updating the one literal to match the now-current floor. Locally re-verified: fresh pack of all ten publishable packages passes every inspect-packed-nupkgs.sh assertion.

build / build: the "Test Code" step was cancelled both times, with "The runner has received a shutdown signal" — infrastructure-level, not a code/test failure (every test that ran before cancellation passed; no assertion failures anywhere in the logs). Confirmed as transient CI flakiness, not caused by this PR: an unrelated PR (feat/testdoubles-invocation-aware-callbacks) hit the identical failure signature earlier the same day and passed on retry with no code change. build / build is now green after a second rerun, and green again on the pushed fix commit.

All checks (build, build / build, package-validation, validate, draft / autolabel) are passing as of 5cfe446.

@ncipollinancipollina left a comment

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎉 Follow-up review of the corrective delta from b44fc691 to 5cfe446: the prior CI blockers are resolved. The TUnit.Core package-validation mismatch was a stale expected range after main/Dependabot moved Directory.Packages.props to [1.65.63, 2.0.0), and the validation script now matches the packed nuspec; current package-validation is green. Current build / build is also green, including the Test Code step. The earlier Codex documentation findings remain corrected, all review threads are resolved, and I found no new merge-blocking issues in the corrective delta. Recommendation: READY TO MERGE.

@ncipollina
ncipollina merged commit 1301101 into mainSep 2, 2026
8 checks passed
@ncipollina
ncipollina deleted the docs/mstest-research-and-adr-0057 branch September 2, 2026 16:15
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type: featNew feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@ncipollina