Uh oh!
There was an error while loading. Please reload this page.
feat: add server conformance tests for SEP-2575 - #271
Merged
Conversation
9 tasks
pcarleton
left a comment
Member
There was a problem hiding this comment.
Thanks for this — covers a good chunk of the version-negotiation and discover surface. (reminder the full SEP-2575 traceability YAML in #273 as a shared reference for this and #270.)
Coverage: the YAML has 20 server-side check: rows; this PR covers 7 (discover, version-error, 400-on-unsupported, header-mismatch, 404-on-unknown, missing-capability ×2). The big uncovered areas are subscriptions/listen (4 rows), HTTP cancellation (3 rows - 2 HTTP, 1 stdio), the core statelessness invariants (no-prior-context / no-connection-reuse), and server-no-log-without-loglevel. Fine to land those as follow-ups, but worth flagging which are intentionally deferred.
Correctness:
- MUST→WARNING severity —
stateless-error-missing-capabilityandstateless-http-missing-capabilityuseWARNINGon failure, but the spec text is MUST for both-32003and HTTP 400. If the worry is the test tool not existing, skip the check rather than downgrade. stateless-version-response-headeris dead — only ever pushed in acatchblock (stateless.ts:360), never in the happy path. Also no spec backing: there's no server-MUST-return-header. Suggest dropping it.- Registered twice in
index.ts(bothpendingClientScenariosListandallClientScenariosList). We should clean this up, but you can put it inalland we filter it by spec version tag. - "Absent header" test isn't absent —
{'MCP-Protocol-Version': ''}sets an empty-string header; the helper always injects the default, so the header is never truly omitted. Needs a delete-key path insendRpc. stateless-version-header-mismatchonly checks HTTP 400; spec says it SHOULD include-32001 HeaderMismatchin the body.stateless-removed-*(5 checks) — there's no normative MUST forbiddingping/initialize; the only relevant text is the generic "unknown method → 404 + -32601". These checks don't assert the 404, and the example returns 200 for them. Suggest folding into onesep-2575-http-server-method-not-found-404slug parametrized by method.stateless-meta-missing*(4 checks) — the draft spec pages only put the MUST on the client (the SEP doc has a server-MUST-reject, but it didn't make it into basic/index.mdx — possibly a spec bug to fix). Either way, one check slug with the field name in details is probably better here.noDriftrequires identical array order between discover and error; spec doesn't mandate order — set comparison.- Check IDs — should be
sep-2575-*to match #273 (same note left on #270);specReferencesshould point to spec anchors not the SEP PR.
Structure: ~60% of the 749 lines is duplicated catch-block boilerplate re-pushing id/name/description. A runCheck(id, desc, fn) helper would cut it to ~250 lines and prevent the try/catch description drift (e.g. stateless-version-unsupported has different descriptions in try vs catch).
Negative test could also be tightened — expect(failures.length).toBeGreaterThan(0) doesn't pin which checks fail; #270 uses expectedFailureSlugs which is the pattern we want.
Yuan325
commented
May 15, 2026
ContributorAuthor
Thankyou for the thorough review @pcarleton , I've updated the test cases based on your review and added new checks according to #273. For the client capability checks, it should only check for payload structure and http status if it detects -32003. These tests were also checked against Toolbox's implementation of SEP-2575 (googleapis/mcp-toolbox#3211). Thank you! |
commit: |
…le capability checks - Add the required `source` field (DRAFT) so the scenario typechecks and is selectable via --spec-version. Without it the class did not satisfy the ClientScenario interface and s.source was dereferenced by the spec-version filter at list time. - Teach runCheck about a SKIPPED status and use it for the two client-capability checks when the server does not return -32003, instead of reporting a green PASS for a requirement that was never exercised.
9 tasks
The subscriptions/listen, statelessness-invariant, list-changed and disconnect-is-cancel checks emitted SUCCESS without probing the server, which inflated coverage in the traceability report. Drop them until they have real assertions; the corresponding rows remain declared in src/seps/sep-2575.yaml so traceability shows them as not-yet-covered. Also fix the checkErrorId helper to push under the sep-2575-http-server-error-jsonrpc-id slug so its failure path actually short-circuits the aggregate SUCCESS check at the end of the scenario.
| } | ||
| ); | ||
| await runCheck( |
Member
There was a problem hiding this comment.
@Yuan325 I deleted these placeholder checks. I want to either have the checks verify things or be marked as excluded if they can't be tested
pcarleton
commented
May 19, 2026
Member
Going to merge this in, but note that several of the SEP-2575 server requirements still need tests (The blank always-pass tests didn't seem useful). I think many of those are actually testable so I don't think we need to mark them as excluded, but let's look at that separately from this PR. |
Uh oh!
There was an error while loading. Please reload this page.
Yuan325
commented
May 19, 2026
ContributorAuthor
@pcarleton Can you please clarify what are the blank always-pass tests? I can take a look at those test cases that we want to add to SEP-2575. Is it the ones that are removed from 568718a? Thank you! |
pcarleton
commented
May 19, 2026
Member
@Yuan325 Yes, those tests did not check anything and returned |
pcarleton added a commit
that referenced
this pull request
May 19, 2026
* Add SEP-2350 scope-accumulation check to auth/scope-step-up SEP-2350 clarifies that on step-up re-authorization, clients SHOULD compute the union of previously-granted and newly-challenged scopes so they don't lose permissions for other operations. - Traceability yaml: 1 client check (two spec sentences merged), 1 server check tracked for a future server-side scenario, 1 excluded reword - ScopeStepUpAuthScenario now challenges with only the missing scope so union accumulation is observable on the second authorization request - New check sep-2350-scope-union-on-reauth (WARNING when the previously granted scope is dropped) - everything-client withOAuthRetry now accumulates prior token scope into the re-auth scope (passing example) - New auth-test-echo-scope negative client + vitest case * feat: add client conformance tests for SEP-2575 (#270) * feat: merge latest request-metadata scenario stacked branch * test: implement optional capability conditional checks and simulated negotiation retry * fix(sep-2243): collapse check IDs to one-per-requirement (#287) The sep-2243 scenario check IDs drifted from the requirement-traceability yaml in #259 — the code emitted one ID per test case while the yaml declares one per normative requirement, so the IDs no longer matched. Rename the emitted IDs so a check ID maps to a single MUST/SHOULD and is emitted once per case (the per-case detail moves to name/description), matching the repo's existing 'same id, vary status/message' convention: - mcp-method-header-* / mcp-name-header-* -> client-includes-standard-headers - reject-invalid-tool-* / keep-valid-tool -> client-reject-invalid-tool - server-accepts-{lower,upper}case-name -> header-name-case-insensitive - server reject status checks (mismatch/missing/case x method/name) -> server-reject-invalid-headers - their error-code variants -> server-reject-error-code Merge the yaml's server-reject-mismatch + server-reject-status into one server-reject-invalid-headers MUST (HTTP 400 on a header-validation failure); keep server-reject-error-code as the SHOULD. Base64/custom-header rejection checks keep their own param-validation IDs (out of scope here). Gates and the whitespace-acceptance check keep their scenario-matching names. No behavior change — only check IDs, descriptions, and the yaml. * feat: `sdk` subcommand to run local conformance against any SDK ref (#277) * feat: add sdk subcommand to run conformance against any SDK ref (#250) * Revise sdk runner: explicit --mode, KNOWN_SDKS-only config, v1/v2 entries Addresses review feedback on the sdk subcommand: - Require --mode (client|server) and remove "both". Each invocation now tests exactly one side with its own exit code; the old default ran client then server but combined exit codes with ||=, which skipped the server side entirely whenever the client run failed. - Resolve build/run config from KNOWN_SDKS + CLI flags only; drop the conformance.config.yaml loader (no SDK ships one yet). The Zod schema stays as the type for the built-in entries. - Split the typescript entry by major version: typescript-sdk (v2/main, pnpm install + build:all, expected-failures.yaml) and typescript-sdk-v1 (v1.x, npm ci + build, conformance-baseline.yml). An entry may set `repo` (real clone target for an alias) and `defaultRef` (branch used when no @ref is given). parseSdkSpec now leaves ref undefined when omitted so defaultRef can apply; a trailing @ is treated as no ref. - Key the clone cache by ref (<repo>/<ref>) so different refs of the same repo no longer share one checkout. - Bound the server readiness probe with a per-request AbortSignal timeout so a server that accepts the socket but never responds can't hang past the overall deadline. * fix(sdk-runner): resolve -o to absolute path; add --expected-failures override; replaceAll for safeName * feat: add SEP conformance traceability manifest (#288) squashed; see PR #288 body * feat: add server conformance tests for SEP-2575 (#271) * feat: add server conformance tests for SEP-2575 * refactor and add new checks * Fix stateless scenario source field and report SKIPPED for inapplicable capability checks - Add the required `source` field (DRAFT) so the scenario typechecks and is selectable via --spec-version. Without it the class did not satisfy the ClientScenario interface and s.source was dereferenced by the spec-version filter at list time. - Teach runCheck about a SKIPPED status and use it for the two client-capability checks when the server does not return -32003, instead of reporting a green PASS for a requirement that was never exercised. * Remove unimplemented placeholder checks from stateless server scenario The subscriptions/listen, statelessness-invariant, list-changed and disconnect-is-cancel checks emitted SUCCESS without probing the server, which inflated coverage in the traceability report. Drop them until they have real assertions; the corresponding rows remain declared in src/seps/sep-2575.yaml so traceability shows them as not-yet-covered. Also fix the checkErrorId helper to push under the sep-2575-http-server-error-jsonrpc-id slug so its failure path actually short-circuits the aggregate SUCCESS check at the end of the scenario. --------- Co-authored-by: Paul Carleton <paulc@anthropic.com> * SEP-2352: authorization-server migration scenario (#286) * Add SEP-2352 authorization-server migration scenario SEP-2352 requires that client credentials are bound to the issuing authorization server: when PRM authorization_servers changes to a new issuer, clients MUST re-register and MUST NOT reuse the previous AS's client credentials. - Traceability yaml: 3 checks, 3 excluded (internal state / UI) - New auth/authorization-server-migration scenario (draft suite): two auth servers; PRM flips from AS1 to AS2 after the first authenticated request; AS2 asserts it received a fresh /register and never saw AS1's client_id at /authorize or /token - ConformanceOAuthProvider gains invalidateCredentials and bindIssuer so the everything-client can key credentials by issuer (passing example) - everything-client adds an issuer-aware handler for this scenario that re-reads PRM on each 401 and rebinds before re-authorizing - auth-test-reuse-credentials negative client + vitest case * Drop application_type from DCR metadata (not in SDK OAuthClientMetadata type) --------- Co-authored-by: Paul Carleton <paulc@anthropic.com> * ci(traceability): enable corepack so the SDK build can use pnpm (#289) The refresh run failed with 'pnpm: not found' — the reference SDK's build command (typescript-sdk: pnpm install && pnpm run build:all) needs pnpm on PATH. Add 'corepack enable' to the run job. * ci(traceability): tolerate SDK conformance failures in the run step (#290) The `sdk` command exits non-zero when the SDK has conformance failures not in its baseline. The traceability manifest only needs the emitted check IDs (written regardless of pass/fail), so a failing SDK must not fail the refresh. Add `|| true`; the existing 'no results produced' guard catches a genuinely broken run. * chore: refresh SEP traceability manifest (typescript-sdk@main) (#291) Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * Make scopesSupported disjoint from challenge scopes so the SEP-2350 union check can't be satisfied by scopes_supported ∪ challenge A client that unions scopes_supported with the challenge — instead of its prior grant with the challenge — would have requested mcp:basic mcp:write and falsely passed sep-2350-scope-union-on-reauth, since scopesSupported happened to coincide with the previously-granted scope. Setting it to a disjoint value makes the check actually require the prior token's scope. Spec backing: clients MUST NOT assume any set relationship between the challenged scope set and scopes_supported, so a disjoint advertisement is realistic. * SEP-837: application_type check in DCR registration (#284) * Add SEP-837 application_type check to DCR registration SEP-837 requires MCP clients to specify an appropriate application_type during Dynamic Client Registration so OIDC authorization servers can apply the correct redirect-URI constraints. - Traceability yaml: 1 check (presence + valid value), 4 excluded (class-specific SHOULDs unobservable; UI/robustness) - Check added in the shared createAuthServer DCR handler so it fires in every auth scenario that performs DCR (no new scenario) - withOAuthRetry now sets application_type: native (passing example; the conformance example clients are CLI tools) - New auth-test-no-application-type negative client + vitest case * Widen ConformanceOAuthProvider metadata type for application_type Fixes CI typecheck (TS2353 at withOAuthRetry.ts:76): the SDK's OAuthClientMetadataSchema doesn't include application_type yet. registerClient() spreads clientMetadata verbatim into the /register POST body, so a local type intersection is sufficient to get the field on the wire without an SDK release. * Set application_type in runAuthMigrationClient The SEP-2352 authorization-server-migration handler constructs its own ConformanceOAuthProvider (not via withOAuthRetry), so it was missing application_type after rebasing onto main. The scenario does DCR twice (old AS, new AS) and the SEP-837 check fires on each. --------- Co-authored-by: Paul Carleton <paulc@anthropic.com> * feat: add conformance tests for iss parameter (SEP-2468) (#220) * feat: add conformance tests for iss parameter (SEP-2468) Adds 5 draft conformance scenarios testing RFC 9207 issuer parameter validation in OAuth authorization responses: - auth/iss-supported: server advertises support and sends correct iss - auth/iss-not-advertised: server omits iss parameter entirely - auth/iss-supported-missing: client must reject missing iss when required - auth/iss-wrong-issuer: client must reject mismatched iss value - auth/iss-unexpected: client must reject iss when not advertised Also adds auth-test-iss-validation.ts, a reference client that correctly validates iss per RFC 9207, and negative tests confirming the standard client fails all three rejection scenarios. TODO: Update RFC_9207_ISS_PARAMETER spec reference once SEP-2468 (modelcontextprotocol/modelcontextprotocol#2468) is merged. * update scenarios * fix: createAuthServer iss option type/guard and NotAdvertised scenario duplication The doc comments said 'Default: not included' but the destructure defaulted to true/'correct', and the `!== undefined` guard at L155 was unreachable — so there was no way to omit the metadata field, and IssParameterNotAdvertised silently advertised support (a duplicate of IssParameterSupported). Kept the on-by-default behavior (mock AS models a well-behaved server) but made issParameterSupported `boolean | null` so callers pass null to omit, matching the codeChallengeMethodsSupported pattern. Doc comments now match. Scenarios that need omission pass null/'omit' explicitly. * fix: rejection scenarios silently pass when client never reaches auth endpoint correctlyRejected = !tokenRequestMade reports SUCCESS if the client errors out before hitting /authorize. Gate on authReached so a setup failure shows as FAILURE with authReached:false in details. * fix: iss-unexpected scenario contradicts SEP-2468 spec table row 3 The spec table says: supported=false/absent + iss present -> *Compare* to the recorded issuer (not reject). The scenario sent a *correct* iss and FAILed compliant clients for proceeding after a successful comparison. Now sends a mismatched iss so the comparison fails and rejection is the spec-required outcome. Reference client updated to compare-when-present instead of throw-on-presence. * refactor: replace harness-config checks with client-proceeded checks iss-advertised-in-metadata / iss-sent-in-redirect (and the not-* variants) fired in onAuthorizationRequest before the redirect happened, asserting only that the harness was configured correctly — a client that ignores iss passes identically. Replaced with one check per scenario keyed on tokenRequestMade, which observes that the client actually proceeded through the iss path. * refactor: rename check IDs to sep-2468-* and align with spec table rows One ID per spec table row; auth/iss-supported and auth/iss-wrong-issuer both emit sep-2468-client-compare-iss-supported (same comparison, opposite input) per the same-slug-for-SUCCESS-and-FAIL convention. * feat: add sep-2468.yaml requirement traceability 8 check rows (4 client table-row checks, 1 metadata-issuer, 2 AS-side, 1 no-normalization), 1 excluded (error-display is UI-facing). The record-issuer MUST is merged into the compare-iss-supported row text since it has no independent wire observation. * fix: migrate iss scenarios specVersions->source (post-#265) Replaces `specVersions: ['draft']` with `source: { introducedIn: DRAFT_PROTOCOL_VERSION }` in the 5 iss-parameter scenarios. This commit typechecks once the stack is rebased onto main >= #265 (the ScenarioSource migration). Adding it now so the rebase is mechanical. * fix: include application_type in iss-validation example DCR (post-#284) The SEP-837 application_type check now runs in every auth scenario; the hand-rolled DCR in auth-test-iss-validation.ts was omitting the field. --------- Co-authored-by: Paul Carleton <paulc@anthropic.com> --------- Co-authored-by: Anubhav Dhawan <anubhavdhawan@google.com> Co-authored-by: Paul Carleton <paulcarletonjr@gmail.com> Co-authored-by: Yuan Teoh <45984206+Yuan325@users.noreply.github.com> Co-authored-by: Paul Carleton <paulc@anthropic.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: Max Gerber <89937743+max-stytch@users.noreply.github.com>
Yuan325
commented
May 19, 2026
ContributorAuthor
@pcarleton Got it, i will submit another PR with the updated tests for those cases. Meanwhile, can you PTAL at this: #293? Thank you again! |
Merged
9 tasks
pcarleton pushed a commit
that referenced
this pull request
Jul 27, 2026
Every JSON-RPC message the harness sends or receives is now validated (ajv) against the vendored schema.json for the run's spec version, at the wire choke points: sendStatelessRequest (outbound request, JSON response, SSE events), connectStateful (request/result/notifications), and both mock servers (inbound client requests, outbound responses/handler results). Beyond the JSONRPCMessage envelope, messages are validated against the most specific definition the schema has: typed requests/notifications by method const, typed error responses by error.code const (e.g. -32021 -> MissingRequiredClientCapabilityError, the #376 hallucination), and typed results via the XxxRequest -> XxxResult pair, with SEP-2322 InputRequiredResult discriminated on resultType. Violations are recorded per scenario and surfaced by the runners as a synthetic wire-schema-valid check (FAILURE when the implementation under test sent an invalid message) plus a wire-schema-harness-error check when the invalid message was harness-authored. A vitest-wide afterEach guard fails any self-test that recorded violations, so hallucinated fixtures can't ship (#376, #271). Opt-outs are explicit and greppable: per-call skipValidation for deliberately malformed traffic, takeWireViolations() drains in tests that exercise deliberately broken fixtures. Offenders the new validation caught, now fixed: - everything-server stateless path omitted resultType (and the caching hints on server/discover) required by the draft schema - sep-2164-empty-contents fixture omitted resultType/ttlMs/cacheScope (its intended violation, empty contents, is schema-valid prose-only) - connection/mock-server unit-test fixtures used schema-invalid stubs (tools/call without name, ListToolsResult without required members, Tool without inputSchema, ProgressNotification without progressToken, ElicitRequest without message/requestedSchema) - http-standard-headers test mocks returned bare { tools: [] } results - input-required-result-validate-input deliberately sends malformed inputResponses; now marked skipValidation Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JHNrpU3Vr6Q1rPZiji7zLY
pcarleton added a commit
that referenced
this pull request
Jul 27, 2026
…399) * Vendor schema/{version}/schema.json alongside schema.ts sync-schema now copies the spec repo's JSON Schema for each version into src/spec-types/{version}.schema.json (same pinned SHA recorded in SOURCE: f817239). These are the inputs for runtime wire-message validation. Vendored verbatim, so they join schema.ts in .prettierignore. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JHNrpU3Vr6Q1rPZiji7zLY * Validate all wire messages against the per-version spec JSON schema Every JSON-RPC message the harness sends or receives is now validated (ajv) against the vendored schema.json for the run's spec version, at the wire choke points: sendStatelessRequest (outbound request, JSON response, SSE events), connectStateful (request/result/notifications), and both mock servers (inbound client requests, outbound responses/handler results). Beyond the JSONRPCMessage envelope, messages are validated against the most specific definition the schema has: typed requests/notifications by method const, typed error responses by error.code const (e.g. -32021 -> MissingRequiredClientCapabilityError, the #376 hallucination), and typed results via the XxxRequest -> XxxResult pair, with SEP-2322 InputRequiredResult discriminated on resultType. Violations are recorded per scenario and surfaced by the runners as a synthetic wire-schema-valid check (FAILURE when the implementation under test sent an invalid message) plus a wire-schema-harness-error check when the invalid message was harness-authored. A vitest-wide afterEach guard fails any self-test that recorded violations, so hallucinated fixtures can't ship (#376, #271). Opt-outs are explicit and greppable: per-call skipValidation for deliberately malformed traffic, takeWireViolations() drains in tests that exercise deliberately broken fixtures. Offenders the new validation caught, now fixed: - everything-server stateless path omitted resultType (and the caching hints on server/discover) required by the draft schema - sep-2164-empty-contents fixture omitted resultType/ttlMs/cacheScope (its intended violation, empty contents, is schema-valid prose-only) - connection/mock-server unit-test fixtures used schema-invalid stubs (tools/call without name, ListToolsResult without required members, Tool without inputSchema, ProgressNotification without progressToken, ElicitRequest without message/requestedSchema) - http-standard-headers test mocks returned bare { tools: [] } results - input-required-result-validate-input deliberately sends malformed inputResponses; now marked skipValidation Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JHNrpU3Vr6Q1rPZiji7zLY * Validate the stateful wire at the SDK transport, close review-panel gaps Address the review-panel findings on the wire-schema validation PR: - Stateful wire: hook the SDK transport inside connectToServer, wrapping send/onmessage so every raw wire message in both directions is validated for the scenario's spec version -- including the initialize handshake, server->client requests (elicitation/sampling), and the harness's responses to them. Request ids are correlated per direction so responses validate against their typed result definition (e.g. an outbound ElicitResult). The reconstructed stand-ins in connectStateful (fake id: 0 envelopes, Zod-reparsed results) are removed; the hook sees the real bytes. The four scenarios that call connectToServer directly (elicitation-defaults, elicitation-enums, tools, lifecycle) now pass ctx.specVersion and get validation for free; the runner surfaces their violations through the existing recorder. Known remaining gap (documented in the module doc): the client-auth scenarios' bespoke express mock is not instrumented. - skipValidation now skips only the request direction; the implementation's response to a deliberately malformed request is still validated, with a narrow carve-out tolerating the JSON-RPC 2.0 id: null an error response must carry when the request could not be processed (the MCP schema's RequestId forbids null). Covered by new unit tests pinning both the still-validated response and the narrowness of the carve-out. - negative-mrtr.test.ts: the file-wide drain now asserts every recorded violation is implementation-origin (the deliberately broken fixture) and throws on any harness-origin violation. - everything-server: hoist the cacheable-methods set to module scope and replace the per-request res.json monkey-patch with an explicit sendStatelessJson(res, method, payload) helper applied at each dispatch site. - Pin the extracted dispatch maps (methodDefs/errorDefs, e.g. -32021 -> MissingRequiredClientCapabilityError under the draft) per spec version in wire-schema.test.ts, so a schema sync that breaks the const extraction fails loudly instead of degrading to envelope-only validation. Also note the batch branch's requestMethod forwarding limitation. Offender the transport hook flushed out: the everything-server's test_elicitation_sep1034_defaults tool sent a number-typed elicitation field with default: 95.5, which the 2025-11-25 schema forbids (NumberSchema.default is typed integer there; widened to number only in the draft). The fixture and the scenario's expected default are now the integer-valued 95, with comments explaining why. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JHNrpU3Vr6Q1rPZiji7zLY * Condense comments to three lines max Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JHNrpU3Vr6Q1rPZiji7zLY * Re-vendor spec schemas at pinned SHA 71e30695 Regenerate src/spec-types/*.schema.json via npm run sync-schema at the SHA already recorded in SOURCE. The previously vendored draft schema predated spec PR #3002, so it still required DiscoverResult.serverInfo at the top level; the mock server now correctly reports serverInfo via result _meta, which the stale schema rejected. * Scope the wire-violation recorder per scenario for parallel suite runs Client --suite runs execute scenarios concurrently over what was a module-global violation recorder, so scenarios could steal or wipe each other's recorded violations. Give each suite scenario its own recorder via AsyncLocalStorage; code outside a scope (single-scenario runs, the sequential server suite, vitest hooks) falls back to the global recorder. Also condense a four-line comment in connection/stateful.ts to three lines. * Patch the 2025-11-25 NumberSchema generator bug as a load-time erratum The released 2025-11-25 schema.json types NumberSchema minimum/maximum/default as integer, contradicting its own schema.ts (default?: number) — a typescript-json-schema artifact, fixed for draft in modelcontextprotocol#2710 and proposed for 2025-11-25 in modelcontextprotocol#3139. Patch the schema at load time so SDK fixtures with fractional number defaults (all tier-1 SDKs send 95.5) are not flagged; restore the sep-1034 scenario and example-server fixture to 95.5. Delete the erratum when #3139 lands and the schemas are re-vendored. Also scope the sequential server-suite loop with withWireRecorder so a connection leaked by a failed scenario cannot attribute late traffic to the next scenario. --------- Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Paul Carleton <paulc@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add conformance tests for SEP-2575 to validate the stateless-MCP behavior. The checks was checked against #273
Motivation and Context
Ensure that servers adhere to the SEP.
How Has This Been Tested?
Tested locally
Breaking Changes
n/a. This tests are for the draft specs.
Types of changes
Checklist
server conformance tests for #266