Skip to content

3.2 improve schema test coverage - #4780

Closed
ralfhandl wants to merge 28 commits into
OAI:v3.2-devfrom
ralfhandl:3.2-schema-coverage
Closed

3.2 improve schema test coverage#4780
ralfhandl wants to merge 28 commits into
OAI:v3.2-devfrom
ralfhandl:3.2-schema-coverage

Conversation

@ralfhandl

@ralfhandlralfhandl commented Jul 14, 2025

Copy link
Copy Markdown
Contributor

We switched to https://github.com/hyperjump-io/json-schema-coverage for schema test coverage, and the new tool is more diligent than the preliminary tool used so far, reporting branches that are not covered by the existing schema tests.

This PR brings schema test coverage back up to 100%.

It includes the changes in

which can't be merged due to the insufficient schema test coverage.

It does not change the spec.

It does not change the schemas.

It only adds test cases.

  • schema changes are included in this pull request
  • schema changes are needed for this pull request but not done yet
  • no schema changes are needed for this pull request

@ralfhandl
ralfhandl marked this pull request as ready for review July 16, 2025 11:57
@ralfhandl
ralfhandl requested review from a team as code ownersJuly 16, 2025 11:57
@ralfhandlralfhandl added the Schema changes related to the schema(s) label Jul 16, 2025
@ralfhandlralfhandl mentioned this pull request Jul 16, 2025
@ralfhandlralfhandl changed the title 3.2 improve schema coverage3.2 improve schema test coverageJul 16, 2025
@ralfhandlralfhandl mentioned this pull request Jul 17, 2025

@handrewshandrews left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm concerned that the actual validation coverage is not as thorough as the files make it appear- I have commented on one such example. This all depends on how the JSON Schema implementation handles things, and whether we expect this suite to be usable with other implementations. If we want it to work with any conforming JSON Schema implementation, we need to only test one failure condition per case.

Comment threadtests/schema/fail/invalid-components-object.yaml Outdated
@ralfhandl
ralfhandl marked this pull request as draft July 19, 2025 20:37
and harmonized naming
@ralfhandl
ralfhandl marked this pull request as ready for review July 23, 2025 09:54
@ralfhandl

Copy link
Copy Markdown
ContributorAuthor

@handrews all test cases are now documented

@ralfhandl
ralfhandl requested review from a team and handrewsJuly 23, 2025 09:55
@karenetheridge

Copy link
Copy Markdown
Member

Given the number of commits in this PR, including multiple merge commits, could this be squashed at merge time? There should be an option for it right in the merge button. TIA!

@ralfhandl

Copy link
Copy Markdown
ContributorAuthor

Given the number of commits in this PR, including multiple merge commits, could this be squashed at merge time?

I've squashed my last commits after the merge-commit from v3.2-dev.

I'm a bit hesitant to squash more because that may cause problems with merging the overdue PR

which contains six commits that are also in this PR.

@ralfhandl

ralfhandl commented Aug 1, 2025

Copy link
Copy Markdown
ContributorAuthor

There should be an option for [squash merging] right in the merge button

After a discussion in the TDC on preserving the commit history of PRs we disabled squash merging and rebase merging for this repo.

@ralfhandl
ralfhandl deleted the 3.2-schema-coverage branch August 7, 2025 22:03
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Schemachanges related to the schema(s)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@ralfhandl@karenetheridge@handrews@jdesrosiers