Add identity model - #57

Merged
Ma11hewThomas merged 8 commits into
mainfrom
spec/add-identity-model
Jul 26, 2026
Merged

Add identity model#57
Ma11hewThomas merged 8 commits into
mainfrom
spec/add-identity-model

Conversation

@Ma11hewThomas

@Ma11hewThomasMa11hewThomas commented Jul 26, 2026

Copy link
Copy Markdown
Collaborator

Summary

Adds an optional identity model to CTRF so reports can distinguish between document identity, logical run identity, stable test-case identity, execution identity, retry-attempt identity, attachment identity, and shard identity.

The goal is to avoid overloading a single id field and give producers and consumers clearer semantics for cross-run analysis, sharding, retries, artifact correlation, and future derived-report work.

What Changed

  • Reworked the design principle currently focused on deterministic test identity into a broader Identity Model principle.
  • Added a normative Identity Model section covering:
    • reportId for the emitted CTRF document artifact
    • runId for the logical test run
    • testId for the stable logical test case
    • executionId for a concrete test execution within a run
    • attemptId for an individual retry attempt
    • attachmentId for a specific attachment reference instance
    • shardId for the shard or partition that produced the document
  • Clarified that reportId identifies the report artifact/content, not the logical test run.
  • Clarified that retransmitting the same unchanged report should preserve the same reportId, while materially changed, regenerated, merged, filtered, or transformed reports should receive a new reportId.
  • Added optional runId as a top-level field.
  • Clarified tests[].id as a legacy compatibility field and stated that new producers should prefer tests[].testId.
  • Added optional test-level testId and executionId.
  • Added optional retry-attempt-level attemptId.
  • Added optional attachment-level attachmentId in both test attachments and retry-attempt attachments.
  • Added optional environment-level shardId.
  • Added minLength: 1 constraints for identifier strings so empty IDs are rejected.
  • Updated producer and consumer conformance guidance for identity fields.
  • Updated Appendix B to describe deterministic testId generation with canonical strings as the primary producer-generated form, while keeping UUID v5 and hashes as optional deterministic encodings.
  • Updated the comprehensive example in both the spec and standalone examples.
  • Updated the JSON Schema and synced the embedded normative schema block in the spec.
  • Added normative type-validation tests for the new identity fields.
  • Added normative string-constraint tests for empty identifier strings.
  • Added a valid normative example showing repeated testId with different executionId values.
  • Added a changelog entry for the identity model.

Why

CTRF currently has reportId at the document level and id at the test level, but those identifiers do not cover all correlation needs clearly.

In practice, producers and consumers need to distinguish:

  • the emitted CTRF document artifact
  • a logical test run that may span shards or multiple CTRF documents
  • a stable logical test case within a producer's chosen scope
  • a specific execution of that test case
  • individual retry attempts
  • attachment reference instances
  • the shard or partition that produced a document

This PR introduces those layers as optional fields so existing documents remain valid, while producers that need stronger correlation semantics have a clear standard shape.

Compatibility

This is intended to be backward compatible:

  • All new identity fields are optional.
  • Existing tests[].id remains valid.
  • tests[].id is retained as a legacy compatibility field.
  • Consumers MUST NOT assume that identity fields are present.
  • Consumers SHOULD prefer testId when both id and testId are present.
  • Consumers MUST NOT assume that testId values generated by different producers are globally comparable unless the producers document a shared generation scheme.

Out Of Scope

This PR defines identity primitives only. It does not define provenance, derivation, parent/child report relationships, transformation history, or trust metadata. Those concepts can build on these identifiers in a future change.

Assessment Notes

This branch was rebased onto the latest main after the immutability and namespace guidance changes landed.

During preparation:

  • Conformance guidance conflicts were resolved by preserving both immutability guidance and identity guidance.
  • Examples were kept aligned with the new namespaced extra guidance.
  • The embedded JSON Schema block in spec/ctrf.md was synced exactly with schema/ctrf.schema.json.
  • Uniqueness wording was softened so the spec describes intended identity scopes without making unrealistic global-enforcement claims.
  • testId wording was tightened so producers keep it stable within their own documented scope, without implying independent producers will generate globally comparable values.

Follow-Up

This change will require downstream follow-up after the core spec/schema change lands.

Expected follow-up areas:

  • ctrf-js: types, helpers, schema exports, fixtures, and deprecated id handling.
  • ctrf-web: documentation, examples, and migration guidance for id to testId.
  • ctrf-cli: bundled schema, validation fixtures, and user-facing validation/output messaging.
  • Reporters and integrations: generated CTRF output, README examples, snapshots, runtime metadata APIs, and bundled schema copies.
  • Non-JS implementations: schemas, generated output, fixtures, snapshots, and docs.

Validation

  • jsonschema validate schema/ctrf.schema.json examples/*.json
  • jsonschema test tests/ctrf.test.json
  • jsonschema test tests/normative
  • jsonschema test tests/informative
  • Confirmed embedded CTRF JSON Schema block in spec/ctrf.md matches schema/ctrf.schema.json
  • Confirmed standalone and embedded examples with extra use namespaced keys where present

Ma11hewThomasand others added 6 commits July 26, 2026 17:24
- Add normative Section 4.9 Identity and Lineage Model with uniqueness
and stability tables for all seven identity layers
- Add runId (root): logical test run identifier
- Add testId (test): stable test-case identifier (string, not UUID-constrained)
- Add executionId (test): per-execution identifier within a run
- Add attemptId (retryAttempt): per-retry-attempt identifier
- Add attachmentId (attachment): per-attachment-reference identifier
- Add shardId (environment): shard/partition label
- Update reportId description to clarify as document artifact identifier
- Deprecate tests[].id in favour of testId
- Update Section 2.4 design principle to cover full identity model
- Update Section 19 conformance (producer and consumer)
- Update Appendix B references from id to testId
- Update Appendix D.5 comprehensive example with all identity fields
- Update schema with all new optional fields and deprecation note
@Ma11hewThomasMa11hewThomas changed the title Add identity and lineage modelAdd identity modelJul 26, 2026

CopilotAI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR expands CTRF’s identifier semantics by introducing an optional, layered Identity Model (document, run, test case, execution, retry attempt, attachment, shard) and updates the spec, JSON Schema, examples, and normative tests to support these new fields while preserving backward compatibility with tests[].id.

Changes:

  • Adds new optional identity fields (runId, testId, executionId, attemptId, attachmentId, shardId) and documents their intended scopes and stability in the spec.
  • Updates schema/ctrf.schema.json (and the embedded normative schema in spec/ctrf.md) with the new fields plus minLength: 1 constraints for identifier strings.
  • Extends normative test suites and examples to validate identity field types/constraints and demonstrate repeated testId with differing executionId.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.

Show a summary per file
FileDescription
tests/normative/valid-documents.test.jsonAdds a valid normative case demonstrating repeated testId with different executionId values.
tests/normative/type-validation.test.jsonAdds negative type-validation cases for the new identity fields.
tests/normative/string-constraints.test.jsonAdds empty-string constraint cases for new identifier fields (and needs one more for baseline).
spec/ctrf.mdIntroduces and normatively defines the Identity Model; updates field docs and embedded schema.
schema/ctrf.schema.jsonImplements Identity Model fields and non-empty string constraints in the normative JSON Schema.
examples/comprehensive.jsonUpdates the comprehensive example to include the new identity fields.
CHANGELOG.mdRecords the Identity Model addition in the changelog.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment threadtests/normative/string-constraints.test.json
@Ma11hewThomas
Ma11hewThomas marked this pull request as ready for review July 26, 2026 17:55
@Ma11hewThomas
Ma11hewThomas merged commit 84ddec5 into mainJul 26, 2026
2 checks passed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@Ma11hewThomas
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

Add identity model - #57

Merged
Ma11hewThomas merged 8 commits into
mainfrom
spec/add-identity-model
Jul 26, 2026
Merged

Add identity model#57
Ma11hewThomas merged 8 commits into
mainfrom
spec/add-identity-model

Conversation

@Ma11hewThomas

@Ma11hewThomasMa11hewThomas commented Jul 26, 2026

Copy link
Copy Markdown
Collaborator

Summary

Adds an optional identity model to CTRF so reports can distinguish between document identity, logical run identity, stable test-case identity, execution identity, retry-attempt identity, attachment identity, and shard identity.

The goal is to avoid overloading a single id field and give producers and consumers clearer semantics for cross-run analysis, sharding, retries, artifact correlation, and future derived-report work.

What Changed

  • Reworked the design principle currently focused on deterministic test identity into a broader Identity Model principle.
  • Added a normative Identity Model section covering:
    • reportId for the emitted CTRF document artifact
    • runId for the logical test run
    • testId for the stable logical test case
    • executionId for a concrete test execution within a run
    • attemptId for an individual retry attempt
    • attachmentId for a specific attachment reference instance
    • shardId for the shard or partition that produced the document
  • Clarified that reportId identifies the report artifact/content, not the logical test run.
  • Clarified that retransmitting the same unchanged report should preserve the same reportId, while materially changed, regenerated, merged, filtered, or transformed reports should receive a new reportId.
  • Added optional runId as a top-level field.
  • Clarified tests[].id as a legacy compatibility field and stated that new producers should prefer tests[].testId.
  • Added optional test-level testId and executionId.
  • Added optional retry-attempt-level attemptId.
  • Added optional attachment-level attachmentId in both test attachments and retry-attempt attachments.
  • Added optional environment-level shardId.
  • Added minLength: 1 constraints for identifier strings so empty IDs are rejected.
  • Updated producer and consumer conformance guidance for identity fields.
  • Updated Appendix B to describe deterministic testId generation with canonical strings as the primary producer-generated form, while keeping UUID v5 and hashes as optional deterministic encodings.
  • Updated the comprehensive example in both the spec and standalone examples.
  • Updated the JSON Schema and synced the embedded normative schema block in the spec.
  • Added normative type-validation tests for the new identity fields.
  • Added normative string-constraint tests for empty identifier strings.
  • Added a valid normative example showing repeated testId with different executionId values.
  • Added a changelog entry for the identity model.

Why

CTRF currently has reportId at the document level and id at the test level, but those identifiers do not cover all correlation needs clearly.

In practice, producers and consumers need to distinguish:

  • the emitted CTRF document artifact
  • a logical test run that may span shards or multiple CTRF documents
  • a stable logical test case within a producer's chosen scope
  • a specific execution of that test case
  • individual retry attempts
  • attachment reference instances
  • the shard or partition that produced a document

This PR introduces those layers as optional fields so existing documents remain valid, while producers that need stronger correlation semantics have a clear standard shape.

Compatibility

This is intended to be backward compatible:

  • All new identity fields are optional.
  • Existing tests[].id remains valid.
  • tests[].id is retained as a legacy compatibility field.
  • Consumers MUST NOT assume that identity fields are present.
  • Consumers SHOULD prefer testId when both id and testId are present.
  • Consumers MUST NOT assume that testId values generated by different producers are globally comparable unless the producers document a shared generation scheme.

Out Of Scope

This PR defines identity primitives only. It does not define provenance, derivation, parent/child report relationships, transformation history, or trust metadata. Those concepts can build on these identifiers in a future change.

Assessment Notes

This branch was rebased onto the latest main after the immutability and namespace guidance changes landed.

During preparation:

  • Conformance guidance conflicts were resolved by preserving both immutability guidance and identity guidance.
  • Examples were kept aligned with the new namespaced extra guidance.
  • The embedded JSON Schema block in spec/ctrf.md was synced exactly with schema/ctrf.schema.json.
  • Uniqueness wording was softened so the spec describes intended identity scopes without making unrealistic global-enforcement claims.
  • testId wording was tightened so producers keep it stable within their own documented scope, without implying independent producers will generate globally comparable values.

Follow-Up

This change will require downstream follow-up after the core spec/schema change lands.

Expected follow-up areas:

  • ctrf-js: types, helpers, schema exports, fixtures, and deprecated id handling.
  • ctrf-web: documentation, examples, and migration guidance for id to testId.
  • ctrf-cli: bundled schema, validation fixtures, and user-facing validation/output messaging.
  • Reporters and integrations: generated CTRF output, README examples, snapshots, runtime metadata APIs, and bundled schema copies.
  • Non-JS implementations: schemas, generated output, fixtures, snapshots, and docs.

Validation

  • jsonschema validate schema/ctrf.schema.json examples/*.json
  • jsonschema test tests/ctrf.test.json
  • jsonschema test tests/normative
  • jsonschema test tests/informative
  • Confirmed embedded CTRF JSON Schema block in spec/ctrf.md matches schema/ctrf.schema.json
  • Confirmed standalone and embedded examples with extra use namespaced keys where present

Ma11hewThomasand others added 6 commits July 26, 2026 17:24
- Add normative Section 4.9 Identity and Lineage Model with uniqueness
and stability tables for all seven identity layers
- Add runId (root): logical test run identifier
- Add testId (test): stable test-case identifier (string, not UUID-constrained)
- Add executionId (test): per-execution identifier within a run
- Add attemptId (retryAttempt): per-retry-attempt identifier
- Add attachmentId (attachment): per-attachment-reference identifier
- Add shardId (environment): shard/partition label
- Update reportId description to clarify as document artifact identifier
- Deprecate tests[].id in favour of testId
- Update Section 2.4 design principle to cover full identity model
- Update Section 19 conformance (producer and consumer)
- Update Appendix B references from id to testId
- Update Appendix D.5 comprehensive example with all identity fields
- Update schema with all new optional fields and deprecation note
@Ma11hewThomasMa11hewThomas changed the title Add identity and lineage modelAdd identity modelJul 26, 2026

CopilotAI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR expands CTRF’s identifier semantics by introducing an optional, layered Identity Model (document, run, test case, execution, retry attempt, attachment, shard) and updates the spec, JSON Schema, examples, and normative tests to support these new fields while preserving backward compatibility with tests[].id.

Changes:

  • Adds new optional identity fields (runId, testId, executionId, attemptId, attachmentId, shardId) and documents their intended scopes and stability in the spec.
  • Updates schema/ctrf.schema.json (and the embedded normative schema in spec/ctrf.md) with the new fields plus minLength: 1 constraints for identifier strings.
  • Extends normative test suites and examples to validate identity field types/constraints and demonstrate repeated testId with differing executionId.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.

Show a summary per file
FileDescription
tests/normative/valid-documents.test.jsonAdds a valid normative case demonstrating repeated testId with different executionId values.
tests/normative/type-validation.test.jsonAdds negative type-validation cases for the new identity fields.
tests/normative/string-constraints.test.jsonAdds empty-string constraint cases for new identifier fields (and needs one more for baseline).
spec/ctrf.mdIntroduces and normatively defines the Identity Model; updates field docs and embedded schema.
schema/ctrf.schema.jsonImplements Identity Model fields and non-empty string constraints in the normative JSON Schema.
examples/comprehensive.jsonUpdates the comprehensive example to include the new identity fields.
CHANGELOG.mdRecords the Identity Model addition in the changelog.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment threadtests/normative/string-constraints.test.json
@Ma11hewThomas
Ma11hewThomas marked this pull request as ready for review July 26, 2026 17:55
@Ma11hewThomas
Ma11hewThomas merged commit 84ddec5 into mainJul 26, 2026
2 checks passed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@Ma11hewThomas
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Add identity model - #57

Merged
Ma11hewThomas merged 8 commits into
mainfrom
spec/add-identity-model
Jul 26, 2026
Merged

Add identity model#57
Ma11hewThomas merged 8 commits into
mainfrom
spec/add-identity-model

Conversation

@Ma11hewThomas

@Ma11hewThomasMa11hewThomas commented Jul 26, 2026

Copy link
Copy Markdown
Collaborator

Summary

Adds an optional identity model to CTRF so reports can distinguish between document identity, logical run identity, stable test-case identity, execution identity, retry-attempt identity, attachment identity, and shard identity.

The goal is to avoid overloading a single id field and give producers and consumers clearer semantics for cross-run analysis, sharding, retries, artifact correlation, and future derived-report work.

What Changed

  • Reworked the design principle currently focused on deterministic test identity into a broader Identity Model principle.
  • Added a normative Identity Model section covering:
    • reportId for the emitted CTRF document artifact
    • runId for the logical test run
    • testId for the stable logical test case
    • executionId for a concrete test execution within a run
    • attemptId for an individual retry attempt
    • attachmentId for a specific attachment reference instance
    • shardId for the shard or partition that produced the document
  • Clarified that reportId identifies the report artifact/content, not the logical test run.
  • Clarified that retransmitting the same unchanged report should preserve the same reportId, while materially changed, regenerated, merged, filtered, or transformed reports should receive a new reportId.
  • Added optional runId as a top-level field.
  • Clarified tests[].id as a legacy compatibility field and stated that new producers should prefer tests[].testId.
  • Added optional test-level testId and executionId.
  • Added optional retry-attempt-level attemptId.
  • Added optional attachment-level attachmentId in both test attachments and retry-attempt attachments.
  • Added optional environment-level shardId.
  • Added minLength: 1 constraints for identifier strings so empty IDs are rejected.
  • Updated producer and consumer conformance guidance for identity fields.
  • Updated Appendix B to describe deterministic testId generation with canonical strings as the primary producer-generated form, while keeping UUID v5 and hashes as optional deterministic encodings.
  • Updated the comprehensive example in both the spec and standalone examples.
  • Updated the JSON Schema and synced the embedded normative schema block in the spec.
  • Added normative type-validation tests for the new identity fields.
  • Added normative string-constraint tests for empty identifier strings.
  • Added a valid normative example showing repeated testId with different executionId values.
  • Added a changelog entry for the identity model.

Why

CTRF currently has reportId at the document level and id at the test level, but those identifiers do not cover all correlation needs clearly.

In practice, producers and consumers need to distinguish:

  • the emitted CTRF document artifact
  • a logical test run that may span shards or multiple CTRF documents
  • a stable logical test case within a producer's chosen scope
  • a specific execution of that test case
  • individual retry attempts
  • attachment reference instances
  • the shard or partition that produced a document

This PR introduces those layers as optional fields so existing documents remain valid, while producers that need stronger correlation semantics have a clear standard shape.

Compatibility

This is intended to be backward compatible:

  • All new identity fields are optional.
  • Existing tests[].id remains valid.
  • tests[].id is retained as a legacy compatibility field.
  • Consumers MUST NOT assume that identity fields are present.
  • Consumers SHOULD prefer testId when both id and testId are present.
  • Consumers MUST NOT assume that testId values generated by different producers are globally comparable unless the producers document a shared generation scheme.

Out Of Scope

This PR defines identity primitives only. It does not define provenance, derivation, parent/child report relationships, transformation history, or trust metadata. Those concepts can build on these identifiers in a future change.

Assessment Notes

This branch was rebased onto the latest main after the immutability and namespace guidance changes landed.

During preparation:

  • Conformance guidance conflicts were resolved by preserving both immutability guidance and identity guidance.
  • Examples were kept aligned with the new namespaced extra guidance.
  • The embedded JSON Schema block in spec/ctrf.md was synced exactly with schema/ctrf.schema.json.
  • Uniqueness wording was softened so the spec describes intended identity scopes without making unrealistic global-enforcement claims.
  • testId wording was tightened so producers keep it stable within their own documented scope, without implying independent producers will generate globally comparable values.

Follow-Up

This change will require downstream follow-up after the core spec/schema change lands.

Expected follow-up areas:

  • ctrf-js: types, helpers, schema exports, fixtures, and deprecated id handling.
  • ctrf-web: documentation, examples, and migration guidance for id to testId.
  • ctrf-cli: bundled schema, validation fixtures, and user-facing validation/output messaging.
  • Reporters and integrations: generated CTRF output, README examples, snapshots, runtime metadata APIs, and bundled schema copies.
  • Non-JS implementations: schemas, generated output, fixtures, snapshots, and docs.

Validation

  • jsonschema validate schema/ctrf.schema.json examples/*.json
  • jsonschema test tests/ctrf.test.json
  • jsonschema test tests/normative
  • jsonschema test tests/informative
  • Confirmed embedded CTRF JSON Schema block in spec/ctrf.md matches schema/ctrf.schema.json
  • Confirmed standalone and embedded examples with extra use namespaced keys where present

Ma11hewThomasand others added 6 commits July 26, 2026 17:24
- Add normative Section 4.9 Identity and Lineage Model with uniqueness
and stability tables for all seven identity layers
- Add runId (root): logical test run identifier
- Add testId (test): stable test-case identifier (string, not UUID-constrained)
- Add executionId (test): per-execution identifier within a run
- Add attemptId (retryAttempt): per-retry-attempt identifier
- Add attachmentId (attachment): per-attachment-reference identifier
- Add shardId (environment): shard/partition label
- Update reportId description to clarify as document artifact identifier
- Deprecate tests[].id in favour of testId
- Update Section 2.4 design principle to cover full identity model
- Update Section 19 conformance (producer and consumer)
- Update Appendix B references from id to testId
- Update Appendix D.5 comprehensive example with all identity fields
- Update schema with all new optional fields and deprecation note
@Ma11hewThomasMa11hewThomas changed the title Add identity and lineage modelAdd identity modelJul 26, 2026

CopilotAI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR expands CTRF’s identifier semantics by introducing an optional, layered Identity Model (document, run, test case, execution, retry attempt, attachment, shard) and updates the spec, JSON Schema, examples, and normative tests to support these new fields while preserving backward compatibility with tests[].id.

Changes:

  • Adds new optional identity fields (runId, testId, executionId, attemptId, attachmentId, shardId) and documents their intended scopes and stability in the spec.
  • Updates schema/ctrf.schema.json (and the embedded normative schema in spec/ctrf.md) with the new fields plus minLength: 1 constraints for identifier strings.
  • Extends normative test suites and examples to validate identity field types/constraints and demonstrate repeated testId with differing executionId.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.

Show a summary per file
FileDescription
tests/normative/valid-documents.test.jsonAdds a valid normative case demonstrating repeated testId with different executionId values.
tests/normative/type-validation.test.jsonAdds negative type-validation cases for the new identity fields.
tests/normative/string-constraints.test.jsonAdds empty-string constraint cases for new identifier fields (and needs one more for baseline).
spec/ctrf.mdIntroduces and normatively defines the Identity Model; updates field docs and embedded schema.
schema/ctrf.schema.jsonImplements Identity Model fields and non-empty string constraints in the normative JSON Schema.
examples/comprehensive.jsonUpdates the comprehensive example to include the new identity fields.
CHANGELOG.mdRecords the Identity Model addition in the changelog.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment threadtests/normative/string-constraints.test.json
@Ma11hewThomas
Ma11hewThomas marked this pull request as ready for review July 26, 2026 17:55
@Ma11hewThomas
Ma11hewThomas merged commit 84ddec5 into mainJul 26, 2026
2 checks passed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@Ma11hewThomas
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Add identity model - #57

Merged
Ma11hewThomas merged 8 commits into
mainfrom
spec/add-identity-model
Jul 26, 2026
Merged

Add identity model#57
Ma11hewThomas merged 8 commits into
mainfrom
spec/add-identity-model

Conversation

@Ma11hewThomas

@Ma11hewThomasMa11hewThomas commented Jul 26, 2026

Copy link
Copy Markdown
Collaborator

Summary

Adds an optional identity model to CTRF so reports can distinguish between document identity, logical run identity, stable test-case identity, execution identity, retry-attempt identity, attachment identity, and shard identity.

The goal is to avoid overloading a single id field and give producers and consumers clearer semantics for cross-run analysis, sharding, retries, artifact correlation, and future derived-report work.

What Changed

  • Reworked the design principle currently focused on deterministic test identity into a broader Identity Model principle.
  • Added a normative Identity Model section covering:
    • reportId for the emitted CTRF document artifact
    • runId for the logical test run
    • testId for the stable logical test case
    • executionId for a concrete test execution within a run
    • attemptId for an individual retry attempt
    • attachmentId for a specific attachment reference instance
    • shardId for the shard or partition that produced the document
  • Clarified that reportId identifies the report artifact/content, not the logical test run.
  • Clarified that retransmitting the same unchanged report should preserve the same reportId, while materially changed, regenerated, merged, filtered, or transformed reports should receive a new reportId.
  • Added optional runId as a top-level field.
  • Clarified tests[].id as a legacy compatibility field and stated that new producers should prefer tests[].testId.
  • Added optional test-level testId and executionId.
  • Added optional retry-attempt-level attemptId.
  • Added optional attachment-level attachmentId in both test attachments and retry-attempt attachments.
  • Added optional environment-level shardId.
  • Added minLength: 1 constraints for identifier strings so empty IDs are rejected.
  • Updated producer and consumer conformance guidance for identity fields.
  • Updated Appendix B to describe deterministic testId generation with canonical strings as the primary producer-generated form, while keeping UUID v5 and hashes as optional deterministic encodings.
  • Updated the comprehensive example in both the spec and standalone examples.
  • Updated the JSON Schema and synced the embedded normative schema block in the spec.
  • Added normative type-validation tests for the new identity fields.
  • Added normative string-constraint tests for empty identifier strings.
  • Added a valid normative example showing repeated testId with different executionId values.
  • Added a changelog entry for the identity model.

Why

CTRF currently has reportId at the document level and id at the test level, but those identifiers do not cover all correlation needs clearly.

In practice, producers and consumers need to distinguish:

  • the emitted CTRF document artifact
  • a logical test run that may span shards or multiple CTRF documents
  • a stable logical test case within a producer's chosen scope
  • a specific execution of that test case
  • individual retry attempts
  • attachment reference instances
  • the shard or partition that produced a document

This PR introduces those layers as optional fields so existing documents remain valid, while producers that need stronger correlation semantics have a clear standard shape.

Compatibility

This is intended to be backward compatible:

  • All new identity fields are optional.
  • Existing tests[].id remains valid.
  • tests[].id is retained as a legacy compatibility field.
  • Consumers MUST NOT assume that identity fields are present.
  • Consumers SHOULD prefer testId when both id and testId are present.
  • Consumers MUST NOT assume that testId values generated by different producers are globally comparable unless the producers document a shared generation scheme.

Out Of Scope

This PR defines identity primitives only. It does not define provenance, derivation, parent/child report relationships, transformation history, or trust metadata. Those concepts can build on these identifiers in a future change.

Assessment Notes

This branch was rebased onto the latest main after the immutability and namespace guidance changes landed.

During preparation:

  • Conformance guidance conflicts were resolved by preserving both immutability guidance and identity guidance.
  • Examples were kept aligned with the new namespaced extra guidance.
  • The embedded JSON Schema block in spec/ctrf.md was synced exactly with schema/ctrf.schema.json.
  • Uniqueness wording was softened so the spec describes intended identity scopes without making unrealistic global-enforcement claims.
  • testId wording was tightened so producers keep it stable within their own documented scope, without implying independent producers will generate globally comparable values.

Follow-Up

This change will require downstream follow-up after the core spec/schema change lands.

Expected follow-up areas:

  • ctrf-js: types, helpers, schema exports, fixtures, and deprecated id handling.
  • ctrf-web: documentation, examples, and migration guidance for id to testId.
  • ctrf-cli: bundled schema, validation fixtures, and user-facing validation/output messaging.
  • Reporters and integrations: generated CTRF output, README examples, snapshots, runtime metadata APIs, and bundled schema copies.
  • Non-JS implementations: schemas, generated output, fixtures, snapshots, and docs.

Validation

  • jsonschema validate schema/ctrf.schema.json examples/*.json
  • jsonschema test tests/ctrf.test.json
  • jsonschema test tests/normative
  • jsonschema test tests/informative
  • Confirmed embedded CTRF JSON Schema block in spec/ctrf.md matches schema/ctrf.schema.json
  • Confirmed standalone and embedded examples with extra use namespaced keys where present

Ma11hewThomasand others added 6 commits July 26, 2026 17:24
- Add normative Section 4.9 Identity and Lineage Model with uniqueness
and stability tables for all seven identity layers
- Add runId (root): logical test run identifier
- Add testId (test): stable test-case identifier (string, not UUID-constrained)
- Add executionId (test): per-execution identifier within a run
- Add attemptId (retryAttempt): per-retry-attempt identifier
- Add attachmentId (attachment): per-attachment-reference identifier
- Add shardId (environment): shard/partition label
- Update reportId description to clarify as document artifact identifier
- Deprecate tests[].id in favour of testId
- Update Section 2.4 design principle to cover full identity model
- Update Section 19 conformance (producer and consumer)
- Update Appendix B references from id to testId
- Update Appendix D.5 comprehensive example with all identity fields
- Update schema with all new optional fields and deprecation note
@Ma11hewThomasMa11hewThomas changed the title Add identity and lineage modelAdd identity modelJul 26, 2026

CopilotAI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR expands CTRF’s identifier semantics by introducing an optional, layered Identity Model (document, run, test case, execution, retry attempt, attachment, shard) and updates the spec, JSON Schema, examples, and normative tests to support these new fields while preserving backward compatibility with tests[].id.

Changes:

  • Adds new optional identity fields (runId, testId, executionId, attemptId, attachmentId, shardId) and documents their intended scopes and stability in the spec.
  • Updates schema/ctrf.schema.json (and the embedded normative schema in spec/ctrf.md) with the new fields plus minLength: 1 constraints for identifier strings.
  • Extends normative test suites and examples to validate identity field types/constraints and demonstrate repeated testId with differing executionId.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.

Show a summary per file
FileDescription
tests/normative/valid-documents.test.jsonAdds a valid normative case demonstrating repeated testId with different executionId values.
tests/normative/type-validation.test.jsonAdds negative type-validation cases for the new identity fields.
tests/normative/string-constraints.test.jsonAdds empty-string constraint cases for new identifier fields (and needs one more for baseline).
spec/ctrf.mdIntroduces and normatively defines the Identity Model; updates field docs and embedded schema.
schema/ctrf.schema.jsonImplements Identity Model fields and non-empty string constraints in the normative JSON Schema.
examples/comprehensive.jsonUpdates the comprehensive example to include the new identity fields.
CHANGELOG.mdRecords the Identity Model addition in the changelog.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment threadtests/normative/string-constraints.test.json
@Ma11hewThomas
Ma11hewThomas marked this pull request as ready for review July 26, 2026 17:55
@Ma11hewThomas
Ma11hewThomas merged commit 84ddec5 into mainJul 26, 2026
2 checks passed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@Ma11hewThomas
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

Add identity model - #57

Merged
Ma11hewThomas merged 8 commits into
mainfrom
spec/add-identity-model
Jul 26, 2026
Merged

Add identity model#57
Ma11hewThomas merged 8 commits into
mainfrom
spec/add-identity-model

Conversation

@Ma11hewThomas

@Ma11hewThomasMa11hewThomas commented Jul 26, 2026

Copy link
Copy Markdown
Collaborator

Summary

Adds an optional identity model to CTRF so reports can distinguish between document identity, logical run identity, stable test-case identity, execution identity, retry-attempt identity, attachment identity, and shard identity.

The goal is to avoid overloading a single id field and give producers and consumers clearer semantics for cross-run analysis, sharding, retries, artifact correlation, and future derived-report work.

What Changed

  • Reworked the design principle currently focused on deterministic test identity into a broader Identity Model principle.
  • Added a normative Identity Model section covering:
    • reportId for the emitted CTRF document artifact
    • runId for the logical test run
    • testId for the stable logical test case
    • executionId for a concrete test execution within a run
    • attemptId for an individual retry attempt
    • attachmentId for a specific attachment reference instance
    • shardId for the shard or partition that produced the document
  • Clarified that reportId identifies the report artifact/content, not the logical test run.
  • Clarified that retransmitting the same unchanged report should preserve the same reportId, while materially changed, regenerated, merged, filtered, or transformed reports should receive a new reportId.
  • Added optional runId as a top-level field.
  • Clarified tests[].id as a legacy compatibility field and stated that new producers should prefer tests[].testId.
  • Added optional test-level testId and executionId.
  • Added optional retry-attempt-level attemptId.
  • Added optional attachment-level attachmentId in both test attachments and retry-attempt attachments.
  • Added optional environment-level shardId.
  • Added minLength: 1 constraints for identifier strings so empty IDs are rejected.
  • Updated producer and consumer conformance guidance for identity fields.
  • Updated Appendix B to describe deterministic testId generation with canonical strings as the primary producer-generated form, while keeping UUID v5 and hashes as optional deterministic encodings.
  • Updated the comprehensive example in both the spec and standalone examples.
  • Updated the JSON Schema and synced the embedded normative schema block in the spec.
  • Added normative type-validation tests for the new identity fields.
  • Added normative string-constraint tests for empty identifier strings.
  • Added a valid normative example showing repeated testId with different executionId values.
  • Added a changelog entry for the identity model.

Why

CTRF currently has reportId at the document level and id at the test level, but those identifiers do not cover all correlation needs clearly.

In practice, producers and consumers need to distinguish:

  • the emitted CTRF document artifact
  • a logical test run that may span shards or multiple CTRF documents
  • a stable logical test case within a producer's chosen scope
  • a specific execution of that test case
  • individual retry attempts
  • attachment reference instances
  • the shard or partition that produced a document

This PR introduces those layers as optional fields so existing documents remain valid, while producers that need stronger correlation semantics have a clear standard shape.

Compatibility

This is intended to be backward compatible:

  • All new identity fields are optional.
  • Existing tests[].id remains valid.
  • tests[].id is retained as a legacy compatibility field.
  • Consumers MUST NOT assume that identity fields are present.
  • Consumers SHOULD prefer testId when both id and testId are present.
  • Consumers MUST NOT assume that testId values generated by different producers are globally comparable unless the producers document a shared generation scheme.

Out Of Scope

This PR defines identity primitives only. It does not define provenance, derivation, parent/child report relationships, transformation history, or trust metadata. Those concepts can build on these identifiers in a future change.

Assessment Notes

This branch was rebased onto the latest main after the immutability and namespace guidance changes landed.

During preparation:

  • Conformance guidance conflicts were resolved by preserving both immutability guidance and identity guidance.
  • Examples were kept aligned with the new namespaced extra guidance.
  • The embedded JSON Schema block in spec/ctrf.md was synced exactly with schema/ctrf.schema.json.
  • Uniqueness wording was softened so the spec describes intended identity scopes without making unrealistic global-enforcement claims.
  • testId wording was tightened so producers keep it stable within their own documented scope, without implying independent producers will generate globally comparable values.

Follow-Up

This change will require downstream follow-up after the core spec/schema change lands.

Expected follow-up areas:

  • ctrf-js: types, helpers, schema exports, fixtures, and deprecated id handling.
  • ctrf-web: documentation, examples, and migration guidance for id to testId.
  • ctrf-cli: bundled schema, validation fixtures, and user-facing validation/output messaging.
  • Reporters and integrations: generated CTRF output, README examples, snapshots, runtime metadata APIs, and bundled schema copies.
  • Non-JS implementations: schemas, generated output, fixtures, snapshots, and docs.

Validation

  • jsonschema validate schema/ctrf.schema.json examples/*.json
  • jsonschema test tests/ctrf.test.json
  • jsonschema test tests/normative
  • jsonschema test tests/informative
  • Confirmed embedded CTRF JSON Schema block in spec/ctrf.md matches schema/ctrf.schema.json
  • Confirmed standalone and embedded examples with extra use namespaced keys where present

Ma11hewThomasand others added 6 commits July 26, 2026 17:24
- Add normative Section 4.9 Identity and Lineage Model with uniqueness
and stability tables for all seven identity layers
- Add runId (root): logical test run identifier
- Add testId (test): stable test-case identifier (string, not UUID-constrained)
- Add executionId (test): per-execution identifier within a run
- Add attemptId (retryAttempt): per-retry-attempt identifier
- Add attachmentId (attachment): per-attachment-reference identifier
- Add shardId (environment): shard/partition label
- Update reportId description to clarify as document artifact identifier
- Deprecate tests[].id in favour of testId
- Update Section 2.4 design principle to cover full identity model
- Update Section 19 conformance (producer and consumer)
- Update Appendix B references from id to testId
- Update Appendix D.5 comprehensive example with all identity fields
- Update schema with all new optional fields and deprecation note
@Ma11hewThomasMa11hewThomas changed the title Add identity and lineage modelAdd identity modelJul 26, 2026

CopilotAI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR expands CTRF’s identifier semantics by introducing an optional, layered Identity Model (document, run, test case, execution, retry attempt, attachment, shard) and updates the spec, JSON Schema, examples, and normative tests to support these new fields while preserving backward compatibility with tests[].id.

Changes:

  • Adds new optional identity fields (runId, testId, executionId, attemptId, attachmentId, shardId) and documents their intended scopes and stability in the spec.
  • Updates schema/ctrf.schema.json (and the embedded normative schema in spec/ctrf.md) with the new fields plus minLength: 1 constraints for identifier strings.
  • Extends normative test suites and examples to validate identity field types/constraints and demonstrate repeated testId with differing executionId.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.

Show a summary per file
FileDescription
tests/normative/valid-documents.test.jsonAdds a valid normative case demonstrating repeated testId with different executionId values.
tests/normative/type-validation.test.jsonAdds negative type-validation cases for the new identity fields.
tests/normative/string-constraints.test.jsonAdds empty-string constraint cases for new identifier fields (and needs one more for baseline).
spec/ctrf.mdIntroduces and normatively defines the Identity Model; updates field docs and embedded schema.
schema/ctrf.schema.jsonImplements Identity Model fields and non-empty string constraints in the normative JSON Schema.
examples/comprehensive.jsonUpdates the comprehensive example to include the new identity fields.
CHANGELOG.mdRecords the Identity Model addition in the changelog.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment threadtests/normative/string-constraints.test.json
@Ma11hewThomas
Ma11hewThomas marked this pull request as ready for review July 26, 2026 17:55
@Ma11hewThomas
Ma11hewThomas merged commit 84ddec5 into mainJul 26, 2026
2 checks passed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@Ma11hewThomas
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Add identity model - #57

Merged
Ma11hewThomas merged 8 commits into
mainfrom
spec/add-identity-model
Jul 26, 2026
Merged

Add identity model#57
Ma11hewThomas merged 8 commits into
mainfrom
spec/add-identity-model

Conversation

@Ma11hewThomas

@Ma11hewThomasMa11hewThomas commented Jul 26, 2026

Copy link
Copy Markdown
Collaborator

Summary

Adds an optional identity model to CTRF so reports can distinguish between document identity, logical run identity, stable test-case identity, execution identity, retry-attempt identity, attachment identity, and shard identity.

The goal is to avoid overloading a single id field and give producers and consumers clearer semantics for cross-run analysis, sharding, retries, artifact correlation, and future derived-report work.

What Changed

  • Reworked the design principle currently focused on deterministic test identity into a broader Identity Model principle.
  • Added a normative Identity Model section covering:
    • reportId for the emitted CTRF document artifact
    • runId for the logical test run
    • testId for the stable logical test case
    • executionId for a concrete test execution within a run
    • attemptId for an individual retry attempt
    • attachmentId for a specific attachment reference instance
    • shardId for the shard or partition that produced the document
  • Clarified that reportId identifies the report artifact/content, not the logical test run.
  • Clarified that retransmitting the same unchanged report should preserve the same reportId, while materially changed, regenerated, merged, filtered, or transformed reports should receive a new reportId.
  • Added optional runId as a top-level field.
  • Clarified tests[].id as a legacy compatibility field and stated that new producers should prefer tests[].testId.
  • Added optional test-level testId and executionId.
  • Added optional retry-attempt-level attemptId.
  • Added optional attachment-level attachmentId in both test attachments and retry-attempt attachments.
  • Added optional environment-level shardId.
  • Added minLength: 1 constraints for identifier strings so empty IDs are rejected.
  • Updated producer and consumer conformance guidance for identity fields.
  • Updated Appendix B to describe deterministic testId generation with canonical strings as the primary producer-generated form, while keeping UUID v5 and hashes as optional deterministic encodings.
  • Updated the comprehensive example in both the spec and standalone examples.
  • Updated the JSON Schema and synced the embedded normative schema block in the spec.
  • Added normative type-validation tests for the new identity fields.
  • Added normative string-constraint tests for empty identifier strings.
  • Added a valid normative example showing repeated testId with different executionId values.
  • Added a changelog entry for the identity model.

Why

CTRF currently has reportId at the document level and id at the test level, but those identifiers do not cover all correlation needs clearly.

In practice, producers and consumers need to distinguish:

  • the emitted CTRF document artifact
  • a logical test run that may span shards or multiple CTRF documents
  • a stable logical test case within a producer's chosen scope
  • a specific execution of that test case
  • individual retry attempts
  • attachment reference instances
  • the shard or partition that produced a document

This PR introduces those layers as optional fields so existing documents remain valid, while producers that need stronger correlation semantics have a clear standard shape.

Compatibility

This is intended to be backward compatible:

  • All new identity fields are optional.
  • Existing tests[].id remains valid.
  • tests[].id is retained as a legacy compatibility field.
  • Consumers MUST NOT assume that identity fields are present.
  • Consumers SHOULD prefer testId when both id and testId are present.
  • Consumers MUST NOT assume that testId values generated by different producers are globally comparable unless the producers document a shared generation scheme.

Out Of Scope

This PR defines identity primitives only. It does not define provenance, derivation, parent/child report relationships, transformation history, or trust metadata. Those concepts can build on these identifiers in a future change.

Assessment Notes

This branch was rebased onto the latest main after the immutability and namespace guidance changes landed.

During preparation:

  • Conformance guidance conflicts were resolved by preserving both immutability guidance and identity guidance.
  • Examples were kept aligned with the new namespaced extra guidance.
  • The embedded JSON Schema block in spec/ctrf.md was synced exactly with schema/ctrf.schema.json.
  • Uniqueness wording was softened so the spec describes intended identity scopes without making unrealistic global-enforcement claims.
  • testId wording was tightened so producers keep it stable within their own documented scope, without implying independent producers will generate globally comparable values.

Follow-Up

This change will require downstream follow-up after the core spec/schema change lands.

Expected follow-up areas:

  • ctrf-js: types, helpers, schema exports, fixtures, and deprecated id handling.
  • ctrf-web: documentation, examples, and migration guidance for id to testId.
  • ctrf-cli: bundled schema, validation fixtures, and user-facing validation/output messaging.
  • Reporters and integrations: generated CTRF output, README examples, snapshots, runtime metadata APIs, and bundled schema copies.
  • Non-JS implementations: schemas, generated output, fixtures, snapshots, and docs.

Validation

  • jsonschema validate schema/ctrf.schema.json examples/*.json
  • jsonschema test tests/ctrf.test.json
  • jsonschema test tests/normative
  • jsonschema test tests/informative
  • Confirmed embedded CTRF JSON Schema block in spec/ctrf.md matches schema/ctrf.schema.json
  • Confirmed standalone and embedded examples with extra use namespaced keys where present

Ma11hewThomasand others added 6 commits July 26, 2026 17:24
- Add normative Section 4.9 Identity and Lineage Model with uniqueness
and stability tables for all seven identity layers
- Add runId (root): logical test run identifier
- Add testId (test): stable test-case identifier (string, not UUID-constrained)
- Add executionId (test): per-execution identifier within a run
- Add attemptId (retryAttempt): per-retry-attempt identifier
- Add attachmentId (attachment): per-attachment-reference identifier
- Add shardId (environment): shard/partition label
- Update reportId description to clarify as document artifact identifier
- Deprecate tests[].id in favour of testId
- Update Section 2.4 design principle to cover full identity model
- Update Section 19 conformance (producer and consumer)
- Update Appendix B references from id to testId
- Update Appendix D.5 comprehensive example with all identity fields
- Update schema with all new optional fields and deprecation note
@Ma11hewThomasMa11hewThomas changed the title Add identity and lineage modelAdd identity modelJul 26, 2026

CopilotAI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR expands CTRF’s identifier semantics by introducing an optional, layered Identity Model (document, run, test case, execution, retry attempt, attachment, shard) and updates the spec, JSON Schema, examples, and normative tests to support these new fields while preserving backward compatibility with tests[].id.

Changes:

  • Adds new optional identity fields (runId, testId, executionId, attemptId, attachmentId, shardId) and documents their intended scopes and stability in the spec.
  • Updates schema/ctrf.schema.json (and the embedded normative schema in spec/ctrf.md) with the new fields plus minLength: 1 constraints for identifier strings.
  • Extends normative test suites and examples to validate identity field types/constraints and demonstrate repeated testId with differing executionId.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.

Show a summary per file
FileDescription
tests/normative/valid-documents.test.jsonAdds a valid normative case demonstrating repeated testId with different executionId values.
tests/normative/type-validation.test.jsonAdds negative type-validation cases for the new identity fields.
tests/normative/string-constraints.test.jsonAdds empty-string constraint cases for new identifier fields (and needs one more for baseline).
spec/ctrf.mdIntroduces and normatively defines the Identity Model; updates field docs and embedded schema.
schema/ctrf.schema.jsonImplements Identity Model fields and non-empty string constraints in the normative JSON Schema.
examples/comprehensive.jsonUpdates the comprehensive example to include the new identity fields.
CHANGELOG.mdRecords the Identity Model addition in the changelog.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment threadtests/normative/string-constraints.test.json
@Ma11hewThomas
Ma11hewThomas marked this pull request as ready for review July 26, 2026 17:55
@Ma11hewThomas
Ma11hewThomas merged commit 84ddec5 into mainJul 26, 2026
2 checks passed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@Ma11hewThomas
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Add identity model - #57

Merged
Ma11hewThomas merged 8 commits into
mainfrom
spec/add-identity-model
Jul 26, 2026
Merged

Add identity model#57
Ma11hewThomas merged 8 commits into
mainfrom
spec/add-identity-model

Conversation

@Ma11hewThomas

@Ma11hewThomasMa11hewThomas commented Jul 26, 2026

Copy link
Copy Markdown
Collaborator

Summary

Adds an optional identity model to CTRF so reports can distinguish between document identity, logical run identity, stable test-case identity, execution identity, retry-attempt identity, attachment identity, and shard identity.

The goal is to avoid overloading a single id field and give producers and consumers clearer semantics for cross-run analysis, sharding, retries, artifact correlation, and future derived-report work.

What Changed

  • Reworked the design principle currently focused on deterministic test identity into a broader Identity Model principle.
  • Added a normative Identity Model section covering:
    • reportId for the emitted CTRF document artifact
    • runId for the logical test run
    • testId for the stable logical test case
    • executionId for a concrete test execution within a run
    • attemptId for an individual retry attempt
    • attachmentId for a specific attachment reference instance
    • shardId for the shard or partition that produced the document
  • Clarified that reportId identifies the report artifact/content, not the logical test run.
  • Clarified that retransmitting the same unchanged report should preserve the same reportId, while materially changed, regenerated, merged, filtered, or transformed reports should receive a new reportId.
  • Added optional runId as a top-level field.
  • Clarified tests[].id as a legacy compatibility field and stated that new producers should prefer tests[].testId.
  • Added optional test-level testId and executionId.
  • Added optional retry-attempt-level attemptId.
  • Added optional attachment-level attachmentId in both test attachments and retry-attempt attachments.
  • Added optional environment-level shardId.
  • Added minLength: 1 constraints for identifier strings so empty IDs are rejected.
  • Updated producer and consumer conformance guidance for identity fields.
  • Updated Appendix B to describe deterministic testId generation with canonical strings as the primary producer-generated form, while keeping UUID v5 and hashes as optional deterministic encodings.
  • Updated the comprehensive example in both the spec and standalone examples.
  • Updated the JSON Schema and synced the embedded normative schema block in the spec.
  • Added normative type-validation tests for the new identity fields.
  • Added normative string-constraint tests for empty identifier strings.
  • Added a valid normative example showing repeated testId with different executionId values.
  • Added a changelog entry for the identity model.

Why

CTRF currently has reportId at the document level and id at the test level, but those identifiers do not cover all correlation needs clearly.

In practice, producers and consumers need to distinguish:

  • the emitted CTRF document artifact
  • a logical test run that may span shards or multiple CTRF documents
  • a stable logical test case within a producer's chosen scope
  • a specific execution of that test case
  • individual retry attempts
  • attachment reference instances
  • the shard or partition that produced a document

This PR introduces those layers as optional fields so existing documents remain valid, while producers that need stronger correlation semantics have a clear standard shape.

Compatibility

This is intended to be backward compatible:

  • All new identity fields are optional.
  • Existing tests[].id remains valid.
  • tests[].id is retained as a legacy compatibility field.
  • Consumers MUST NOT assume that identity fields are present.
  • Consumers SHOULD prefer testId when both id and testId are present.
  • Consumers MUST NOT assume that testId values generated by different producers are globally comparable unless the producers document a shared generation scheme.

Out Of Scope

This PR defines identity primitives only. It does not define provenance, derivation, parent/child report relationships, transformation history, or trust metadata. Those concepts can build on these identifiers in a future change.

Assessment Notes

This branch was rebased onto the latest main after the immutability and namespace guidance changes landed.

During preparation:

  • Conformance guidance conflicts were resolved by preserving both immutability guidance and identity guidance.
  • Examples were kept aligned with the new namespaced extra guidance.
  • The embedded JSON Schema block in spec/ctrf.md was synced exactly with schema/ctrf.schema.json.
  • Uniqueness wording was softened so the spec describes intended identity scopes without making unrealistic global-enforcement claims.
  • testId wording was tightened so producers keep it stable within their own documented scope, without implying independent producers will generate globally comparable values.

Follow-Up

This change will require downstream follow-up after the core spec/schema change lands.

Expected follow-up areas:

  • ctrf-js: types, helpers, schema exports, fixtures, and deprecated id handling.
  • ctrf-web: documentation, examples, and migration guidance for id to testId.
  • ctrf-cli: bundled schema, validation fixtures, and user-facing validation/output messaging.
  • Reporters and integrations: generated CTRF output, README examples, snapshots, runtime metadata APIs, and bundled schema copies.
  • Non-JS implementations: schemas, generated output, fixtures, snapshots, and docs.

Validation

  • jsonschema validate schema/ctrf.schema.json examples/*.json
  • jsonschema test tests/ctrf.test.json
  • jsonschema test tests/normative
  • jsonschema test tests/informative
  • Confirmed embedded CTRF JSON Schema block in spec/ctrf.md matches schema/ctrf.schema.json
  • Confirmed standalone and embedded examples with extra use namespaced keys where present

Ma11hewThomasand others added 6 commits July 26, 2026 17:24
- Add normative Section 4.9 Identity and Lineage Model with uniqueness
and stability tables for all seven identity layers
- Add runId (root): logical test run identifier
- Add testId (test): stable test-case identifier (string, not UUID-constrained)
- Add executionId (test): per-execution identifier within a run
- Add attemptId (retryAttempt): per-retry-attempt identifier
- Add attachmentId (attachment): per-attachment-reference identifier
- Add shardId (environment): shard/partition label
- Update reportId description to clarify as document artifact identifier
- Deprecate tests[].id in favour of testId
- Update Section 2.4 design principle to cover full identity model
- Update Section 19 conformance (producer and consumer)
- Update Appendix B references from id to testId
- Update Appendix D.5 comprehensive example with all identity fields
- Update schema with all new optional fields and deprecation note
@Ma11hewThomasMa11hewThomas changed the title Add identity and lineage modelAdd identity modelJul 26, 2026

CopilotAI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR expands CTRF’s identifier semantics by introducing an optional, layered Identity Model (document, run, test case, execution, retry attempt, attachment, shard) and updates the spec, JSON Schema, examples, and normative tests to support these new fields while preserving backward compatibility with tests[].id.

Changes:

  • Adds new optional identity fields (runId, testId, executionId, attemptId, attachmentId, shardId) and documents their intended scopes and stability in the spec.
  • Updates schema/ctrf.schema.json (and the embedded normative schema in spec/ctrf.md) with the new fields plus minLength: 1 constraints for identifier strings.
  • Extends normative test suites and examples to validate identity field types/constraints and demonstrate repeated testId with differing executionId.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.

Show a summary per file
FileDescription
tests/normative/valid-documents.test.jsonAdds a valid normative case demonstrating repeated testId with different executionId values.
tests/normative/type-validation.test.jsonAdds negative type-validation cases for the new identity fields.
tests/normative/string-constraints.test.jsonAdds empty-string constraint cases for new identifier fields (and needs one more for baseline).
spec/ctrf.mdIntroduces and normatively defines the Identity Model; updates field docs and embedded schema.
schema/ctrf.schema.jsonImplements Identity Model fields and non-empty string constraints in the normative JSON Schema.
examples/comprehensive.jsonUpdates the comprehensive example to include the new identity fields.
CHANGELOG.mdRecords the Identity Model addition in the changelog.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment threadtests/normative/string-constraints.test.json
@Ma11hewThomas
Ma11hewThomas marked this pull request as ready for review July 26, 2026 17:55
@Ma11hewThomas
Ma11hewThomas merged commit 84ddec5 into mainJul 26, 2026
2 checks passed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@Ma11hewThomas
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

Add identity model - #57

Merged
Ma11hewThomas merged 8 commits into
mainfrom
spec/add-identity-model
Jul 26, 2026
Merged

Add identity model#57
Ma11hewThomas merged 8 commits into
mainfrom
spec/add-identity-model

Conversation

@Ma11hewThomas

@Ma11hewThomasMa11hewThomas commented Jul 26, 2026

Copy link
Copy Markdown
Collaborator

Summary

Adds an optional identity model to CTRF so reports can distinguish between document identity, logical run identity, stable test-case identity, execution identity, retry-attempt identity, attachment identity, and shard identity.

The goal is to avoid overloading a single id field and give producers and consumers clearer semantics for cross-run analysis, sharding, retries, artifact correlation, and future derived-report work.

What Changed

  • Reworked the design principle currently focused on deterministic test identity into a broader Identity Model principle.
  • Added a normative Identity Model section covering:
    • reportId for the emitted CTRF document artifact
    • runId for the logical test run
    • testId for the stable logical test case
    • executionId for a concrete test execution within a run
    • attemptId for an individual retry attempt
    • attachmentId for a specific attachment reference instance
    • shardId for the shard or partition that produced the document
  • Clarified that reportId identifies the report artifact/content, not the logical test run.
  • Clarified that retransmitting the same unchanged report should preserve the same reportId, while materially changed, regenerated, merged, filtered, or transformed reports should receive a new reportId.
  • Added optional runId as a top-level field.
  • Clarified tests[].id as a legacy compatibility field and stated that new producers should prefer tests[].testId.
  • Added optional test-level testId and executionId.
  • Added optional retry-attempt-level attemptId.
  • Added optional attachment-level attachmentId in both test attachments and retry-attempt attachments.
  • Added optional environment-level shardId.
  • Added minLength: 1 constraints for identifier strings so empty IDs are rejected.
  • Updated producer and consumer conformance guidance for identity fields.
  • Updated Appendix B to describe deterministic testId generation with canonical strings as the primary producer-generated form, while keeping UUID v5 and hashes as optional deterministic encodings.
  • Updated the comprehensive example in both the spec and standalone examples.
  • Updated the JSON Schema and synced the embedded normative schema block in the spec.
  • Added normative type-validation tests for the new identity fields.
  • Added normative string-constraint tests for empty identifier strings.
  • Added a valid normative example showing repeated testId with different executionId values.
  • Added a changelog entry for the identity model.

Why

CTRF currently has reportId at the document level and id at the test level, but those identifiers do not cover all correlation needs clearly.

In practice, producers and consumers need to distinguish:

  • the emitted CTRF document artifact
  • a logical test run that may span shards or multiple CTRF documents
  • a stable logical test case within a producer's chosen scope
  • a specific execution of that test case
  • individual retry attempts
  • attachment reference instances
  • the shard or partition that produced a document

This PR introduces those layers as optional fields so existing documents remain valid, while producers that need stronger correlation semantics have a clear standard shape.

Compatibility

This is intended to be backward compatible:

  • All new identity fields are optional.
  • Existing tests[].id remains valid.
  • tests[].id is retained as a legacy compatibility field.
  • Consumers MUST NOT assume that identity fields are present.
  • Consumers SHOULD prefer testId when both id and testId are present.
  • Consumers MUST NOT assume that testId values generated by different producers are globally comparable unless the producers document a shared generation scheme.

Out Of Scope

This PR defines identity primitives only. It does not define provenance, derivation, parent/child report relationships, transformation history, or trust metadata. Those concepts can build on these identifiers in a future change.

Assessment Notes

This branch was rebased onto the latest main after the immutability and namespace guidance changes landed.

During preparation:

  • Conformance guidance conflicts were resolved by preserving both immutability guidance and identity guidance.
  • Examples were kept aligned with the new namespaced extra guidance.
  • The embedded JSON Schema block in spec/ctrf.md was synced exactly with schema/ctrf.schema.json.
  • Uniqueness wording was softened so the spec describes intended identity scopes without making unrealistic global-enforcement claims.
  • testId wording was tightened so producers keep it stable within their own documented scope, without implying independent producers will generate globally comparable values.

Follow-Up

This change will require downstream follow-up after the core spec/schema change lands.

Expected follow-up areas:

  • ctrf-js: types, helpers, schema exports, fixtures, and deprecated id handling.
  • ctrf-web: documentation, examples, and migration guidance for id to testId.
  • ctrf-cli: bundled schema, validation fixtures, and user-facing validation/output messaging.
  • Reporters and integrations: generated CTRF output, README examples, snapshots, runtime metadata APIs, and bundled schema copies.
  • Non-JS implementations: schemas, generated output, fixtures, snapshots, and docs.

Validation

  • jsonschema validate schema/ctrf.schema.json examples/*.json
  • jsonschema test tests/ctrf.test.json
  • jsonschema test tests/normative
  • jsonschema test tests/informative
  • Confirmed embedded CTRF JSON Schema block in spec/ctrf.md matches schema/ctrf.schema.json
  • Confirmed standalone and embedded examples with extra use namespaced keys where present

Ma11hewThomasand others added 6 commits July 26, 2026 17:24
- Add normative Section 4.9 Identity and Lineage Model with uniqueness
and stability tables for all seven identity layers
- Add runId (root): logical test run identifier
- Add testId (test): stable test-case identifier (string, not UUID-constrained)
- Add executionId (test): per-execution identifier within a run
- Add attemptId (retryAttempt): per-retry-attempt identifier
- Add attachmentId (attachment): per-attachment-reference identifier
- Add shardId (environment): shard/partition label
- Update reportId description to clarify as document artifact identifier
- Deprecate tests[].id in favour of testId
- Update Section 2.4 design principle to cover full identity model
- Update Section 19 conformance (producer and consumer)
- Update Appendix B references from id to testId
- Update Appendix D.5 comprehensive example with all identity fields
- Update schema with all new optional fields and deprecation note
@Ma11hewThomasMa11hewThomas changed the title Add identity and lineage modelAdd identity modelJul 26, 2026

CopilotAI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR expands CTRF’s identifier semantics by introducing an optional, layered Identity Model (document, run, test case, execution, retry attempt, attachment, shard) and updates the spec, JSON Schema, examples, and normative tests to support these new fields while preserving backward compatibility with tests[].id.

Changes:

  • Adds new optional identity fields (runId, testId, executionId, attemptId, attachmentId, shardId) and documents their intended scopes and stability in the spec.
  • Updates schema/ctrf.schema.json (and the embedded normative schema in spec/ctrf.md) with the new fields plus minLength: 1 constraints for identifier strings.
  • Extends normative test suites and examples to validate identity field types/constraints and demonstrate repeated testId with differing executionId.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.

Show a summary per file
FileDescription
tests/normative/valid-documents.test.jsonAdds a valid normative case demonstrating repeated testId with different executionId values.
tests/normative/type-validation.test.jsonAdds negative type-validation cases for the new identity fields.
tests/normative/string-constraints.test.jsonAdds empty-string constraint cases for new identifier fields (and needs one more for baseline).
spec/ctrf.mdIntroduces and normatively defines the Identity Model; updates field docs and embedded schema.
schema/ctrf.schema.jsonImplements Identity Model fields and non-empty string constraints in the normative JSON Schema.
examples/comprehensive.jsonUpdates the comprehensive example to include the new identity fields.
CHANGELOG.mdRecords the Identity Model addition in the changelog.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment threadtests/normative/string-constraints.test.json
@Ma11hewThomas
Ma11hewThomas marked this pull request as ready for review July 26, 2026 17:55
@Ma11hewThomas
Ma11hewThomas merged commit 84ddec5 into mainJul 26, 2026
2 checks passed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@Ma11hewThomas