[cdac] Physical contract descriptor spec - #100365

Merged
lambdageek merged 11 commits into
dotnet:mainfrom
lambdageek:cdac-contract-descriptor
Apr 4, 2024
Merged

[cdac] Physical contract descriptor spec#100365
lambdageek merged 11 commits into
dotnet:mainfrom
lambdageek:cdac-contract-descriptor

Conversation

@lambdageek

@lambdageeklambdageek commented Mar 27, 2024•
edited
Loading

Copy link
Copy Markdown
Member

Building on #100253 , describe an in-memory representation of the toplevel contract descriptor, comprisied of:

  • some target architecture properties
  • a data descriptor
  • a collection of compatible contracts

Contributes to #99298
Fixes#99299

@ghostghost added the needs-area-label An area label is needed to ensure this gets routed to the appropriate area owners label Mar 27, 2024
@lambdageeklambdageek added documentation Documentation bug or enhancement, does not impact product or test code area-Meta and removed needs-area-label An area label is needed to ensure this gets routed to the appropriate area owners labels Mar 27, 2024
@lambdageeklambdageek mentioned this pull request Mar 27, 2024
Merged
davidwrighton
davidwrighton reviewed Mar 27, 2024
View reviewed changes

@davidwrightondavidwrighton left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is a good direction, but I have a few comments.

Comment threaddocs/design/datacontracts/contract-descriptor.md Outdated
Comment threaddocs/design/datacontracts/contract-descriptor.md Outdated
Comment threaddocs/design/datacontracts/contract-descriptor.md Outdated
elinor-fung
elinor-fung reviewed Mar 27, 2024
View reviewed changes
Comment threaddocs/design/datacontracts/contract-descriptor.md Outdated
jkotas
jkotas reviewed Mar 28, 2024
View reviewed changes
Comment threaddocs/design/datacontracts/contract-descriptor.md Outdated
jkotas
jkotas reviewed Mar 28, 2024
View reviewed changes
Comment threaddocs/design/datacontracts/contract-descriptor.md Outdated
@lambdageek
lambdageekforce-pushed the cdac-contract-descriptor branch 3 times, most recently from f37c8a8 to 4bd9a58CompareMarch 28, 2024 17:22
jkotas
jkotas reviewed Mar 29, 2024
View reviewed changes
Comment threaddocs/design/datacontracts/contract-descriptor.md Outdated
Comment threaddocs/design/datacontracts/contract-descriptor.md Outdated
Comment threaddocs/design/datacontracts/contract-descriptor.md Outdated
@lambdageek
lambdageekforce-pushed the cdac-contract-descriptor branch from 4bd9a58 to 77afeacCompareMarch 29, 2024 13:30
@lambdageek
lambdageek marked this pull request as ready for review March 29, 2024 13:36
lambdageekand others added 4 commits March 29, 2024 09:37
@lambdageek
markdownlint
7640997
@lambdageek
don't use a pointer
a63782d
the issue is that we can't dereference pointers until we know the
target endianness and pointer size. which we can only discover by
reading the actual descriptor magic and flags
@lambdageek
flags not size_and_flags
2a23d12
@lambdageek
remove DotNetRuntimeContractDescriptorList - one runtime per module
e3d41c6
if there are multiple hosted runtimes, diagnostic tooling should look
in each loaded module to discover the contract descriptor
elinor-fung
elinor-fung approved these changes Apr 2, 2024
View reviewed changes
Comment threaddocs/design/datacontracts/datacontracts_design.md Outdated
Comment threaddocs/design/datacontracts/contract-descriptor.md Outdated
Comment threaddocs/design/datacontracts/contract-descriptor.md Outdated
Comment on lines +25 to +26
uint32_t aux_data_count;
uint32_t descriptor_size;

@elinor-fungelinor-fungApr 2, 2024

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we put the count/size right before or after the descriptor/aux_data instead?

Comment threaddocs/design/datacontracts/contract-descriptor.md Outdated
Comment threaddocs/design/datacontracts/contract-descriptor.md Outdated
Comment threaddocs/design/datacontracts/contract-descriptor.md Outdated
jkotas
jkotas reviewed Apr 2, 2024
View reviewed changes
Comment threaddocs/design/datacontracts/contract-descriptor.md Outdated

To aid in the discovery the contract descriptor should be exported by the module hosting the .NET
runtime with the name `DotNetRuntimeContractDescriptor`. (Using the C symbol conventions of the
target platform. That is, on platforms where such symbols typically have an `_` prepended, this

@jkotasjkotasApr 2, 2024

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What are those platforms?

FWIW, the existing DotNetRuntimeDebugHeader seems to be exported without _ everywhere:

runtime/src/coreclr/nativeaot/BuildIntegration/Microsoft.NETCore.Native.targets

Lines 245 to 246 in ab2bd48

<IlcArgCondition="'$(_targetOS)' == 'win' and '$(DebuggerSupport)' != 'false'"Include="--export-dynamic-symbol:DotNetRuntimeDebugHeader,DATA" />
<IlcArgCondition="'$(_targetOS)' != 'win' and '$(DebuggerSupport)' != 'false'"Include="--export-dynamic-symbol:DotNetRuntimeDebugHeader" />

jkotas
jkotas reviewed Apr 2, 2024
View reviewed changes
Comment threaddocs/design/datacontracts/contract-descriptor.md Outdated
If `ptrSize` is 0, the architecture is 64-bit. If it is 1, the architecture is 32-bit. The
reserved bits should be written as zero. Diagnostic tooling may ignore non-zero reserved bits.

The `descriptor` is a pointer to a JSON string described in [data descriptor physical layout](./data_descriptor.md#Physical_JSON_descriptor). The total length (including nul terminator character) is given by `descriptor_size`.

@jkotasjkotasApr 2, 2024

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: I would omit the terminating null in the descriptor_size.

The data contract reader cannot assume that the descriptor is null terminated. It needs to be robust against malicious input. Including the terminating null does not seem to be beneficial, it just adds extra logic to the reader to verify that there is a null at the end and strip it.

lambdageekand others added 2 commits April 4, 2024 10:29
@lambdageek@elinor-fung
Apply suggestions from code review
8c27593
Co-authored-by: Elinor Fung <elfung@microsoft.com>
@lambdageek
Review feedback
1553aa5
- put the aux data and descriptor sizes closer to the pointers
- Don't include trailing nul `descriptor_size`. Clarify it is
counting bytes and that `descriptor` is in UTF-8
- Simplify `DotNetRuntimeContractDescriptor` naming discussion
@lambdageek
lambdageek merged commit e629779 into dotnet:mainApr 4, 2024
@lambdageeklambdageek mentioned this pull request Apr 4, 2024
Closed
matouskozak pushed a commit to matouskozak/runtime that referenced this pull request Apr 30, 2024
@lambdageek@elinor-fung@matouskozak
[cdac] Physical contract descriptor spec (dotnet#100365)
091c101
Building on dotnet#100253 , describe an in-memory representation of the toplevel contract descriptor, comprisied of:
* some target architecture properties
* a data descriptor
* a collection of compatible contracts
Contributes to dotnet#99298Fixesdotnet#99299
---
* [cdac] Physical contract descriptor spec
* Add "contracts" to the data descriptor
* one runtime per module
if there are multiple hosted runtimes, diagnostic tooling should look in each loaded module to discover the contract descriptor
* Apply suggestions from code review
* Review feedback
- put the aux data and descriptor sizes closer to the pointers
- Don't include trailing nul `descriptor_size`. Clarify it is counting bytes and that `descriptor` is in UTF-8
- Simplify `DotNetRuntimeContractDescriptor` naming discussion
---------
Co-authored-by: Elinor Fung <elfung@microsoft.com>
@github-actionsgithub-actionsBot locked and limited conversation to collaborators May 5, 2024
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

@jkotasjkotasjkotas left review comments
@davidwrightondavidwrightondavidwrighton left review comments
@elinor-fungelinor-fungelinor-fung approved these changes
+1 more reviewer
@mikem8361mikem8361mikem8361 left review comments
Reviewers whose approvals may not affect merge requirements

Assignees

@lambdageeklambdageek

Labels

area-MetadocumentationDocumentation bug or enhancement, does not impact product or test code

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[cdac] Publish data stream spec

5 participants

@lambdageek@jkotas@mikem8361@davidwrighton@elinor-fung
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all 
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '

Skip to content
[cdac] Physical contract descriptor spec - #100365
Merged
lambdageek merged 11 commits into
dotnet:mainfrom 
lambdageek:cdac-contract-descriptor
Apr 4, 2024
Merged
[cdac] Physical contract descriptor spec#100365
lambdageek merged 11 commits into
dotnet:mainfrom 
lambdageek:cdac-contract-descriptor
Conversation
@lambdageek
@lambdageeklambdageek
commented
Mar 27, 2024•

edited

Loading

Copy link


Copy Markdown

Member
Building on #100253 , describe an in-memory representation of the toplevel contract descriptor, comprisied of:
  • some target architecture properties
  • a data descriptor
  • a collection of compatible contracts
Contributes to #99298
Fixes#99299
@ghostghost
added
the 
needs-area-label
An area label is needed to ensure this gets routed to the appropriate area owners
label
Mar 27, 2024
@lambdageeklambdageek
added

documentation
Documentation bug or enhancement, does not impact product or test code
area-Meta

and removed

needs-area-label
An area label is needed to ensure this gets routed to the appropriate area owners
labels
Mar 27, 2024
@lambdageeklambdageek
mentioned this pull request
Mar 27, 2024
 Merged

davidwrighton
davidwrighton
reviewed
Mar 27, 2024
View reviewed changes
@davidwrightondavidwrighton
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is a good direction, but I have a few comments.
Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

elinor-fung
elinor-fung
reviewed
Mar 27, 2024
View reviewed changes
Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

jkotas
jkotas
reviewed
Mar 28, 2024
View reviewed changes
Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

jkotas
jkotas
reviewed
Mar 28, 2024
View reviewed changes
Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

@lambdageek
lambdageekforce-pushed
the
cdac-contract-descriptor
branch
3 times, most recently
from
f37c8a8 to
4bd9a58CompareMarch 28, 2024 17:22
jkotas
jkotas
reviewed
Mar 29, 2024
View reviewed changes
Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

@lambdageek
lambdageekforce-pushed
the
cdac-contract-descriptor
branch
from
4bd9a58 to
77afeacCompareMarch 29, 2024 13:30
@lambdageek
lambdageek
marked this pull request as ready for review
March 29, 2024 13:36
lambdageekand others
added 4 commits
March 29, 2024 09:37
@lambdageek
markdownlint
7640997
@lambdageek
don't use a pointer
a63782d
the issue is that we can't dereference pointers until we know the
target endianness and pointer size. which we can only discover by
reading the actual descriptor magic and flags
@lambdageek
flags not size_and_flags
2a23d12
@lambdageek
remove DotNetRuntimeContractDescriptorList - one runtime per module
e3d41c6
if there are multiple hosted runtimes, diagnostic tooling should look
in each loaded module to discover the contract descriptor
elinor-fung
elinor-fung
approved these changes
Apr 2, 2024
View reviewed changes
Comment threaddocs/design/datacontracts/datacontracts_design.md
Outdated

Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated


Comment on lines

+25
 to 
+26

 uint32_t aux_data_count;
 uint32_t descriptor_size;
@elinor-fungelinor-fungApr 2, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we put the count/size right before or after the descriptor/aux_data instead?
Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

jkotas
jkotas
reviewed
Apr 2, 2024
View reviewed changes
Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated


To aid in the discovery the contract descriptor should be exported by the module hosting the .NET
runtime with the name `DotNetRuntimeContractDescriptor`. (Using the C symbol conventions of the
target platform. That is, on platforms where such symbols typically have an `_` prepended, this
@jkotasjkotasApr 2, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What are those platforms?
FWIW, the existing DotNetRuntimeDebugHeader seems to be exported without _ everywhere: 
runtime/src/coreclr/nativeaot/BuildIntegration/Microsoft.NETCore.Native.targets

Lines 245 to 246
in
ab2bd48
 <IlcArgCondition="'$(_targetOS)' == 'win' and '$(DebuggerSupport)' != 'false'"Include="--export-dynamic-symbol:DotNetRuntimeDebugHeader,DATA" /> 
 <IlcArgCondition="'$(_targetOS)' != 'win' and '$(DebuggerSupport)' != 'false'"Include="--export-dynamic-symbol:DotNetRuntimeDebugHeader" /> 
jkotas
jkotas
reviewed
Apr 2, 2024
View reviewed changes
Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

If `ptrSize` is 0, the architecture is 64-bit. If it is 1, the architecture is 32-bit. The
reserved bits should be written as zero. Diagnostic tooling may ignore non-zero reserved bits.

The `descriptor` is a pointer to a JSON string described in [data descriptor physical layout](./data_descriptor.md#Physical_JSON_descriptor). The total length (including nul terminator character) is given by `descriptor_size`.
@jkotasjkotasApr 2, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: I would omit the terminating null in the descriptor_size.
The data contract reader cannot assume that the descriptor is null terminated. It needs to be robust against malicious input. Including the terminating null does not seem to be beneficial, it just adds extra logic to the reader to verify that there is a null at the end and strip it.
lambdageekand others
added 2 commits
April 4, 2024 10:29
@lambdageek@elinor-fung
Apply suggestions from code review
8c27593
Co-authored-by: Elinor Fung <elfung@microsoft.com>
@lambdageek
Review feedback
1553aa5
- put the aux data and descriptor sizes closer to the pointers
- Don't include trailing nul `descriptor_size`. Clarify it is
counting bytes and that `descriptor` is in UTF-8
- Simplify `DotNetRuntimeContractDescriptor` naming discussion
@lambdageek
lambdageek
merged commit e629779
into
dotnet:mainApr 4, 2024
@lambdageeklambdageek
mentioned this pull request
Apr 4, 2024

Closed

matouskozak pushed a commit
to matouskozak/runtime
that referenced
this pull request
Apr 30, 2024
@lambdageek@elinor-fung@matouskozak
[cdac] Physical contract descriptor spec (dotnet#100365)
091c101
Building on dotnet#100253 , describe an in-memory representation of the toplevel contract descriptor, comprisied of:
* some target architecture properties
* a data descriptor
* a collection of compatible contracts
Contributes to dotnet#99298Fixesdotnet#99299
---
* [cdac] Physical contract descriptor spec
* Add "contracts" to the data descriptor
* one runtime per module
if there are multiple hosted runtimes, diagnostic tooling should look in each loaded module to discover the contract descriptor
* Apply suggestions from code review
* Review feedback
- put the aux data and descriptor sizes closer to the pointers
- Don't include trailing nul `descriptor_size`. Clarify it is counting bytes and that `descriptor` is in UTF-8
- Simplify `DotNetRuntimeContractDescriptor` naming discussion
---------
Co-authored-by: Elinor Fung <elfung@microsoft.com>
@github-actionsgithub-actionsBot
locked and limited conversation to collaborators
May 5, 2024
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@jkotasjkotasjkotas left review comments
@davidwrightondavidwrightondavidwrighton left review comments
@elinor-fungelinor-fungelinor-fung approved these changes

+1 more reviewer

@mikem8361mikem8361mikem8361 left review comments
Reviewers whose approvals may not affect merge requirements

Assignees

@lambdageeklambdageek

Labels

area-MetadocumentationDocumentation bug or enhancement, does not impact product or test code

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

[cdac] Publish data stream spec


5 participants

@lambdageek@jkotas@mikem8361@davidwrighton@elinor-fung

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix");
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
[cdac] Physical contract descriptor spec - #100365
Merged
lambdageek merged 11 commits into
dotnet:mainfrom 
lambdageek:cdac-contract-descriptor
Apr 4, 2024
Merged
[cdac] Physical contract descriptor spec#100365
lambdageek merged 11 commits into
dotnet:mainfrom 
lambdageek:cdac-contract-descriptor
Conversation
@lambdageek
@lambdageeklambdageek
commented
Mar 27, 2024•

edited

Loading

Copy link


Copy Markdown

Member
Building on #100253 , describe an in-memory representation of the toplevel contract descriptor, comprisied of:
  • some target architecture properties
  • a data descriptor
  • a collection of compatible contracts
Contributes to #99298
Fixes#99299
@ghostghost
added
the 
needs-area-label
An area label is needed to ensure this gets routed to the appropriate area owners
label
Mar 27, 2024
@lambdageeklambdageek
added

documentation
Documentation bug or enhancement, does not impact product or test code
area-Meta

and removed

needs-area-label
An area label is needed to ensure this gets routed to the appropriate area owners
labels
Mar 27, 2024
@lambdageeklambdageek
mentioned this pull request
Mar 27, 2024
 Merged

davidwrighton
davidwrighton
reviewed
Mar 27, 2024
View reviewed changes
@davidwrightondavidwrighton
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is a good direction, but I have a few comments.
Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

elinor-fung
elinor-fung
reviewed
Mar 27, 2024
View reviewed changes
Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

jkotas
jkotas
reviewed
Mar 28, 2024
View reviewed changes
Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

jkotas
jkotas
reviewed
Mar 28, 2024
View reviewed changes
Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

@lambdageek
lambdageekforce-pushed
the
cdac-contract-descriptor
branch
3 times, most recently
from
f37c8a8 to
4bd9a58CompareMarch 28, 2024 17:22
jkotas
jkotas
reviewed
Mar 29, 2024
View reviewed changes
Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

@lambdageek
lambdageekforce-pushed
the
cdac-contract-descriptor
branch
from
4bd9a58 to
77afeacCompareMarch 29, 2024 13:30
@lambdageek
lambdageek
marked this pull request as ready for review
March 29, 2024 13:36
lambdageekand others
added 4 commits
March 29, 2024 09:37
@lambdageek
markdownlint
7640997
@lambdageek
don't use a pointer
a63782d
the issue is that we can't dereference pointers until we know the
target endianness and pointer size. which we can only discover by
reading the actual descriptor magic and flags
@lambdageek
flags not size_and_flags
2a23d12
@lambdageek
remove DotNetRuntimeContractDescriptorList - one runtime per module
e3d41c6
if there are multiple hosted runtimes, diagnostic tooling should look
in each loaded module to discover the contract descriptor
elinor-fung
elinor-fung
approved these changes
Apr 2, 2024
View reviewed changes
Comment threaddocs/design/datacontracts/datacontracts_design.md
Outdated

Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated


Comment on lines

+25
 to 
+26

 uint32_t aux_data_count;
 uint32_t descriptor_size;
@elinor-fungelinor-fungApr 2, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we put the count/size right before or after the descriptor/aux_data instead?
Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

jkotas
jkotas
reviewed
Apr 2, 2024
View reviewed changes
Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated


To aid in the discovery the contract descriptor should be exported by the module hosting the .NET
runtime with the name `DotNetRuntimeContractDescriptor`. (Using the C symbol conventions of the
target platform. That is, on platforms where such symbols typically have an `_` prepended, this
@jkotasjkotasApr 2, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What are those platforms?
FWIW, the existing DotNetRuntimeDebugHeader seems to be exported without _ everywhere: 
runtime/src/coreclr/nativeaot/BuildIntegration/Microsoft.NETCore.Native.targets

Lines 245 to 246
in
ab2bd48
 <IlcArgCondition="'$(_targetOS)' == 'win' and '$(DebuggerSupport)' != 'false'"Include="--export-dynamic-symbol:DotNetRuntimeDebugHeader,DATA" /> 
 <IlcArgCondition="'$(_targetOS)' != 'win' and '$(DebuggerSupport)' != 'false'"Include="--export-dynamic-symbol:DotNetRuntimeDebugHeader" /> 
jkotas
jkotas
reviewed
Apr 2, 2024
View reviewed changes
Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

If `ptrSize` is 0, the architecture is 64-bit. If it is 1, the architecture is 32-bit. The
reserved bits should be written as zero. Diagnostic tooling may ignore non-zero reserved bits.

The `descriptor` is a pointer to a JSON string described in [data descriptor physical layout](./data_descriptor.md#Physical_JSON_descriptor). The total length (including nul terminator character) is given by `descriptor_size`.
@jkotasjkotasApr 2, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: I would omit the terminating null in the descriptor_size.
The data contract reader cannot assume that the descriptor is null terminated. It needs to be robust against malicious input. Including the terminating null does not seem to be beneficial, it just adds extra logic to the reader to verify that there is a null at the end and strip it.
lambdageekand others
added 2 commits
April 4, 2024 10:29
@lambdageek@elinor-fung
Apply suggestions from code review
8c27593
Co-authored-by: Elinor Fung <elfung@microsoft.com>
@lambdageek
Review feedback
1553aa5
- put the aux data and descriptor sizes closer to the pointers
- Don't include trailing nul `descriptor_size`. Clarify it is
counting bytes and that `descriptor` is in UTF-8
- Simplify `DotNetRuntimeContractDescriptor` naming discussion
@lambdageek
lambdageek
merged commit e629779
into
dotnet:mainApr 4, 2024
@lambdageeklambdageek
mentioned this pull request
Apr 4, 2024

Closed

matouskozak pushed a commit
to matouskozak/runtime
that referenced
this pull request
Apr 30, 2024
@lambdageek@elinor-fung@matouskozak
[cdac] Physical contract descriptor spec (dotnet#100365)
091c101
Building on dotnet#100253 , describe an in-memory representation of the toplevel contract descriptor, comprisied of:
* some target architecture properties
* a data descriptor
* a collection of compatible contracts
Contributes to dotnet#99298Fixesdotnet#99299
---
* [cdac] Physical contract descriptor spec
* Add "contracts" to the data descriptor
* one runtime per module
if there are multiple hosted runtimes, diagnostic tooling should look in each loaded module to discover the contract descriptor
* Apply suggestions from code review
* Review feedback
- put the aux data and descriptor sizes closer to the pointers
- Don't include trailing nul `descriptor_size`. Clarify it is counting bytes and that `descriptor` is in UTF-8
- Simplify `DotNetRuntimeContractDescriptor` naming discussion
---------
Co-authored-by: Elinor Fung <elfung@microsoft.com>
@github-actionsgithub-actionsBot
locked and limited conversation to collaborators
May 5, 2024
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@jkotasjkotasjkotas left review comments
@davidwrightondavidwrightondavidwrighton left review comments
@elinor-fungelinor-fungelinor-fung approved these changes

+1 more reviewer

@mikem8361mikem8361mikem8361 left review comments
Reviewers whose approvals may not affect merge requirements

Assignees

@lambdageeklambdageek

Labels

area-MetadocumentationDocumentation bug or enhancement, does not impact product or test code

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

[cdac] Publish data stream spec


5 participants

@lambdageek@jkotas@mikem8361@davidwrighton@elinor-fung

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms");
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
[cdac] Physical contract descriptor spec - #100365
Merged
lambdageek merged 11 commits into
dotnet:mainfrom 
lambdageek:cdac-contract-descriptor
Apr 4, 2024
Merged
[cdac] Physical contract descriptor spec#100365
lambdageek merged 11 commits into
dotnet:mainfrom 
lambdageek:cdac-contract-descriptor
Conversation
@lambdageek
@lambdageeklambdageek
commented
Mar 27, 2024•

edited

Loading

Copy link


Copy Markdown

Member
Building on #100253 , describe an in-memory representation of the toplevel contract descriptor, comprisied of:
  • some target architecture properties
  • a data descriptor
  • a collection of compatible contracts
Contributes to #99298
Fixes#99299
@ghostghost
added
the 
needs-area-label
An area label is needed to ensure this gets routed to the appropriate area owners
label
Mar 27, 2024
@lambdageeklambdageek
added

documentation
Documentation bug or enhancement, does not impact product or test code
area-Meta

and removed

needs-area-label
An area label is needed to ensure this gets routed to the appropriate area owners
labels
Mar 27, 2024
@lambdageeklambdageek
mentioned this pull request
Mar 27, 2024
 Merged

davidwrighton
davidwrighton
reviewed
Mar 27, 2024
View reviewed changes
@davidwrightondavidwrighton
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is a good direction, but I have a few comments.
Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

elinor-fung
elinor-fung
reviewed
Mar 27, 2024
View reviewed changes
Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

jkotas
jkotas
reviewed
Mar 28, 2024
View reviewed changes
Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

jkotas
jkotas
reviewed
Mar 28, 2024
View reviewed changes
Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

@lambdageek
lambdageekforce-pushed
the
cdac-contract-descriptor
branch
3 times, most recently
from
f37c8a8 to
4bd9a58CompareMarch 28, 2024 17:22
jkotas
jkotas
reviewed
Mar 29, 2024
View reviewed changes
Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

@lambdageek
lambdageekforce-pushed
the
cdac-contract-descriptor
branch
from
4bd9a58 to
77afeacCompareMarch 29, 2024 13:30
@lambdageek
lambdageek
marked this pull request as ready for review
March 29, 2024 13:36
lambdageekand others
added 4 commits
March 29, 2024 09:37
@lambdageek
markdownlint
7640997
@lambdageek
don't use a pointer
a63782d
the issue is that we can't dereference pointers until we know the
target endianness and pointer size. which we can only discover by
reading the actual descriptor magic and flags
@lambdageek
flags not size_and_flags
2a23d12
@lambdageek
remove DotNetRuntimeContractDescriptorList - one runtime per module
e3d41c6
if there are multiple hosted runtimes, diagnostic tooling should look
in each loaded module to discover the contract descriptor
elinor-fung
elinor-fung
approved these changes
Apr 2, 2024
View reviewed changes
Comment threaddocs/design/datacontracts/datacontracts_design.md
Outdated

Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated


Comment on lines

+25
 to 
+26

 uint32_t aux_data_count;
 uint32_t descriptor_size;
@elinor-fungelinor-fungApr 2, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we put the count/size right before or after the descriptor/aux_data instead?
Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

jkotas
jkotas
reviewed
Apr 2, 2024
View reviewed changes
Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated


To aid in the discovery the contract descriptor should be exported by the module hosting the .NET
runtime with the name `DotNetRuntimeContractDescriptor`. (Using the C symbol conventions of the
target platform. That is, on platforms where such symbols typically have an `_` prepended, this
@jkotasjkotasApr 2, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What are those platforms?
FWIW, the existing DotNetRuntimeDebugHeader seems to be exported without _ everywhere: 
runtime/src/coreclr/nativeaot/BuildIntegration/Microsoft.NETCore.Native.targets

Lines 245 to 246
in
ab2bd48
 <IlcArgCondition="'$(_targetOS)' == 'win' and '$(DebuggerSupport)' != 'false'"Include="--export-dynamic-symbol:DotNetRuntimeDebugHeader,DATA" /> 
 <IlcArgCondition="'$(_targetOS)' != 'win' and '$(DebuggerSupport)' != 'false'"Include="--export-dynamic-symbol:DotNetRuntimeDebugHeader" /> 
jkotas
jkotas
reviewed
Apr 2, 2024
View reviewed changes
Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

If `ptrSize` is 0, the architecture is 64-bit. If it is 1, the architecture is 32-bit. The
reserved bits should be written as zero. Diagnostic tooling may ignore non-zero reserved bits.

The `descriptor` is a pointer to a JSON string described in [data descriptor physical layout](./data_descriptor.md#Physical_JSON_descriptor). The total length (including nul terminator character) is given by `descriptor_size`.
@jkotasjkotasApr 2, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: I would omit the terminating null in the descriptor_size.
The data contract reader cannot assume that the descriptor is null terminated. It needs to be robust against malicious input. Including the terminating null does not seem to be beneficial, it just adds extra logic to the reader to verify that there is a null at the end and strip it.
lambdageekand others
added 2 commits
April 4, 2024 10:29
@lambdageek@elinor-fung
Apply suggestions from code review
8c27593
Co-authored-by: Elinor Fung <elfung@microsoft.com>
@lambdageek
Review feedback
1553aa5
- put the aux data and descriptor sizes closer to the pointers
- Don't include trailing nul `descriptor_size`. Clarify it is
counting bytes and that `descriptor` is in UTF-8
- Simplify `DotNetRuntimeContractDescriptor` naming discussion
@lambdageek
lambdageek
merged commit e629779
into
dotnet:mainApr 4, 2024
@lambdageeklambdageek
mentioned this pull request
Apr 4, 2024

Closed

matouskozak pushed a commit
to matouskozak/runtime
that referenced
this pull request
Apr 30, 2024
@lambdageek@elinor-fung@matouskozak
[cdac] Physical contract descriptor spec (dotnet#100365)
091c101
Building on dotnet#100253 , describe an in-memory representation of the toplevel contract descriptor, comprisied of:
* some target architecture properties
* a data descriptor
* a collection of compatible contracts
Contributes to dotnet#99298Fixesdotnet#99299
---
* [cdac] Physical contract descriptor spec
* Add "contracts" to the data descriptor
* one runtime per module
if there are multiple hosted runtimes, diagnostic tooling should look in each loaded module to discover the contract descriptor
* Apply suggestions from code review
* Review feedback
- put the aux data and descriptor sizes closer to the pointers
- Don't include trailing nul `descriptor_size`. Clarify it is counting bytes and that `descriptor` is in UTF-8
- Simplify `DotNetRuntimeContractDescriptor` naming discussion
---------
Co-authored-by: Elinor Fung <elfung@microsoft.com>
@github-actionsgithub-actionsBot
locked and limited conversation to collaborators
May 5, 2024
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@jkotasjkotasjkotas left review comments
@davidwrightondavidwrightondavidwrighton left review comments
@elinor-fungelinor-fungelinor-fung approved these changes

+1 more reviewer

@mikem8361mikem8361mikem8361 left review comments
Reviewers whose approvals may not affect merge requirements

Assignees

@lambdageeklambdageek

Labels

area-MetadocumentationDocumentation bug or enhancement, does not impact product or test code

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

[cdac] Publish data stream spec


5 participants

@lambdageek@jkotas@mikem8361@davidwrighton@elinor-fung

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links");
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '

Skip to content
[cdac] Physical contract descriptor spec - #100365
Merged
lambdageek merged 11 commits into
dotnet:mainfrom 
lambdageek:cdac-contract-descriptor
Apr 4, 2024
Merged
[cdac] Physical contract descriptor spec#100365
lambdageek merged 11 commits into
dotnet:mainfrom 
lambdageek:cdac-contract-descriptor
Conversation
@lambdageek
@lambdageeklambdageek
commented
Mar 27, 2024•

edited

Loading

Copy link


Copy Markdown

Member
Building on #100253 , describe an in-memory representation of the toplevel contract descriptor, comprisied of:
  • some target architecture properties
  • a data descriptor
  • a collection of compatible contracts
Contributes to #99298
Fixes#99299
@ghostghost
added
the 
needs-area-label
An area label is needed to ensure this gets routed to the appropriate area owners
label
Mar 27, 2024
@lambdageeklambdageek
added

documentation
Documentation bug or enhancement, does not impact product or test code
area-Meta

and removed

needs-area-label
An area label is needed to ensure this gets routed to the appropriate area owners
labels
Mar 27, 2024
@lambdageeklambdageek
mentioned this pull request
Mar 27, 2024
 Merged

davidwrighton
davidwrighton
reviewed
Mar 27, 2024
View reviewed changes
@davidwrightondavidwrighton
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is a good direction, but I have a few comments.
Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

elinor-fung
elinor-fung
reviewed
Mar 27, 2024
View reviewed changes
Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

jkotas
jkotas
reviewed
Mar 28, 2024
View reviewed changes
Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

jkotas
jkotas
reviewed
Mar 28, 2024
View reviewed changes
Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

@lambdageek
lambdageekforce-pushed
the
cdac-contract-descriptor
branch
3 times, most recently
from
f37c8a8 to
4bd9a58CompareMarch 28, 2024 17:22
jkotas
jkotas
reviewed
Mar 29, 2024
View reviewed changes
Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

@lambdageek
lambdageekforce-pushed
the
cdac-contract-descriptor
branch
from
4bd9a58 to
77afeacCompareMarch 29, 2024 13:30
@lambdageek
lambdageek
marked this pull request as ready for review
March 29, 2024 13:36
lambdageekand others
added 4 commits
March 29, 2024 09:37
@lambdageek
markdownlint
7640997
@lambdageek
don't use a pointer
a63782d
the issue is that we can't dereference pointers until we know the
target endianness and pointer size. which we can only discover by
reading the actual descriptor magic and flags
@lambdageek
flags not size_and_flags
2a23d12
@lambdageek
remove DotNetRuntimeContractDescriptorList - one runtime per module
e3d41c6
if there are multiple hosted runtimes, diagnostic tooling should look
in each loaded module to discover the contract descriptor
elinor-fung
elinor-fung
approved these changes
Apr 2, 2024
View reviewed changes
Comment threaddocs/design/datacontracts/datacontracts_design.md
Outdated

Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated


Comment on lines

+25
 to 
+26

 uint32_t aux_data_count;
 uint32_t descriptor_size;
@elinor-fungelinor-fungApr 2, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we put the count/size right before or after the descriptor/aux_data instead?
Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

jkotas
jkotas
reviewed
Apr 2, 2024
View reviewed changes
Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated


To aid in the discovery the contract descriptor should be exported by the module hosting the .NET
runtime with the name `DotNetRuntimeContractDescriptor`. (Using the C symbol conventions of the
target platform. That is, on platforms where such symbols typically have an `_` prepended, this
@jkotasjkotasApr 2, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What are those platforms?
FWIW, the existing DotNetRuntimeDebugHeader seems to be exported without _ everywhere: 
runtime/src/coreclr/nativeaot/BuildIntegration/Microsoft.NETCore.Native.targets

Lines 245 to 246
in
ab2bd48
 <IlcArgCondition="'$(_targetOS)' == 'win' and '$(DebuggerSupport)' != 'false'"Include="--export-dynamic-symbol:DotNetRuntimeDebugHeader,DATA" /> 
 <IlcArgCondition="'$(_targetOS)' != 'win' and '$(DebuggerSupport)' != 'false'"Include="--export-dynamic-symbol:DotNetRuntimeDebugHeader" /> 
jkotas
jkotas
reviewed
Apr 2, 2024
View reviewed changes
Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

If `ptrSize` is 0, the architecture is 64-bit. If it is 1, the architecture is 32-bit. The
reserved bits should be written as zero. Diagnostic tooling may ignore non-zero reserved bits.

The `descriptor` is a pointer to a JSON string described in [data descriptor physical layout](./data_descriptor.md#Physical_JSON_descriptor). The total length (including nul terminator character) is given by `descriptor_size`.
@jkotasjkotasApr 2, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: I would omit the terminating null in the descriptor_size.
The data contract reader cannot assume that the descriptor is null terminated. It needs to be robust against malicious input. Including the terminating null does not seem to be beneficial, it just adds extra logic to the reader to verify that there is a null at the end and strip it.
lambdageekand others
added 2 commits
April 4, 2024 10:29
@lambdageek@elinor-fung
Apply suggestions from code review
8c27593
Co-authored-by: Elinor Fung <elfung@microsoft.com>
@lambdageek
Review feedback
1553aa5
- put the aux data and descriptor sizes closer to the pointers
- Don't include trailing nul `descriptor_size`. Clarify it is
counting bytes and that `descriptor` is in UTF-8
- Simplify `DotNetRuntimeContractDescriptor` naming discussion
@lambdageek
lambdageek
merged commit e629779
into
dotnet:mainApr 4, 2024
@lambdageeklambdageek
mentioned this pull request
Apr 4, 2024

Closed

matouskozak pushed a commit
to matouskozak/runtime
that referenced
this pull request
Apr 30, 2024
@lambdageek@elinor-fung@matouskozak
[cdac] Physical contract descriptor spec (dotnet#100365)
091c101
Building on dotnet#100253 , describe an in-memory representation of the toplevel contract descriptor, comprisied of:
* some target architecture properties
* a data descriptor
* a collection of compatible contracts
Contributes to dotnet#99298Fixesdotnet#99299
---
* [cdac] Physical contract descriptor spec
* Add "contracts" to the data descriptor
* one runtime per module
if there are multiple hosted runtimes, diagnostic tooling should look in each loaded module to discover the contract descriptor
* Apply suggestions from code review
* Review feedback
- put the aux data and descriptor sizes closer to the pointers
- Don't include trailing nul `descriptor_size`. Clarify it is counting bytes and that `descriptor` is in UTF-8
- Simplify `DotNetRuntimeContractDescriptor` naming discussion
---------
Co-authored-by: Elinor Fung <elfung@microsoft.com>
@github-actionsgithub-actionsBot
locked and limited conversation to collaborators
May 5, 2024
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@jkotasjkotasjkotas left review comments
@davidwrightondavidwrightondavidwrighton left review comments
@elinor-fungelinor-fungelinor-fung approved these changes

+1 more reviewer

@mikem8361mikem8361mikem8361 left review comments
Reviewers whose approvals may not affect merge requirements

Assignees

@lambdageeklambdageek

Labels

area-MetadocumentationDocumentation bug or enhancement, does not impact product or test code

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

[cdac] Publish data stream spec


5 participants

@lambdageek@jkotas@mikem8361@davidwrighton@elinor-fung

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default");
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
[cdac] Physical contract descriptor spec - #100365
Merged
lambdageek merged 11 commits into
dotnet:mainfrom 
lambdageek:cdac-contract-descriptor
Apr 4, 2024
Merged
[cdac] Physical contract descriptor spec#100365
lambdageek merged 11 commits into
dotnet:mainfrom 
lambdageek:cdac-contract-descriptor
Conversation
@lambdageek
@lambdageeklambdageek
commented
Mar 27, 2024•

edited

Loading

Copy link


Copy Markdown

Member
Building on #100253 , describe an in-memory representation of the toplevel contract descriptor, comprisied of:
  • some target architecture properties
  • a data descriptor
  • a collection of compatible contracts
Contributes to #99298
Fixes#99299
@ghostghost
added
the 
needs-area-label
An area label is needed to ensure this gets routed to the appropriate area owners
label
Mar 27, 2024
@lambdageeklambdageek
added

documentation
Documentation bug or enhancement, does not impact product or test code
area-Meta

and removed

needs-area-label
An area label is needed to ensure this gets routed to the appropriate area owners
labels
Mar 27, 2024
@lambdageeklambdageek
mentioned this pull request
Mar 27, 2024
 Merged

davidwrighton
davidwrighton
reviewed
Mar 27, 2024
View reviewed changes
@davidwrightondavidwrighton
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is a good direction, but I have a few comments.
Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

elinor-fung
elinor-fung
reviewed
Mar 27, 2024
View reviewed changes
Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

jkotas
jkotas
reviewed
Mar 28, 2024
View reviewed changes
Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

jkotas
jkotas
reviewed
Mar 28, 2024
View reviewed changes
Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

@lambdageek
lambdageekforce-pushed
the
cdac-contract-descriptor
branch
3 times, most recently
from
f37c8a8 to
4bd9a58CompareMarch 28, 2024 17:22
jkotas
jkotas
reviewed
Mar 29, 2024
View reviewed changes
Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

@lambdageek
lambdageekforce-pushed
the
cdac-contract-descriptor
branch
from
4bd9a58 to
77afeacCompareMarch 29, 2024 13:30
@lambdageek
lambdageek
marked this pull request as ready for review
March 29, 2024 13:36
lambdageekand others
added 4 commits
March 29, 2024 09:37
@lambdageek
markdownlint
7640997
@lambdageek
don't use a pointer
a63782d
the issue is that we can't dereference pointers until we know the
target endianness and pointer size. which we can only discover by
reading the actual descriptor magic and flags
@lambdageek
flags not size_and_flags
2a23d12
@lambdageek
remove DotNetRuntimeContractDescriptorList - one runtime per module
e3d41c6
if there are multiple hosted runtimes, diagnostic tooling should look
in each loaded module to discover the contract descriptor
elinor-fung
elinor-fung
approved these changes
Apr 2, 2024
View reviewed changes
Comment threaddocs/design/datacontracts/datacontracts_design.md
Outdated

Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated


Comment on lines

+25
 to 
+26

 uint32_t aux_data_count;
 uint32_t descriptor_size;
@elinor-fungelinor-fungApr 2, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we put the count/size right before or after the descriptor/aux_data instead?
Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

jkotas
jkotas
reviewed
Apr 2, 2024
View reviewed changes
Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated


To aid in the discovery the contract descriptor should be exported by the module hosting the .NET
runtime with the name `DotNetRuntimeContractDescriptor`. (Using the C symbol conventions of the
target platform. That is, on platforms where such symbols typically have an `_` prepended, this
@jkotasjkotasApr 2, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What are those platforms?
FWIW, the existing DotNetRuntimeDebugHeader seems to be exported without _ everywhere: 
runtime/src/coreclr/nativeaot/BuildIntegration/Microsoft.NETCore.Native.targets

Lines 245 to 246
in
ab2bd48
 <IlcArgCondition="'$(_targetOS)' == 'win' and '$(DebuggerSupport)' != 'false'"Include="--export-dynamic-symbol:DotNetRuntimeDebugHeader,DATA" /> 
 <IlcArgCondition="'$(_targetOS)' != 'win' and '$(DebuggerSupport)' != 'false'"Include="--export-dynamic-symbol:DotNetRuntimeDebugHeader" /> 
jkotas
jkotas
reviewed
Apr 2, 2024
View reviewed changes
Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

If `ptrSize` is 0, the architecture is 64-bit. If it is 1, the architecture is 32-bit. The
reserved bits should be written as zero. Diagnostic tooling may ignore non-zero reserved bits.

The `descriptor` is a pointer to a JSON string described in [data descriptor physical layout](./data_descriptor.md#Physical_JSON_descriptor). The total length (including nul terminator character) is given by `descriptor_size`.
@jkotasjkotasApr 2, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: I would omit the terminating null in the descriptor_size.
The data contract reader cannot assume that the descriptor is null terminated. It needs to be robust against malicious input. Including the terminating null does not seem to be beneficial, it just adds extra logic to the reader to verify that there is a null at the end and strip it.
lambdageekand others
added 2 commits
April 4, 2024 10:29
@lambdageek@elinor-fung
Apply suggestions from code review
8c27593
Co-authored-by: Elinor Fung <elfung@microsoft.com>
@lambdageek
Review feedback
1553aa5
- put the aux data and descriptor sizes closer to the pointers
- Don't include trailing nul `descriptor_size`. Clarify it is
counting bytes and that `descriptor` is in UTF-8
- Simplify `DotNetRuntimeContractDescriptor` naming discussion
@lambdageek
lambdageek
merged commit e629779
into
dotnet:mainApr 4, 2024
@lambdageeklambdageek
mentioned this pull request
Apr 4, 2024

Closed

matouskozak pushed a commit
to matouskozak/runtime
that referenced
this pull request
Apr 30, 2024
@lambdageek@elinor-fung@matouskozak
[cdac] Physical contract descriptor spec (dotnet#100365)
091c101
Building on dotnet#100253 , describe an in-memory representation of the toplevel contract descriptor, comprisied of:
* some target architecture properties
* a data descriptor
* a collection of compatible contracts
Contributes to dotnet#99298Fixesdotnet#99299
---
* [cdac] Physical contract descriptor spec
* Add "contracts" to the data descriptor
* one runtime per module
if there are multiple hosted runtimes, diagnostic tooling should look in each loaded module to discover the contract descriptor
* Apply suggestions from code review
* Review feedback
- put the aux data and descriptor sizes closer to the pointers
- Don't include trailing nul `descriptor_size`. Clarify it is counting bytes and that `descriptor` is in UTF-8
- Simplify `DotNetRuntimeContractDescriptor` naming discussion
---------
Co-authored-by: Elinor Fung <elfung@microsoft.com>
@github-actionsgithub-actionsBot
locked and limited conversation to collaborators
May 5, 2024
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@jkotasjkotasjkotas left review comments
@davidwrightondavidwrightondavidwrighton left review comments
@elinor-fungelinor-fungelinor-fung approved these changes

+1 more reviewer

@mikem8361mikem8361mikem8361 left review comments
Reviewers whose approvals may not affect merge requirements

Assignees

@lambdageeklambdageek

Labels

area-MetadocumentationDocumentation bug or enhancement, does not impact product or test code

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

[cdac] Publish data stream spec


5 participants

@lambdageek@jkotas@mikem8361@davidwrighton@elinor-fung

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers");
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
[cdac] Physical contract descriptor spec - #100365
Merged
lambdageek merged 11 commits into
dotnet:mainfrom 
lambdageek:cdac-contract-descriptor
Apr 4, 2024
Merged
[cdac] Physical contract descriptor spec#100365
lambdageek merged 11 commits into
dotnet:mainfrom 
lambdageek:cdac-contract-descriptor
Conversation
@lambdageek
@lambdageeklambdageek
commented
Mar 27, 2024•

edited

Loading

Copy link


Copy Markdown

Member
Building on #100253 , describe an in-memory representation of the toplevel contract descriptor, comprisied of:
  • some target architecture properties
  • a data descriptor
  • a collection of compatible contracts
Contributes to #99298
Fixes#99299
@ghostghost
added
the 
needs-area-label
An area label is needed to ensure this gets routed to the appropriate area owners
label
Mar 27, 2024
@lambdageeklambdageek
added

documentation
Documentation bug or enhancement, does not impact product or test code
area-Meta

and removed

needs-area-label
An area label is needed to ensure this gets routed to the appropriate area owners
labels
Mar 27, 2024
@lambdageeklambdageek
mentioned this pull request
Mar 27, 2024
 Merged

davidwrighton
davidwrighton
reviewed
Mar 27, 2024
View reviewed changes
@davidwrightondavidwrighton
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is a good direction, but I have a few comments.
Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

elinor-fung
elinor-fung
reviewed
Mar 27, 2024
View reviewed changes
Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

jkotas
jkotas
reviewed
Mar 28, 2024
View reviewed changes
Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

jkotas
jkotas
reviewed
Mar 28, 2024
View reviewed changes
Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

@lambdageek
lambdageekforce-pushed
the
cdac-contract-descriptor
branch
3 times, most recently
from
f37c8a8 to
4bd9a58CompareMarch 28, 2024 17:22
jkotas
jkotas
reviewed
Mar 29, 2024
View reviewed changes
Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

@lambdageek
lambdageekforce-pushed
the
cdac-contract-descriptor
branch
from
4bd9a58 to
77afeacCompareMarch 29, 2024 13:30
@lambdageek
lambdageek
marked this pull request as ready for review
March 29, 2024 13:36
lambdageekand others
added 4 commits
March 29, 2024 09:37
@lambdageek
markdownlint
7640997
@lambdageek
don't use a pointer
a63782d
the issue is that we can't dereference pointers until we know the
target endianness and pointer size. which we can only discover by
reading the actual descriptor magic and flags
@lambdageek
flags not size_and_flags
2a23d12
@lambdageek
remove DotNetRuntimeContractDescriptorList - one runtime per module
e3d41c6
if there are multiple hosted runtimes, diagnostic tooling should look
in each loaded module to discover the contract descriptor
elinor-fung
elinor-fung
approved these changes
Apr 2, 2024
View reviewed changes
Comment threaddocs/design/datacontracts/datacontracts_design.md
Outdated

Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated


Comment on lines

+25
 to 
+26

 uint32_t aux_data_count;
 uint32_t descriptor_size;
@elinor-fungelinor-fungApr 2, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we put the count/size right before or after the descriptor/aux_data instead?
Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

jkotas
jkotas
reviewed
Apr 2, 2024
View reviewed changes
Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated


To aid in the discovery the contract descriptor should be exported by the module hosting the .NET
runtime with the name `DotNetRuntimeContractDescriptor`. (Using the C symbol conventions of the
target platform. That is, on platforms where such symbols typically have an `_` prepended, this
@jkotasjkotasApr 2, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What are those platforms?
FWIW, the existing DotNetRuntimeDebugHeader seems to be exported without _ everywhere: 
runtime/src/coreclr/nativeaot/BuildIntegration/Microsoft.NETCore.Native.targets

Lines 245 to 246
in
ab2bd48
 <IlcArgCondition="'$(_targetOS)' == 'win' and '$(DebuggerSupport)' != 'false'"Include="--export-dynamic-symbol:DotNetRuntimeDebugHeader,DATA" /> 
 <IlcArgCondition="'$(_targetOS)' != 'win' and '$(DebuggerSupport)' != 'false'"Include="--export-dynamic-symbol:DotNetRuntimeDebugHeader" /> 
jkotas
jkotas
reviewed
Apr 2, 2024
View reviewed changes
Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

If `ptrSize` is 0, the architecture is 64-bit. If it is 1, the architecture is 32-bit. The
reserved bits should be written as zero. Diagnostic tooling may ignore non-zero reserved bits.

The `descriptor` is a pointer to a JSON string described in [data descriptor physical layout](./data_descriptor.md#Physical_JSON_descriptor). The total length (including nul terminator character) is given by `descriptor_size`.
@jkotasjkotasApr 2, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: I would omit the terminating null in the descriptor_size.
The data contract reader cannot assume that the descriptor is null terminated. It needs to be robust against malicious input. Including the terminating null does not seem to be beneficial, it just adds extra logic to the reader to verify that there is a null at the end and strip it.
lambdageekand others
added 2 commits
April 4, 2024 10:29
@lambdageek@elinor-fung
Apply suggestions from code review
8c27593
Co-authored-by: Elinor Fung <elfung@microsoft.com>
@lambdageek
Review feedback
1553aa5
- put the aux data and descriptor sizes closer to the pointers
- Don't include trailing nul `descriptor_size`. Clarify it is
counting bytes and that `descriptor` is in UTF-8
- Simplify `DotNetRuntimeContractDescriptor` naming discussion
@lambdageek
lambdageek
merged commit e629779
into
dotnet:mainApr 4, 2024
@lambdageeklambdageek
mentioned this pull request
Apr 4, 2024

Closed

matouskozak pushed a commit
to matouskozak/runtime
that referenced
this pull request
Apr 30, 2024
@lambdageek@elinor-fung@matouskozak
[cdac] Physical contract descriptor spec (dotnet#100365)
091c101
Building on dotnet#100253 , describe an in-memory representation of the toplevel contract descriptor, comprisied of:
* some target architecture properties
* a data descriptor
* a collection of compatible contracts
Contributes to dotnet#99298Fixesdotnet#99299
---
* [cdac] Physical contract descriptor spec
* Add "contracts" to the data descriptor
* one runtime per module
if there are multiple hosted runtimes, diagnostic tooling should look in each loaded module to discover the contract descriptor
* Apply suggestions from code review
* Review feedback
- put the aux data and descriptor sizes closer to the pointers
- Don't include trailing nul `descriptor_size`. Clarify it is counting bytes and that `descriptor` is in UTF-8
- Simplify `DotNetRuntimeContractDescriptor` naming discussion
---------
Co-authored-by: Elinor Fung <elfung@microsoft.com>
@github-actionsgithub-actionsBot
locked and limited conversation to collaborators
May 5, 2024
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@jkotasjkotasjkotas left review comments
@davidwrightondavidwrightondavidwrighton left review comments
@elinor-fungelinor-fungelinor-fung approved these changes

+1 more reviewer

@mikem8361mikem8361mikem8361 left review comments
Reviewers whose approvals may not affect merge requirements

Assignees

@lambdageeklambdageek

Labels

area-MetadocumentationDocumentation bug or enhancement, does not impact product or test code

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

[cdac] Publish data stream spec


5 participants

@lambdageek@jkotas@mikem8361@davidwrighton@elinor-fung

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode");
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();

Skip to content
[cdac] Physical contract descriptor spec - #100365
Merged
lambdageek merged 11 commits into
dotnet:mainfrom 
lambdageek:cdac-contract-descriptor
Apr 4, 2024
Merged
[cdac] Physical contract descriptor spec#100365
lambdageek merged 11 commits into
dotnet:mainfrom 
lambdageek:cdac-contract-descriptor
Conversation
@lambdageek
@lambdageeklambdageek
commented
Mar 27, 2024•

edited

Loading

Copy link


Copy Markdown

Member
Building on #100253 , describe an in-memory representation of the toplevel contract descriptor, comprisied of:
  • some target architecture properties
  • a data descriptor
  • a collection of compatible contracts
Contributes to #99298
Fixes#99299
@ghostghost
added
the 
needs-area-label
An area label is needed to ensure this gets routed to the appropriate area owners
label
Mar 27, 2024
@lambdageeklambdageek
added

documentation
Documentation bug or enhancement, does not impact product or test code
area-Meta

and removed

needs-area-label
An area label is needed to ensure this gets routed to the appropriate area owners
labels
Mar 27, 2024
@lambdageeklambdageek
mentioned this pull request
Mar 27, 2024
 Merged

davidwrighton
davidwrighton
reviewed
Mar 27, 2024
View reviewed changes
@davidwrightondavidwrighton
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is a good direction, but I have a few comments.
Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

elinor-fung
elinor-fung
reviewed
Mar 27, 2024
View reviewed changes
Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

jkotas
jkotas
reviewed
Mar 28, 2024
View reviewed changes
Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

jkotas
jkotas
reviewed
Mar 28, 2024
View reviewed changes
Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

@lambdageek
lambdageekforce-pushed
the
cdac-contract-descriptor
branch
3 times, most recently
from
f37c8a8 to
4bd9a58CompareMarch 28, 2024 17:22
jkotas
jkotas
reviewed
Mar 29, 2024
View reviewed changes
Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

@lambdageek
lambdageekforce-pushed
the
cdac-contract-descriptor
branch
from
4bd9a58 to
77afeacCompareMarch 29, 2024 13:30
@lambdageek
lambdageek
marked this pull request as ready for review
March 29, 2024 13:36
lambdageekand others
added 4 commits
March 29, 2024 09:37
@lambdageek
markdownlint
7640997
@lambdageek
don't use a pointer
a63782d
the issue is that we can't dereference pointers until we know the
target endianness and pointer size. which we can only discover by
reading the actual descriptor magic and flags
@lambdageek
flags not size_and_flags
2a23d12
@lambdageek
remove DotNetRuntimeContractDescriptorList - one runtime per module
e3d41c6
if there are multiple hosted runtimes, diagnostic tooling should look
in each loaded module to discover the contract descriptor
elinor-fung
elinor-fung
approved these changes
Apr 2, 2024
View reviewed changes
Comment threaddocs/design/datacontracts/datacontracts_design.md
Outdated

Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated


Comment on lines

+25
 to 
+26

 uint32_t aux_data_count;
 uint32_t descriptor_size;
@elinor-fungelinor-fungApr 2, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we put the count/size right before or after the descriptor/aux_data instead?
Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

jkotas
jkotas
reviewed
Apr 2, 2024
View reviewed changes
Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated


To aid in the discovery the contract descriptor should be exported by the module hosting the .NET
runtime with the name `DotNetRuntimeContractDescriptor`. (Using the C symbol conventions of the
target platform. That is, on platforms where such symbols typically have an `_` prepended, this
@jkotasjkotasApr 2, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What are those platforms?
FWIW, the existing DotNetRuntimeDebugHeader seems to be exported without _ everywhere: 
runtime/src/coreclr/nativeaot/BuildIntegration/Microsoft.NETCore.Native.targets

Lines 245 to 246
in
ab2bd48
 <IlcArgCondition="'$(_targetOS)' == 'win' and '$(DebuggerSupport)' != 'false'"Include="--export-dynamic-symbol:DotNetRuntimeDebugHeader,DATA" /> 
 <IlcArgCondition="'$(_targetOS)' != 'win' and '$(DebuggerSupport)' != 'false'"Include="--export-dynamic-symbol:DotNetRuntimeDebugHeader" /> 
jkotas
jkotas
reviewed
Apr 2, 2024
View reviewed changes
Comment threaddocs/design/datacontracts/contract-descriptor.md
Outdated

If `ptrSize` is 0, the architecture is 64-bit. If it is 1, the architecture is 32-bit. The
reserved bits should be written as zero. Diagnostic tooling may ignore non-zero reserved bits.

The `descriptor` is a pointer to a JSON string described in [data descriptor physical layout](./data_descriptor.md#Physical_JSON_descriptor). The total length (including nul terminator character) is given by `descriptor_size`.
@jkotasjkotasApr 2, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: I would omit the terminating null in the descriptor_size.
The data contract reader cannot assume that the descriptor is null terminated. It needs to be robust against malicious input. Including the terminating null does not seem to be beneficial, it just adds extra logic to the reader to verify that there is a null at the end and strip it.
lambdageekand others
added 2 commits
April 4, 2024 10:29
@lambdageek@elinor-fung
Apply suggestions from code review
8c27593
Co-authored-by: Elinor Fung <elfung@microsoft.com>
@lambdageek
Review feedback
1553aa5
- put the aux data and descriptor sizes closer to the pointers
- Don't include trailing nul `descriptor_size`. Clarify it is
counting bytes and that `descriptor` is in UTF-8
- Simplify `DotNetRuntimeContractDescriptor` naming discussion
@lambdageek
lambdageek
merged commit e629779
into
dotnet:mainApr 4, 2024
@lambdageeklambdageek
mentioned this pull request
Apr 4, 2024

Closed

matouskozak pushed a commit
to matouskozak/runtime
that referenced
this pull request
Apr 30, 2024
@lambdageek@elinor-fung@matouskozak
[cdac] Physical contract descriptor spec (dotnet#100365)
091c101
Building on dotnet#100253 , describe an in-memory representation of the toplevel contract descriptor, comprisied of:
* some target architecture properties
* a data descriptor
* a collection of compatible contracts
Contributes to dotnet#99298Fixesdotnet#99299
---
* [cdac] Physical contract descriptor spec
* Add "contracts" to the data descriptor
* one runtime per module
if there are multiple hosted runtimes, diagnostic tooling should look in each loaded module to discover the contract descriptor
* Apply suggestions from code review
* Review feedback
- put the aux data and descriptor sizes closer to the pointers
- Don't include trailing nul `descriptor_size`. Clarify it is counting bytes and that `descriptor` is in UTF-8
- Simplify `DotNetRuntimeContractDescriptor` naming discussion
---------
Co-authored-by: Elinor Fung <elfung@microsoft.com>
@github-actionsgithub-actionsBot
locked and limited conversation to collaborators
May 5, 2024
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@jkotasjkotasjkotas left review comments
@davidwrightondavidwrightondavidwrighton left review comments
@elinor-fungelinor-fungelinor-fung approved these changes

+1 more reviewer

@mikem8361mikem8361mikem8361 left review comments
Reviewers whose approvals may not affect merge requirements

Assignees

@lambdageeklambdageek

Labels

area-MetadocumentationDocumentation bug or enhancement, does not impact product or test code

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

[cdac] Publish data stream spec


5 participants

@lambdageek@jkotas@mikem8361@davidwrighton@elinor-fung