[cdac] Read contract descriptor from target by elinor-fung · Pull Request #101208 · dotnet/runtime

elinor-fung · 2024-04-17T20:21:17Z

Get DotNetRuntimeContractDescriptor address from the target
Read contract descriptor to determine endianness and pointer size
Parse JSON descriptor and store contracts

Depends on #100650 to actually work.

Contributes to #99298

dotnet-policy-service · 2024-04-17T20:21:45Z

Tagging subscribers to this area: @tommcdon
See info in area-owners.md if you want to be subscribed.

lambdageek · 2024-04-19T18:12:52Z

Please consider cherrypicking lambdageek@dad6f1e from https://github.com/lambdageek/runtime/tree/cdac-roundtrip-main to hook up JSON parser

Until we get dotnet#101048 from codeflow, work around dotnet#101205 by adding a trimmer root for JsonDerivedTypeAttribute[].

… export symbol

elinor-fung · 2024-04-19T22:39:38Z

Cherry-picked the JSON parser change. One edit I made on top of that was to make the ContractDescriptorParser specify a DynamicDependency to Root instead going through the xml file.

elinor-fung · 2024-04-20T01:38:10Z

This change is also disabling the cDAC on platforms where we don't read export symbols via TryGetSymbol in dbgutil (c215266).

elfreader.cpp is not included for all platforms:

runtime/src/coreclr/debug/dbgutil/CMakeLists.txt

Lines 20 to 23 in 907eff8

if(CLR_CMAKE_TARGET_LINUX OR CLR_CMAKE_TARGET_HAIKU) 
list(APPEND DBGUTIL_SOURCES 
 elfreader.cpp 
 ) 

@mikem8361 do you know if there's a specific reason elfreader shouldn't work on other Unix platforms?

I could include it in freebsd with something like elinor-fung@18a7dee. That builds, but I don't know if it actually works.

mikem8361 · 2024-04-20T18:25:55Z

@mikem8361 do you know if there's a specific reason elfreader shouldn't work on other Unix platforms?
I could include it in freebsd with something like https://github.com/elinor-fung/runtime/commit/18a7deee67ecfccddc1e1b364333cabf8b56192b. That builds, but I don't know if it actually works.

The ELFReader should work just fine other platforms but it will just take some work/testing.

lambdageek · 2024-04-22T17:54:46Z

 for (int i = 0; i < pointerDataCount; i++)
 {
- _pointerData[i] = ReadPointer(pointerData.Value + (uint)(i * _pointerSize));
+ if (!TryReadPointer(pointerDataAddr.Value + (uint)(i * pointerSize), config, reader, out pointerData[i]))


something to consider as we go forward: should we have a struct TargetSpan<T> where T: unmanaged for "pointer+length" for remote memory

- Get `DotNetRuntimeContractDescriptor` address from the target - Read contract descriptor to determine endianness and pointer size - Parse JSON descriptor and store contracts

elinor-fung · 2024-04-17T20:21:17Z

Get DotNetRuntimeContractDescriptor address from the target
Read contract descriptor to determine endianness and pointer size
Parse JSON descriptor and store contracts

Depends on #100650 to actually work.

Contributes to #99298

dotnet-policy-service · 2024-04-17T20:21:45Z

Tagging subscribers to this area: @tommcdon
See info in area-owners.md if you want to be subscribed.

lambdageek · 2024-04-19T18:12:52Z

Please consider cherrypicking lambdageek@dad6f1e from https://github.com/lambdageek/runtime/tree/cdac-roundtrip-main to hook up JSON parser

Until we get dotnet#101048 from codeflow, work around dotnet#101205 by adding a trimmer root for JsonDerivedTypeAttribute[].

… export symbol

elinor-fung · 2024-04-19T22:39:38Z

Cherry-picked the JSON parser change. One edit I made on top of that was to make the ContractDescriptorParser specify a DynamicDependency to Root instead going through the xml file.

elinor-fung · 2024-04-20T01:38:10Z

This change is also disabling the cDAC on platforms where we don't read export symbols via TryGetSymbol in dbgutil (c215266).

elfreader.cpp is not included for all platforms:

runtime/src/coreclr/debug/dbgutil/CMakeLists.txt

Lines 20 to 23 in 907eff8

if(CLR_CMAKE_TARGET_LINUX OR CLR_CMAKE_TARGET_HAIKU) 
list(APPEND DBGUTIL_SOURCES 
 elfreader.cpp 
 ) 

@mikem8361 do you know if there's a specific reason elfreader shouldn't work on other Unix platforms?

I could include it in freebsd with something like elinor-fung@18a7dee. That builds, but I don't know if it actually works.

mikem8361 · 2024-04-20T18:25:55Z

@mikem8361 do you know if there's a specific reason elfreader shouldn't work on other Unix platforms?
I could include it in freebsd with something like https://github.com/elinor-fung/runtime/commit/18a7deee67ecfccddc1e1b364333cabf8b56192b. That builds, but I don't know if it actually works.

The ELFReader should work just fine other platforms but it will just take some work/testing.

lambdageek · 2024-04-22T17:54:46Z

 for (int i = 0; i < pointerDataCount; i++)
 {
- _pointerData[i] = ReadPointer(pointerData.Value + (uint)(i * _pointerSize));
+ if (!TryReadPointer(pointerDataAddr.Value + (uint)(i * pointerSize), config, reader, out pointerData[i]))


something to consider as we go forward: should we have a struct TargetSpan<T> where T: unmanaged for "pointer+length" for remote memory

- Get `DotNetRuntimeContractDescriptor` address from the target - Read contract descriptor to determine endianness and pointer size - Parse JSON descriptor and store contracts

elinor-fung · 2024-04-17T20:21:17Z

Get DotNetRuntimeContractDescriptor address from the target
Read contract descriptor to determine endianness and pointer size
Parse JSON descriptor and store contracts

Depends on #100650 to actually work.

Contributes to #99298

dotnet-policy-service · 2024-04-17T20:21:45Z

Tagging subscribers to this area: @tommcdon
See info in area-owners.md if you want to be subscribed.

lambdageek · 2024-04-19T18:12:52Z

Please consider cherrypicking lambdageek@dad6f1e from https://github.com/lambdageek/runtime/tree/cdac-roundtrip-main to hook up JSON parser

Until we get dotnet#101048 from codeflow, work around dotnet#101205 by adding a trimmer root for JsonDerivedTypeAttribute[].

… export symbol

elinor-fung · 2024-04-19T22:39:38Z

Cherry-picked the JSON parser change. One edit I made on top of that was to make the ContractDescriptorParser specify a DynamicDependency to Root instead going through the xml file.

elinor-fung · 2024-04-20T01:38:10Z

This change is also disabling the cDAC on platforms where we don't read export symbols via TryGetSymbol in dbgutil (c215266).

elfreader.cpp is not included for all platforms:

runtime/src/coreclr/debug/dbgutil/CMakeLists.txt

Lines 20 to 23 in 907eff8

if(CLR_CMAKE_TARGET_LINUX OR CLR_CMAKE_TARGET_HAIKU) 
list(APPEND DBGUTIL_SOURCES 
 elfreader.cpp 
 ) 

@mikem8361 do you know if there's a specific reason elfreader shouldn't work on other Unix platforms?

I could include it in freebsd with something like elinor-fung@18a7dee. That builds, but I don't know if it actually works.

mikem8361 · 2024-04-20T18:25:55Z

@mikem8361 do you know if there's a specific reason elfreader shouldn't work on other Unix platforms?
I could include it in freebsd with something like https://github.com/elinor-fung/runtime/commit/18a7deee67ecfccddc1e1b364333cabf8b56192b. That builds, but I don't know if it actually works.

The ELFReader should work just fine other platforms but it will just take some work/testing.

lambdageek · 2024-04-22T17:54:46Z

 for (int i = 0; i < pointerDataCount; i++)
 {
- _pointerData[i] = ReadPointer(pointerData.Value + (uint)(i * _pointerSize));
+ if (!TryReadPointer(pointerDataAddr.Value + (uint)(i * pointerSize), config, reader, out pointerData[i]))


something to consider as we go forward: should we have a struct TargetSpan<T> where T: unmanaged for "pointer+length" for remote memory

- Get `DotNetRuntimeContractDescriptor` address from the target - Read contract descriptor to determine endianness and pointer size - Parse JSON descriptor and store contracts

elinor-fung · 2024-04-17T20:21:17Z

Get DotNetRuntimeContractDescriptor address from the target
Read contract descriptor to determine endianness and pointer size
Parse JSON descriptor and store contracts

Depends on #100650 to actually work.

Contributes to #99298

dotnet-policy-service · 2024-04-17T20:21:45Z

Tagging subscribers to this area: @tommcdon
See info in area-owners.md if you want to be subscribed.

lambdageek · 2024-04-19T18:12:52Z

Please consider cherrypicking lambdageek@dad6f1e from https://github.com/lambdageek/runtime/tree/cdac-roundtrip-main to hook up JSON parser

Until we get dotnet#101048 from codeflow, work around dotnet#101205 by adding a trimmer root for JsonDerivedTypeAttribute[].

… export symbol

elinor-fung · 2024-04-19T22:39:38Z

Cherry-picked the JSON parser change. One edit I made on top of that was to make the ContractDescriptorParser specify a DynamicDependency to Root instead going through the xml file.

elinor-fung · 2024-04-20T01:38:10Z

This change is also disabling the cDAC on platforms where we don't read export symbols via TryGetSymbol in dbgutil (c215266).

elfreader.cpp is not included for all platforms:

runtime/src/coreclr/debug/dbgutil/CMakeLists.txt

Lines 20 to 23 in 907eff8

if(CLR_CMAKE_TARGET_LINUX OR CLR_CMAKE_TARGET_HAIKU) 
list(APPEND DBGUTIL_SOURCES 
 elfreader.cpp 
 ) 

@mikem8361 do you know if there's a specific reason elfreader shouldn't work on other Unix platforms?

I could include it in freebsd with something like elinor-fung@18a7dee. That builds, but I don't know if it actually works.

mikem8361 · 2024-04-20T18:25:55Z

@mikem8361 do you know if there's a specific reason elfreader shouldn't work on other Unix platforms?
I could include it in freebsd with something like https://github.com/elinor-fung/runtime/commit/18a7deee67ecfccddc1e1b364333cabf8b56192b. That builds, but I don't know if it actually works.

The ELFReader should work just fine other platforms but it will just take some work/testing.

lambdageek · 2024-04-22T17:54:46Z

 for (int i = 0; i < pointerDataCount; i++)
 {
- _pointerData[i] = ReadPointer(pointerData.Value + (uint)(i * _pointerSize));
+ if (!TryReadPointer(pointerDataAddr.Value + (uint)(i * pointerSize), config, reader, out pointerData[i]))


something to consider as we go forward: should we have a struct TargetSpan<T> where T: unmanaged for "pointer+length" for remote memory

- Get `DotNetRuntimeContractDescriptor` address from the target - Read contract descriptor to determine endianness and pointer size - Parse JSON descriptor and store contracts

elinor-fung · 2024-04-17T20:21:17Z

Get DotNetRuntimeContractDescriptor address from the target
Read contract descriptor to determine endianness and pointer size
Parse JSON descriptor and store contracts

Depends on #100650 to actually work.

Contributes to #99298

dotnet-policy-service · 2024-04-17T20:21:45Z

Tagging subscribers to this area: @tommcdon
See info in area-owners.md if you want to be subscribed.

lambdageek · 2024-04-19T18:12:52Z

Please consider cherrypicking lambdageek@dad6f1e from https://github.com/lambdageek/runtime/tree/cdac-roundtrip-main to hook up JSON parser

Until we get dotnet#101048 from codeflow, work around dotnet#101205 by adding a trimmer root for JsonDerivedTypeAttribute[].

… export symbol

elinor-fung · 2024-04-19T22:39:38Z

Cherry-picked the JSON parser change. One edit I made on top of that was to make the ContractDescriptorParser specify a DynamicDependency to Root instead going through the xml file.

elinor-fung · 2024-04-20T01:38:10Z

This change is also disabling the cDAC on platforms where we don't read export symbols via TryGetSymbol in dbgutil (c215266).

elfreader.cpp is not included for all platforms:

runtime/src/coreclr/debug/dbgutil/CMakeLists.txt

Lines 20 to 23 in 907eff8

if(CLR_CMAKE_TARGET_LINUX OR CLR_CMAKE_TARGET_HAIKU) 
list(APPEND DBGUTIL_SOURCES 
 elfreader.cpp 
 ) 

@mikem8361 do you know if there's a specific reason elfreader shouldn't work on other Unix platforms?

I could include it in freebsd with something like elinor-fung@18a7dee. That builds, but I don't know if it actually works.

mikem8361 · 2024-04-20T18:25:55Z

@mikem8361 do you know if there's a specific reason elfreader shouldn't work on other Unix platforms?
I could include it in freebsd with something like https://github.com/elinor-fung/runtime/commit/18a7deee67ecfccddc1e1b364333cabf8b56192b. That builds, but I don't know if it actually works.

The ELFReader should work just fine other platforms but it will just take some work/testing.

lambdageek · 2024-04-22T17:54:46Z

 for (int i = 0; i < pointerDataCount; i++)
 {
- _pointerData[i] = ReadPointer(pointerData.Value + (uint)(i * _pointerSize));
+ if (!TryReadPointer(pointerDataAddr.Value + (uint)(i * pointerSize), config, reader, out pointerData[i]))


something to consider as we go forward: should we have a struct TargetSpan<T> where T: unmanaged for "pointer+length" for remote memory

- Get `DotNetRuntimeContractDescriptor` address from the target - Read contract descriptor to determine endianness and pointer size - Parse JSON descriptor and store contracts

elinor-fung · 2024-04-17T20:21:17Z

Get DotNetRuntimeContractDescriptor address from the target
Read contract descriptor to determine endianness and pointer size
Parse JSON descriptor and store contracts

Depends on #100650 to actually work.

Contributes to #99298

dotnet-policy-service · 2024-04-17T20:21:45Z

Tagging subscribers to this area: @tommcdon
See info in area-owners.md if you want to be subscribed.

lambdageek · 2024-04-19T18:12:52Z

Please consider cherrypicking lambdageek@dad6f1e from https://github.com/lambdageek/runtime/tree/cdac-roundtrip-main to hook up JSON parser

Until we get dotnet#101048 from codeflow, work around dotnet#101205 by adding a trimmer root for JsonDerivedTypeAttribute[].

… export symbol

elinor-fung · 2024-04-19T22:39:38Z

Cherry-picked the JSON parser change. One edit I made on top of that was to make the ContractDescriptorParser specify a DynamicDependency to Root instead going through the xml file.

elinor-fung · 2024-04-20T01:38:10Z

This change is also disabling the cDAC on platforms where we don't read export symbols via TryGetSymbol in dbgutil (c215266).

elfreader.cpp is not included for all platforms:

runtime/src/coreclr/debug/dbgutil/CMakeLists.txt

Lines 20 to 23 in 907eff8

if(CLR_CMAKE_TARGET_LINUX OR CLR_CMAKE_TARGET_HAIKU) 
list(APPEND DBGUTIL_SOURCES 
 elfreader.cpp 
 ) 

@mikem8361 do you know if there's a specific reason elfreader shouldn't work on other Unix platforms?

I could include it in freebsd with something like elinor-fung@18a7dee. That builds, but I don't know if it actually works.

mikem8361 · 2024-04-20T18:25:55Z

@mikem8361 do you know if there's a specific reason elfreader shouldn't work on other Unix platforms?
I could include it in freebsd with something like https://github.com/elinor-fung/runtime/commit/18a7deee67ecfccddc1e1b364333cabf8b56192b. That builds, but I don't know if it actually works.

The ELFReader should work just fine other platforms but it will just take some work/testing.

lambdageek · 2024-04-22T17:54:46Z

 for (int i = 0; i < pointerDataCount; i++)
 {
- _pointerData[i] = ReadPointer(pointerData.Value + (uint)(i * _pointerSize));
+ if (!TryReadPointer(pointerDataAddr.Value + (uint)(i * pointerSize), config, reader, out pointerData[i]))


something to consider as we go forward: should we have a struct TargetSpan<T> where T: unmanaged for "pointer+length" for remote memory

- Get `DotNetRuntimeContractDescriptor` address from the target - Read contract descriptor to determine endianness and pointer size - Parse JSON descriptor and store contracts

elinor-fung · 2024-04-17T20:21:17Z

Get DotNetRuntimeContractDescriptor address from the target
Read contract descriptor to determine endianness and pointer size
Parse JSON descriptor and store contracts

Depends on #100650 to actually work.

Contributes to #99298

dotnet-policy-service · 2024-04-17T20:21:45Z

Tagging subscribers to this area: @tommcdon
See info in area-owners.md if you want to be subscribed.

lambdageek · 2024-04-19T18:12:52Z

Please consider cherrypicking lambdageek@dad6f1e from https://github.com/lambdageek/runtime/tree/cdac-roundtrip-main to hook up JSON parser

Until we get dotnet#101048 from codeflow, work around dotnet#101205 by adding a trimmer root for JsonDerivedTypeAttribute[].

… export symbol

elinor-fung · 2024-04-19T22:39:38Z

Cherry-picked the JSON parser change. One edit I made on top of that was to make the ContractDescriptorParser specify a DynamicDependency to Root instead going through the xml file.

elinor-fung · 2024-04-20T01:38:10Z

This change is also disabling the cDAC on platforms where we don't read export symbols via TryGetSymbol in dbgutil (c215266).

elfreader.cpp is not included for all platforms:

runtime/src/coreclr/debug/dbgutil/CMakeLists.txt

Lines 20 to 23 in 907eff8

if(CLR_CMAKE_TARGET_LINUX OR CLR_CMAKE_TARGET_HAIKU) 
list(APPEND DBGUTIL_SOURCES 
 elfreader.cpp 
 ) 

@mikem8361 do you know if there's a specific reason elfreader shouldn't work on other Unix platforms?

I could include it in freebsd with something like elinor-fung@18a7dee. That builds, but I don't know if it actually works.

mikem8361 · 2024-04-20T18:25:55Z

@mikem8361 do you know if there's a specific reason elfreader shouldn't work on other Unix platforms?
I could include it in freebsd with something like https://github.com/elinor-fung/runtime/commit/18a7deee67ecfccddc1e1b364333cabf8b56192b. That builds, but I don't know if it actually works.

The ELFReader should work just fine other platforms but it will just take some work/testing.

lambdageek · 2024-04-22T17:54:46Z

 for (int i = 0; i < pointerDataCount; i++)
 {
- _pointerData[i] = ReadPointer(pointerData.Value + (uint)(i * _pointerSize));
+ if (!TryReadPointer(pointerDataAddr.Value + (uint)(i * pointerSize), config, reader, out pointerData[i]))


something to consider as we go forward: should we have a struct TargetSpan<T> where T: unmanaged for "pointer+length" for remote memory

- Get `DotNetRuntimeContractDescriptor` address from the target - Read contract descriptor to determine endianness and pointer size - Parse JSON descriptor and store contracts

elinor-fung · 2024-04-17T20:21:17Z

Get DotNetRuntimeContractDescriptor address from the target
Read contract descriptor to determine endianness and pointer size
Parse JSON descriptor and store contracts

Depends on #100650 to actually work.

Contributes to #99298

dotnet-policy-service · 2024-04-17T20:21:45Z

Tagging subscribers to this area: @tommcdon
See info in area-owners.md if you want to be subscribed.

lambdageek · 2024-04-19T18:12:52Z

Please consider cherrypicking lambdageek@dad6f1e from https://github.com/lambdageek/runtime/tree/cdac-roundtrip-main to hook up JSON parser

Until we get dotnet#101048 from codeflow, work around dotnet#101205 by adding a trimmer root for JsonDerivedTypeAttribute[].

… export symbol

elinor-fung · 2024-04-19T22:39:38Z

Cherry-picked the JSON parser change. One edit I made on top of that was to make the ContractDescriptorParser specify a DynamicDependency to Root instead going through the xml file.

elinor-fung · 2024-04-20T01:38:10Z

This change is also disabling the cDAC on platforms where we don't read export symbols via TryGetSymbol in dbgutil (c215266).

elfreader.cpp is not included for all platforms:

runtime/src/coreclr/debug/dbgutil/CMakeLists.txt

Lines 20 to 23 in 907eff8

if(CLR_CMAKE_TARGET_LINUX OR CLR_CMAKE_TARGET_HAIKU) 
list(APPEND DBGUTIL_SOURCES 
 elfreader.cpp 
 ) 

@mikem8361 do you know if there's a specific reason elfreader shouldn't work on other Unix platforms?

I could include it in freebsd with something like elinor-fung@18a7dee. That builds, but I don't know if it actually works.

mikem8361 · 2024-04-20T18:25:55Z

@mikem8361 do you know if there's a specific reason elfreader shouldn't work on other Unix platforms?
I could include it in freebsd with something like https://github.com/elinor-fung/runtime/commit/18a7deee67ecfccddc1e1b364333cabf8b56192b. That builds, but I don't know if it actually works.

The ELFReader should work just fine other platforms but it will just take some work/testing.

lambdageek · 2024-04-22T17:54:46Z

 for (int i = 0; i < pointerDataCount; i++)
 {
- _pointerData[i] = ReadPointer(pointerData.Value + (uint)(i * _pointerSize));
+ if (!TryReadPointer(pointerDataAddr.Value + (uint)(i * pointerSize), config, reader, out pointerData[i]))


something to consider as we go forward: should we have a struct TargetSpan<T> where T: unmanaged for "pointer+length" for remote memory

- Get `DotNetRuntimeContractDescriptor` address from the target - Read contract descriptor to determine endianness and pointer size - Parse JSON descriptor and store contracts

ghost added the area-Diagnostics-coreclr label Apr 17, 2024

dotnet-policy-serviceBot assigned elinor-fung Apr 17, 2024

lambdageek reviewed Apr 17, 2024
View reviewed changes

Comment threadsrc/native/managed/cdacreader/src/Target.cs Outdated

jkoritzinsky reviewed Apr 18, 2024
View reviewed changes

Comment threadsrc/native/managed/cdacreader/src/Target.cs Outdated

elinor-fung added 4 commits April 18, 2024 21:53

Pass DotNetRuntimeContractDescriptor address to cdac
9fdbbe7

Read contract descriptor from target
8db6283

Add threshold for stackalloc when reading descriptor
80af170

Use UTF-8 literals
faa1722

elinor-fung force-pushed the cdac-read-target branch from 294072a to faa1722Compare April 19, 2024 04:53

build-analysisBot mentioned this pull request Apr 19, 2024
System.IO.Net5Compat.Tests and System.IO.Tests suddenly exiting with error 137 #100558
Closed

lambdageekand others added 3 commits April 19, 2024 11:47

Parse JSON and store the contracts
85ffbda
Until we get dotnet#101048 from codeflow, work around dotnet#101205 by adding a trimmer root for JsonDerivedTypeAttribute[].

Only use cDAC on platforms where we implement functionality to get an…
c215266
… export symbol

Use DynamicDependency attribute instead of xml file
7468518

elinor-fung marked this pull request as ready for review April 20, 2024 01:30

elinor-fung requested review from AaronRobinsonMSFT and lambdageek April 20, 2024 01:30

lambdageek reviewed Apr 20, 2024
View reviewed changes

Comment threadsrc/native/managed/cdacreader/src/Target.cs Outdated

mikem8361 reviewed Apr 20, 2024
View reviewed changes

Comment threadsrc/coreclr/debug/daccess/daccess.cpp

Fix reading from target after initialization
1dee369

mikem8361 approved these changes Apr 21, 2024
View reviewed changes

Better target initialization
5e6fbb0

lambdageek approved these changes Apr 22, 2024
View reviewed changes

This was referenced Apr 22, 2024
Portable Data Contract-based DAC #99298
Open
System.Net.Http.Functional.Tests.HttpRequestMessageTest.HttpRequest_BodylessMethod_NoContentLength throwing FormatException #101399
Closed

build-analysisBot mentioned this pull request Apr 22, 2024
[wasm] System.Text.Json.Tests running out of memory #98578
Closed

elinor-fung merged commit e751fe8 into dotnet:mainApr 22, 2024

elinor-fung deleted the cdac-read-target branch April 22, 2024 21:57

build-analysisBot mentioned this pull request Apr 23, 2024
[browser] System.Buffers.ArrayPool.Tests.ArrayPoolUnitTests fails with out of memory #101436
Closed

github-actionsBot locked and limited conversation to collaborators May 23, 2024