Remove --flat from default dsymutil options

[agocke]

--flat is a diagnostic-only option and not fully supported. It can still be specified if necessary through the DsymUtilOptions property and NativeSymbolExt property, but it will no longer be the default.

[ghost]

Tagging subscribers to this area: @agocke, @MichalStrehovsky, @jkotas
See info in area-owners.md if you want to be subscribed.

Issue Details

---

--flat is a diagnostic-only option and not fully supported. It can still be specified if necessary through the DsymUtilOptions property and NativeSymbolExt property, but it will no longer be the default.

Author:	agocke
Assignees:	-
Labels:	area-NativeAOT-coreclr
Milestone:	-

[jkotas]

I assume that this is related to #86734.

Are we going to need a follow up change to upload symbols in the dsym format for AOT compiled binaries in the official builds?

We have been using --flat option for unmanaged binaries

runtime/eng/native/functions.cmake

Line 419 in d40ca00

set(DSYMUTIL_OPTS "--flat")

. Should we remove the --flat option for the unmanaged binaries as well?

[filipnavara]

Should we remove the --flat option for the unmanaged binaries as well?

There's a tracking issue for that - #88286.

I would be happy if this PR goes in since it makes debugging apps much easier but there's always the concern about SymStore and related utilities that are not able to handle dSYM yet.

[agocke]

Are we going to need a follow up change to upload symbols in the dsym format for AOT compiled binaries in the official builds?

Yeah, we can either set the --dwarf flag just for our CI, or we can support zipping up the dsym bundle. I'd prefer the latter, but will see if I can do the former in this PR just to unblock things.

As for native builds, I think they should also use the dsym bundle eventually, but it matters slightly less since it's only for debugging coreclr itself, which is a more uncommon user scenario. In this case, debugging Mac-compiled Native AOT apps is a first-class scenario, so we should use the Apple-preferred thing by default.

[agocke]

Well, I looked in the crossgen2 symbols package and I actually don't see the .dwarf file in there. Is that expected, @hoyosjs?

[hoyosjs]

No, that'd break postmortem of crossgen. There's several problems. Right now the easiest one is telling the SDK that dsym is a symbol file, after you need to teach the uploaders how to index dsyms.

[jkotas]

This change looks good on its own. It would be nice to have the follow up plan figured out before merging.

[agocke]

@hoyosjs can the symbol uploader index .dwarf files today? Also, .dsym is a folder. Would we want to zip it?

[hoyosjs]

I thought it was an archive. Don't know of prior art in symbol packages, need to check

[agocke]

OK, looking into it, I'm pretty sure the existing system is not working for collecting ILC symbols. But, to avoid any regressions, I've special-cased crossgen2 to have the old behavior.

Long-term, I don't see any fundamental restrictions to using .dsym files everywhere. We should just make sure that things up the chain are happy with it.

[agocke]

--flat is a diagnostic-only option and not fully supported. It can still be specified if necessary through the DsymUtilOptions property and NativeSymbolExt property, but it will no longer be the default.

[ghost]

Tagging subscribers to this area: @agocke, @MichalStrehovsky, @jkotas
See info in area-owners.md if you want to be subscribed.

Issue Details

---

--flat is a diagnostic-only option and not fully supported. It can still be specified if necessary through the DsymUtilOptions property and NativeSymbolExt property, but it will no longer be the default.

Author:	agocke
Assignees:	-
Labels:	area-NativeAOT-coreclr
Milestone:	-

[jkotas]

I assume that this is related to #86734.

Are we going to need a follow up change to upload symbols in the dsym format for AOT compiled binaries in the official builds?

We have been using --flat option for unmanaged binaries

runtime/eng/native/functions.cmake

Line 419 in d40ca00

set(DSYMUTIL_OPTS "--flat")

. Should we remove the --flat option for the unmanaged binaries as well?

[filipnavara]

Should we remove the --flat option for the unmanaged binaries as well?

There's a tracking issue for that - #88286.

I would be happy if this PR goes in since it makes debugging apps much easier but there's always the concern about SymStore and related utilities that are not able to handle dSYM yet.

[agocke]

Are we going to need a follow up change to upload symbols in the dsym format for AOT compiled binaries in the official builds?

Yeah, we can either set the --dwarf flag just for our CI, or we can support zipping up the dsym bundle. I'd prefer the latter, but will see if I can do the former in this PR just to unblock things.

As for native builds, I think they should also use the dsym bundle eventually, but it matters slightly less since it's only for debugging coreclr itself, which is a more uncommon user scenario. In this case, debugging Mac-compiled Native AOT apps is a first-class scenario, so we should use the Apple-preferred thing by default.

[agocke]

Well, I looked in the crossgen2 symbols package and I actually don't see the .dwarf file in there. Is that expected, @hoyosjs?

[hoyosjs]

No, that'd break postmortem of crossgen. There's several problems. Right now the easiest one is telling the SDK that dsym is a symbol file, after you need to teach the uploaders how to index dsyms.

[jkotas]

This change looks good on its own. It would be nice to have the follow up plan figured out before merging.

[agocke]

@hoyosjs can the symbol uploader index .dwarf files today? Also, .dsym is a folder. Would we want to zip it?

[hoyosjs]

I thought it was an archive. Don't know of prior art in symbol packages, need to check

[agocke]

OK, looking into it, I'm pretty sure the existing system is not working for collecting ILC symbols. But, to avoid any regressions, I've special-cased crossgen2 to have the old behavior.

Long-term, I don't see any fundamental restrictions to using .dsym files everywhere. We should just make sure that things up the chain are happy with it.

[agocke]

--flat is a diagnostic-only option and not fully supported. It can still be specified if necessary through the DsymUtilOptions property and NativeSymbolExt property, but it will no longer be the default.

[ghost]

Tagging subscribers to this area: @agocke, @MichalStrehovsky, @jkotas
See info in area-owners.md if you want to be subscribed.

Issue Details

---

--flat is a diagnostic-only option and not fully supported. It can still be specified if necessary through the DsymUtilOptions property and NativeSymbolExt property, but it will no longer be the default.

Author:	agocke
Assignees:	-
Labels:	area-NativeAOT-coreclr
Milestone:	-

[jkotas]

I assume that this is related to #86734.

Are we going to need a follow up change to upload symbols in the dsym format for AOT compiled binaries in the official builds?

We have been using --flat option for unmanaged binaries

runtime/eng/native/functions.cmake

Line 419 in d40ca00

set(DSYMUTIL_OPTS "--flat")

. Should we remove the --flat option for the unmanaged binaries as well?

[filipnavara]

Should we remove the --flat option for the unmanaged binaries as well?

There's a tracking issue for that - #88286.

I would be happy if this PR goes in since it makes debugging apps much easier but there's always the concern about SymStore and related utilities that are not able to handle dSYM yet.

[agocke]

Are we going to need a follow up change to upload symbols in the dsym format for AOT compiled binaries in the official builds?

Yeah, we can either set the --dwarf flag just for our CI, or we can support zipping up the dsym bundle. I'd prefer the latter, but will see if I can do the former in this PR just to unblock things.

As for native builds, I think they should also use the dsym bundle eventually, but it matters slightly less since it's only for debugging coreclr itself, which is a more uncommon user scenario. In this case, debugging Mac-compiled Native AOT apps is a first-class scenario, so we should use the Apple-preferred thing by default.

[agocke]

Well, I looked in the crossgen2 symbols package and I actually don't see the .dwarf file in there. Is that expected, @hoyosjs?

[hoyosjs]

No, that'd break postmortem of crossgen. There's several problems. Right now the easiest one is telling the SDK that dsym is a symbol file, after you need to teach the uploaders how to index dsyms.

[jkotas]

This change looks good on its own. It would be nice to have the follow up plan figured out before merging.

[agocke]

@hoyosjs can the symbol uploader index .dwarf files today? Also, .dsym is a folder. Would we want to zip it?

[hoyosjs]

I thought it was an archive. Don't know of prior art in symbol packages, need to check

[agocke]

OK, looking into it, I'm pretty sure the existing system is not working for collecting ILC symbols. But, to avoid any regressions, I've special-cased crossgen2 to have the old behavior.

Long-term, I don't see any fundamental restrictions to using .dsym files everywhere. We should just make sure that things up the chain are happy with it.

[agocke]

--flat is a diagnostic-only option and not fully supported. It can still be specified if necessary through the DsymUtilOptions property and NativeSymbolExt property, but it will no longer be the default.

[ghost]

Tagging subscribers to this area: @agocke, @MichalStrehovsky, @jkotas
See info in area-owners.md if you want to be subscribed.

Issue Details

---

--flat is a diagnostic-only option and not fully supported. It can still be specified if necessary through the DsymUtilOptions property and NativeSymbolExt property, but it will no longer be the default.

Author:	agocke
Assignees:	-
Labels:	area-NativeAOT-coreclr
Milestone:	-

[jkotas]

I assume that this is related to #86734.

Are we going to need a follow up change to upload symbols in the dsym format for AOT compiled binaries in the official builds?

We have been using --flat option for unmanaged binaries

runtime/eng/native/functions.cmake

Line 419 in d40ca00

set(DSYMUTIL_OPTS "--flat")

. Should we remove the --flat option for the unmanaged binaries as well?

[filipnavara]

Should we remove the --flat option for the unmanaged binaries as well?

There's a tracking issue for that - #88286.

I would be happy if this PR goes in since it makes debugging apps much easier but there's always the concern about SymStore and related utilities that are not able to handle dSYM yet.

[agocke]

Are we going to need a follow up change to upload symbols in the dsym format for AOT compiled binaries in the official builds?

Yeah, we can either set the --dwarf flag just for our CI, or we can support zipping up the dsym bundle. I'd prefer the latter, but will see if I can do the former in this PR just to unblock things.

As for native builds, I think they should also use the dsym bundle eventually, but it matters slightly less since it's only for debugging coreclr itself, which is a more uncommon user scenario. In this case, debugging Mac-compiled Native AOT apps is a first-class scenario, so we should use the Apple-preferred thing by default.

[agocke]

Well, I looked in the crossgen2 symbols package and I actually don't see the .dwarf file in there. Is that expected, @hoyosjs?

[hoyosjs]

No, that'd break postmortem of crossgen. There's several problems. Right now the easiest one is telling the SDK that dsym is a symbol file, after you need to teach the uploaders how to index dsyms.

[jkotas]

This change looks good on its own. It would be nice to have the follow up plan figured out before merging.

[agocke]

@hoyosjs can the symbol uploader index .dwarf files today? Also, .dsym is a folder. Would we want to zip it?

[hoyosjs]

I thought it was an archive. Don't know of prior art in symbol packages, need to check

[agocke]

OK, looking into it, I'm pretty sure the existing system is not working for collecting ILC symbols. But, to avoid any regressions, I've special-cased crossgen2 to have the old behavior.

Long-term, I don't see any fundamental restrictions to using .dsym files everywhere. We should just make sure that things up the chain are happy with it.

[agocke]

--flat is a diagnostic-only option and not fully supported. It can still be specified if necessary through the DsymUtilOptions property and NativeSymbolExt property, but it will no longer be the default.

[ghost]

Tagging subscribers to this area: @agocke, @MichalStrehovsky, @jkotas
See info in area-owners.md if you want to be subscribed.

Issue Details

---

--flat is a diagnostic-only option and not fully supported. It can still be specified if necessary through the DsymUtilOptions property and NativeSymbolExt property, but it will no longer be the default.

Author:	agocke
Assignees:	-
Labels:	area-NativeAOT-coreclr
Milestone:	-

[jkotas]

I assume that this is related to #86734.

Are we going to need a follow up change to upload symbols in the dsym format for AOT compiled binaries in the official builds?

We have been using --flat option for unmanaged binaries

runtime/eng/native/functions.cmake

Line 419 in d40ca00

set(DSYMUTIL_OPTS "--flat")

. Should we remove the --flat option for the unmanaged binaries as well?

[filipnavara]

Should we remove the --flat option for the unmanaged binaries as well?

There's a tracking issue for that - #88286.

I would be happy if this PR goes in since it makes debugging apps much easier but there's always the concern about SymStore and related utilities that are not able to handle dSYM yet.

[agocke]

Are we going to need a follow up change to upload symbols in the dsym format for AOT compiled binaries in the official builds?

Yeah, we can either set the --dwarf flag just for our CI, or we can support zipping up the dsym bundle. I'd prefer the latter, but will see if I can do the former in this PR just to unblock things.

As for native builds, I think they should also use the dsym bundle eventually, but it matters slightly less since it's only for debugging coreclr itself, which is a more uncommon user scenario. In this case, debugging Mac-compiled Native AOT apps is a first-class scenario, so we should use the Apple-preferred thing by default.

[agocke]

Well, I looked in the crossgen2 symbols package and I actually don't see the .dwarf file in there. Is that expected, @hoyosjs?

[hoyosjs]

No, that'd break postmortem of crossgen. There's several problems. Right now the easiest one is telling the SDK that dsym is a symbol file, after you need to teach the uploaders how to index dsyms.

[jkotas]

This change looks good on its own. It would be nice to have the follow up plan figured out before merging.

[agocke]

@hoyosjs can the symbol uploader index .dwarf files today? Also, .dsym is a folder. Would we want to zip it?

[hoyosjs]

I thought it was an archive. Don't know of prior art in symbol packages, need to check

[agocke]

OK, looking into it, I'm pretty sure the existing system is not working for collecting ILC symbols. But, to avoid any regressions, I've special-cased crossgen2 to have the old behavior.

Long-term, I don't see any fundamental restrictions to using .dsym files everywhere. We should just make sure that things up the chain are happy with it.

[agocke]

--flat is a diagnostic-only option and not fully supported. It can still be specified if necessary through the DsymUtilOptions property and NativeSymbolExt property, but it will no longer be the default.

[ghost]

Tagging subscribers to this area: @agocke, @MichalStrehovsky, @jkotas
See info in area-owners.md if you want to be subscribed.

Issue Details

---

--flat is a diagnostic-only option and not fully supported. It can still be specified if necessary through the DsymUtilOptions property and NativeSymbolExt property, but it will no longer be the default.

Author:	agocke
Assignees:	-
Labels:	area-NativeAOT-coreclr
Milestone:	-

[jkotas]

I assume that this is related to #86734.

Are we going to need a follow up change to upload symbols in the dsym format for AOT compiled binaries in the official builds?

We have been using --flat option for unmanaged binaries

runtime/eng/native/functions.cmake

Line 419 in d40ca00

set(DSYMUTIL_OPTS "--flat")

. Should we remove the --flat option for the unmanaged binaries as well?

[filipnavara]

Should we remove the --flat option for the unmanaged binaries as well?

There's a tracking issue for that - #88286.

I would be happy if this PR goes in since it makes debugging apps much easier but there's always the concern about SymStore and related utilities that are not able to handle dSYM yet.

[agocke]

Are we going to need a follow up change to upload symbols in the dsym format for AOT compiled binaries in the official builds?

Yeah, we can either set the --dwarf flag just for our CI, or we can support zipping up the dsym bundle. I'd prefer the latter, but will see if I can do the former in this PR just to unblock things.

As for native builds, I think they should also use the dsym bundle eventually, but it matters slightly less since it's only for debugging coreclr itself, which is a more uncommon user scenario. In this case, debugging Mac-compiled Native AOT apps is a first-class scenario, so we should use the Apple-preferred thing by default.

[agocke]

Well, I looked in the crossgen2 symbols package and I actually don't see the .dwarf file in there. Is that expected, @hoyosjs?

[hoyosjs]

No, that'd break postmortem of crossgen. There's several problems. Right now the easiest one is telling the SDK that dsym is a symbol file, after you need to teach the uploaders how to index dsyms.

[jkotas]

This change looks good on its own. It would be nice to have the follow up plan figured out before merging.

[agocke]

@hoyosjs can the symbol uploader index .dwarf files today? Also, .dsym is a folder. Would we want to zip it?

[hoyosjs]

I thought it was an archive. Don't know of prior art in symbol packages, need to check

[agocke]

OK, looking into it, I'm pretty sure the existing system is not working for collecting ILC symbols. But, to avoid any regressions, I've special-cased crossgen2 to have the old behavior.

Long-term, I don't see any fundamental restrictions to using .dsym files everywhere. We should just make sure that things up the chain are happy with it.

[agocke]

--flat is a diagnostic-only option and not fully supported. It can still be specified if necessary through the DsymUtilOptions property and NativeSymbolExt property, but it will no longer be the default.

[ghost]

Tagging subscribers to this area: @agocke, @MichalStrehovsky, @jkotas
See info in area-owners.md if you want to be subscribed.

Issue Details

---

--flat is a diagnostic-only option and not fully supported. It can still be specified if necessary through the DsymUtilOptions property and NativeSymbolExt property, but it will no longer be the default.

Author:	agocke
Assignees:	-
Labels:	area-NativeAOT-coreclr
Milestone:	-

[jkotas]

I assume that this is related to #86734.

Are we going to need a follow up change to upload symbols in the dsym format for AOT compiled binaries in the official builds?

We have been using --flat option for unmanaged binaries

runtime/eng/native/functions.cmake

Line 419 in d40ca00

set(DSYMUTIL_OPTS "--flat")

. Should we remove the --flat option for the unmanaged binaries as well?

[filipnavara]

Should we remove the --flat option for the unmanaged binaries as well?

There's a tracking issue for that - #88286.

I would be happy if this PR goes in since it makes debugging apps much easier but there's always the concern about SymStore and related utilities that are not able to handle dSYM yet.

[agocke]

Are we going to need a follow up change to upload symbols in the dsym format for AOT compiled binaries in the official builds?

Yeah, we can either set the --dwarf flag just for our CI, or we can support zipping up the dsym bundle. I'd prefer the latter, but will see if I can do the former in this PR just to unblock things.

As for native builds, I think they should also use the dsym bundle eventually, but it matters slightly less since it's only for debugging coreclr itself, which is a more uncommon user scenario. In this case, debugging Mac-compiled Native AOT apps is a first-class scenario, so we should use the Apple-preferred thing by default.

[agocke]

Well, I looked in the crossgen2 symbols package and I actually don't see the .dwarf file in there. Is that expected, @hoyosjs?

[hoyosjs]

No, that'd break postmortem of crossgen. There's several problems. Right now the easiest one is telling the SDK that dsym is a symbol file, after you need to teach the uploaders how to index dsyms.

[jkotas]

This change looks good on its own. It would be nice to have the follow up plan figured out before merging.

[agocke]

@hoyosjs can the symbol uploader index .dwarf files today? Also, .dsym is a folder. Would we want to zip it?

[hoyosjs]

I thought it was an archive. Don't know of prior art in symbol packages, need to check

[agocke]

OK, looking into it, I'm pretty sure the existing system is not working for collecting ILC symbols. But, to avoid any regressions, I've special-cased crossgen2 to have the old behavior.

Long-term, I don't see any fundamental restrictions to using .dsym files everywhere. We should just make sure that things up the chain are happy with it.

[agocke]

--flat is a diagnostic-only option and not fully supported. It can still be specified if necessary through the DsymUtilOptions property and NativeSymbolExt property, but it will no longer be the default.

[ghost]

Tagging subscribers to this area: @agocke, @MichalStrehovsky, @jkotas
See info in area-owners.md if you want to be subscribed.

Issue Details

---

--flat is a diagnostic-only option and not fully supported. It can still be specified if necessary through the DsymUtilOptions property and NativeSymbolExt property, but it will no longer be the default.

Author:	agocke
Assignees:	-
Labels:	area-NativeAOT-coreclr
Milestone:	-

[jkotas]

I assume that this is related to #86734.

Are we going to need a follow up change to upload symbols in the dsym format for AOT compiled binaries in the official builds?

We have been using --flat option for unmanaged binaries

runtime/eng/native/functions.cmake

Line 419 in d40ca00

set(DSYMUTIL_OPTS "--flat")

. Should we remove the --flat option for the unmanaged binaries as well?

[filipnavara]

Should we remove the --flat option for the unmanaged binaries as well?

There's a tracking issue for that - #88286.

I would be happy if this PR goes in since it makes debugging apps much easier but there's always the concern about SymStore and related utilities that are not able to handle dSYM yet.

[agocke]

Are we going to need a follow up change to upload symbols in the dsym format for AOT compiled binaries in the official builds?

Yeah, we can either set the --dwarf flag just for our CI, or we can support zipping up the dsym bundle. I'd prefer the latter, but will see if I can do the former in this PR just to unblock things.

As for native builds, I think they should also use the dsym bundle eventually, but it matters slightly less since it's only for debugging coreclr itself, which is a more uncommon user scenario. In this case, debugging Mac-compiled Native AOT apps is a first-class scenario, so we should use the Apple-preferred thing by default.

[agocke]

Well, I looked in the crossgen2 symbols package and I actually don't see the .dwarf file in there. Is that expected, @hoyosjs?

[hoyosjs]

No, that'd break postmortem of crossgen. There's several problems. Right now the easiest one is telling the SDK that dsym is a symbol file, after you need to teach the uploaders how to index dsyms.

[jkotas]

This change looks good on its own. It would be nice to have the follow up plan figured out before merging.

[agocke]

@hoyosjs can the symbol uploader index .dwarf files today? Also, .dsym is a folder. Would we want to zip it?

[hoyosjs]

I thought it was an archive. Don't know of prior art in symbol packages, need to check

[agocke]

OK, looking into it, I'm pretty sure the existing system is not working for collecting ILC symbols. But, to avoid any regressions, I've special-cased crossgen2 to have the old behavior.

Long-term, I don't see any fundamental restrictions to using .dsym files everywhere. We should just make sure that things up the chain are happy with it.