Skip to content

[Feature Request] Diagnostic IPC command to dynamically add startup hook at diagnostic startup suspension #83756

Description

@jander-msft

Request

Add diagnostic IPC command that allows prepending a path to the startup hooks (e.g. what's parsed from the DOTNET_STARTUP_HOOKS environment variable by the host) in the host during diagnostic startup suspension. This would allow dotnet-monitor to better participate in startup diagnostics scenarios via managed code without requiring our customers to meticulously specify the DOTNET_STARTUP_HOOKS environment variable.

Ideally, the tool is able to invoke something to specify the path to the startup hook by some mechanism and not leave it up to the user to figure out what that path is, set it correctly, and make sure that the library is available at the correct time.

Suggested commands would be:

  • AddStartupHook: Adds a startup hook path (preferably prepends) to the existing list of startup hooks in the host.

OR:

  • GetStartupHooks: Gets the currently list of startup hooks from the host.
  • SetStartupHooks: Sets the list of startup hooks in the host.

Background

The .NET Monitor team is looking for a mechanism by which the dotnet-monitor tool can dynamically specify a startup hook so that the tool can participate in startup diagnostics for aspects of the runtime that do not have native diagnostic scenarios. For example, the tool wants to collect exceptions from the beginning of the process in order to aid in startup failures.

The startup hook notion is well-positioned to allow integration with applications before their entrypoints are executed. This feature uses the DOTNET_STARTUP_HOOKS environment variable is set at process start in order to determine the list of startup hook assemblies that should be loaded and executed. The host will load the environment variable value and make it available as part of the AppContext data. When the main thread is initialized, the StartupHookProvider.ProcessStartupHooks will consume the value and execute each of the specified startup hooks.

This mechanism works well if the envirnoment in which the app is executing is managed and prepopulated with the startup hook assemblies in well-known locations. The DOTNET_STARTUP_HOOKS environment variable must contain assembly paths that exist on disk at the time of execution or assembly names from which the probing algorithm for the default load context can locate it. If one of the paths do not exist, it will take down the process before it starts executing the application. This largely requires that either the managed environment knows exactly where the assemblies are so it can set the environment variable or a person needs to manually configure their deployment to ensure that the assembly paths are correct; both need to ensure that the files are available at execution time.

Details

Here's where it gets tricky for .NET Monitor:

  • The dotnet-monitor tool bundles its startup hook assembly with itself for easy acquisition.
  • The dotnet-monitor tool is largely acquirable through two means: .NET Tool install or a Docker image
    • .NET Tool Installation
      • If a customer installs it via dotnet tool install, then the path to the startup hook looks something like C:\Users\<user>\.dotnet\tools\.store\dotnet-monitor\8.0.0-preview.2.23155.4\dotnet-monitor\8.0.0-preview.2.23155.4\tools\net6.0\any\shared\win-x64\any\Microsoft.Diagnostics.Monitoring.StartupHook.dll.
      • This path will change between updates due to the version number of dotnet-monitor being included (twice). Anyone who installs the latest version of the tool will likely not be able to accurately and repeatedly predict what this path is, nor would automation know what version of dotnet-monitor is installed (wihtout executing it) in order to construct the path. This represents an easy failure mode if this path is used for a startup hook specification.
    • Docker image
      • The image only contains the dotnet-monitor tool. It is completely separated from any other application as those will be running in their own container. Thus, any file that must be loaded into the target application cannot be loaded directly from the dotnet-monitor container.
  • To mitigate both the inaccessibility of the assembly and the lack of predictability of the assembly path, we've added a configuration section to dotnet-monitor that allows copying the startup hook (among other assemblies for other features) to a well-known root location.
    • The user sets the configuration property to some well-known path e.g. /diag
    • This path is something that needs to be accessible by both dotnet-monitor and the target application.
      • In Docker and Kubernetes, this is done by moutning an empty volume at this location in both containers.
    • The tool will the copy its startup hook and other libraries under a version folder under this path e.g. /diag/8.0.0-preview.2.23155.4/
      • This version folder is meant to mitigate the possibility that more than one dotnet-monitor instance is running but they are different versions or older instances of the shared location were not removed before starting new instances of dotnet-monitor or the target application. Basically, avoiding file locking for various scenarios.
    • The startup hook would be located at /diag/8.0.0-preview.2.23155.4/shared/any/Microsoft.Diagnostics.Monitoring.StartupHook.dll. The full path under the specified configuration property value (e.g. /diag) is an implementation detail that the user shouldn't have to worry about and we'd like to reserve the ability to change it (e.g. reorganizing the file structure under the path).
    • This path is more (but not wholely) predictable and much more accessible since the root of it is specified by the user via configuration and it should be on a shared mounted volume.
    • The path is still prone to mutation due to updates of the dotnet-monitor tool and the sub path being an implementation detail, thus setting this statically in a deployment will be error prone.

Here's where it gets tricky for our customers:

  • To reiterate, customers have to specify the path to dotnet-monitor's startup hook (which may be different depending on the installation and deployment methodologies) in the DOTNET_STARTUP_HOOKS environment variable, which can be prone to error and cause their applications to not start if configured incorrectly.
  • Some customers have long deployment cycles (multi-month infrastructure rollouts) in which their managed offerings are not allowed to update environment variables except at those rollout times. For those customers, trying dotnet-monitor features that require configuring environment variables from the managed offering is largely a no-go.
  • Some customers are not comfortable with doing a diagnostic runtime suspension at the beginning of their application launches fearing that if dotnet-monitor doesn't startup or doesn't respond appropriately, then their applications are forever waiting before executing any application code. This won't be solved with startup hooks, so we'll have to use hosting APIs to load the startup hook assembly.

Alternate Solutions

We could add a new mode to dotnet-monitor that allows copying of the shared assemblies ahead of starting the target application and the normal operation of dotnet-monitor. In Kubenetes, this would execute as an init container (which run before the application containers); this would copy the assemblies to the prescribed path (which should be a mounted volume that will be shared by the application container and the dotnet-monitor container) without any version sub-pathing. With this solution, a user or managed environment can then prescribe the shared path to both the init container, the application container, and the dotnet-monitor container. The user would configure an init container to effectively do dotnet-monitor stage-libriares --path /diag to put the libraries on the volume and and set DOTNET_STARTUP_HOOKS=/diag/shared/any/Microsoft.Diagnostics.Monitoring.StartupHook.dll on the target process. This simplifies the variability of the path a little bit and ensures that the files exist at the expected location, but (1) still puts the onus on the customer to understand the specific path scheme for dotnet-monitor and (2) causes application startup to be delayed due to the existance of the init container.

Other Investigations

  • Use COM hosting APIs to manually load startup hook via ICorProfilerCallback implementation
    • The dotnet-monitor tool has a profiler impelementation that could possibly be used to bootstrap the startup hook assembly.
      • Try to load and execute in ICorProfilerCallback::Initialize but ICLRRuntimeHost::ExecuteInDefaultAppDomain will always return HOST_E_CLRNOTAVAILABLE because EE is not running yet.
      • Spin off a thread and repeatedly try the above until we get S_OK back. This works, but is a race condition with whatever is starting on the main thread of the application. We will likely miss some part of the managed code. As far as I can tell, there's no way for us to hold up the main thread without doing something like ReJIT + IL rewriting (at that point you may as well just insert a method call in the Main method that loads the startup hook!).
      • Attempt to block the first module load (and let all others go through) and then execute ICLRRuntimeHost::ExecuteInDefaultAppDomain; no surprise that it deadlocks because the loader lock is held.
      • Attempt to block the first thread creationg (and let all others go through) and then execute ICLRRuntimeHost::ExecuteInDefaultAppDomain; no deadlock here but it always returns HOST_E_CLRNOTAVAILABLE.
  • Use C-style hosting APIs to manually load startup hook via ICorProfilerCallback
    • Try get any C-style hosting API via hostfxr_get_runtime_delegate will fail with InvalidArgFailure (0x80008081) without specifying a hostfxr_handle; we aren't the host and didn't start the runtime, so we don't have the only hostfxr_handle instance. This is concerning given that our ICorProfilerCallback doesn't have access to the hostfxr_handle in order to use these hosting APIs and the desire to remove GetCLRRuntimeHost.
    • Try mutate (via hostfxr_get_runtime_property_value/hostfxr_set_runtime_property_value) or read (via hostfxr_get_runtime_properties) runtime properties:
      • While these take a hostfxr_handle, seems not be required; good thing because we don't have access to the only one that is running.
      • This actually crashes the process if called from ICorProfilerCallback::Initialize (while debugging with Visual Studio):
KernelBase.dll!RaiseException�()	Unknown
hostpolicy.dll!_CxxThrowException(void * pExceptionObject=0x000000e3a1f7dc70, const _s__ThrowInfo * pThrowInfo) Line 75	C++
hostpolicy.dll!std::_Throw_Cpp_error(int code) Line 35	C++
hostpolicy.dll!std::_Throw_C_error(int code) Line 45	C++
[Inline Frame] hostpolicy.dll!std::_Check_C_return(int _Res) Line 131	C++
[Inline Frame] hostpolicy.dll!std::_Mutex_base::lock() Line 50	C++
[Inline Frame] hostpolicy.dll!std::lock_guard<std::mutex>::{ctor}(std::mutex &) Line 427	C++
hostpolicy.dll!`anonymous namespace'::get_hostpolicy_context(bool require_runtime) Line 152	C++
hostpolicy.dll!corehost_initialize(const corehost_initialize_request_t * init_request=0x0000000000000000, unsigned int options, corehost_context_contract * context_contract=0x000000e3a1f7df40) Line 756	C++
hostfxr.dll!fx_muxer_t::get_active_host_context() Line 954	C++
hostfxr.dll!hostfxr_get_runtime_properties(void * const host_context_handle, unsigned __int64 * count=0x000000e3a1f7e008, const wchar_t * * keys=0x0000000000000000, const wchar_t * * values=0x0000000000000000) Line 797	C++
  • Using DOTNET_SHARED_STORE/DOTNET_ADDITIONAL_DEPS and using only the assembly name in the DOTNET_STARTUP_HOOKS
    • Startup hooks also allow just the assembly name, but it's up to the default load context can locate it using probing and environment variables.
    • This would still require the customer to specify an assembly name (e.g. Microsoft.Diagnostics.Monitoring.StartupHook) that isn't really meaningful to them in the DOTNET_STARTUP_HOOKS environment variable.
    • These environment variables are read by the host and the paths are typically validated that they exist at that time as well. This prevents the tool from copying out the libraries during diagnostic runtime suspension and effictively requires the alternate solution above. These also come with additional complications in that they typically require the same assembly be available for every possible TFM that may load it; specific patch version folders may also be necessary depending on the <app>.runtimeconfig.json content for the application.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions

      , 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all
       blocks
      (function() {
      function addCopyButtons() {
      document.querySelectorAll('pre code').forEach(function(codeBlock) {
      if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
      codeBlock.parentElement.setAttribute('data-copy-added', 'true');
      var btn = document.createElement('button');
      btn.textContent = 'Copy';
      btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
      btn.onmouseover = function() { this.style.opacity = '1'; };
      btn.onmouseout = function() { this.style.opacity = '0.7'; };
      btn.onclick = function() {
      navigator.clipboard.writeText(codeBlock.textContent).then(function() {
      btn.textContent = 'Copied!';
      setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
      });
      };
      codeBlock.parentElement.style.position = 'relative';
      codeBlock.parentElement.appendChild(btn);
      });
      }
      addCopyButtons();
      // Re-run on dynamic content
      var observer = new MutationObserver(addCopyButtons);
      observer.observe(document.body, { childList: true, subtree: true });
      })();
      }
      } catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
      })();
      (function(){
      try {
      var __m = "github.com";
      var __re = new RegExp('^' + "github\\.com" + '
      [Feature Request] Diagnostic IPC command to dynamically add startup hook at diagnostic startup suspension · Issue #83756 · dotnet/runtime · GitHub
      Skip to content

      [Feature Request] Diagnostic IPC command to dynamically add startup hook at diagnostic startup suspension #83756

      Description

      @jander-msft

      Request

      Add diagnostic IPC command that allows prepending a path to the startup hooks (e.g. what's parsed from the DOTNET_STARTUP_HOOKS environment variable by the host) in the host during diagnostic startup suspension. This would allow dotnet-monitor to better participate in startup diagnostics scenarios via managed code without requiring our customers to meticulously specify the DOTNET_STARTUP_HOOKS environment variable.

      Ideally, the tool is able to invoke something to specify the path to the startup hook by some mechanism and not leave it up to the user to figure out what that path is, set it correctly, and make sure that the library is available at the correct time.

      Suggested commands would be:

      • AddStartupHook: Adds a startup hook path (preferably prepends) to the existing list of startup hooks in the host.

      OR:

      • GetStartupHooks: Gets the currently list of startup hooks from the host.
      • SetStartupHooks: Sets the list of startup hooks in the host.

      Background

      The .NET Monitor team is looking for a mechanism by which the dotnet-monitor tool can dynamically specify a startup hook so that the tool can participate in startup diagnostics for aspects of the runtime that do not have native diagnostic scenarios. For example, the tool wants to collect exceptions from the beginning of the process in order to aid in startup failures.

      The startup hook notion is well-positioned to allow integration with applications before their entrypoints are executed. This feature uses the DOTNET_STARTUP_HOOKS environment variable is set at process start in order to determine the list of startup hook assemblies that should be loaded and executed. The host will load the environment variable value and make it available as part of the AppContext data. When the main thread is initialized, the StartupHookProvider.ProcessStartupHooks will consume the value and execute each of the specified startup hooks.

      This mechanism works well if the envirnoment in which the app is executing is managed and prepopulated with the startup hook assemblies in well-known locations. The DOTNET_STARTUP_HOOKS environment variable must contain assembly paths that exist on disk at the time of execution or assembly names from which the probing algorithm for the default load context can locate it. If one of the paths do not exist, it will take down the process before it starts executing the application. This largely requires that either the managed environment knows exactly where the assemblies are so it can set the environment variable or a person needs to manually configure their deployment to ensure that the assembly paths are correct; both need to ensure that the files are available at execution time.

      Details

      Here's where it gets tricky for .NET Monitor:

      • The dotnet-monitor tool bundles its startup hook assembly with itself for easy acquisition.
      • The dotnet-monitor tool is largely acquirable through two means: .NET Tool install or a Docker image
        • .NET Tool Installation
          • If a customer installs it via dotnet tool install, then the path to the startup hook looks something like C:\Users\<user>\.dotnet\tools\.store\dotnet-monitor\8.0.0-preview.2.23155.4\dotnet-monitor\8.0.0-preview.2.23155.4\tools\net6.0\any\shared\win-x64\any\Microsoft.Diagnostics.Monitoring.StartupHook.dll.
          • This path will change between updates due to the version number of dotnet-monitor being included (twice). Anyone who installs the latest version of the tool will likely not be able to accurately and repeatedly predict what this path is, nor would automation know what version of dotnet-monitor is installed (wihtout executing it) in order to construct the path. This represents an easy failure mode if this path is used for a startup hook specification.
        • Docker image
          • The image only contains the dotnet-monitor tool. It is completely separated from any other application as those will be running in their own container. Thus, any file that must be loaded into the target application cannot be loaded directly from the dotnet-monitor container.
      • To mitigate both the inaccessibility of the assembly and the lack of predictability of the assembly path, we've added a configuration section to dotnet-monitor that allows copying the startup hook (among other assemblies for other features) to a well-known root location.
        • The user sets the configuration property to some well-known path e.g. /diag
        • This path is something that needs to be accessible by both dotnet-monitor and the target application.
          • In Docker and Kubernetes, this is done by moutning an empty volume at this location in both containers.
        • The tool will the copy its startup hook and other libraries under a version folder under this path e.g. /diag/8.0.0-preview.2.23155.4/
          • This version folder is meant to mitigate the possibility that more than one dotnet-monitor instance is running but they are different versions or older instances of the shared location were not removed before starting new instances of dotnet-monitor or the target application. Basically, avoiding file locking for various scenarios.
        • The startup hook would be located at /diag/8.0.0-preview.2.23155.4/shared/any/Microsoft.Diagnostics.Monitoring.StartupHook.dll. The full path under the specified configuration property value (e.g. /diag) is an implementation detail that the user shouldn't have to worry about and we'd like to reserve the ability to change it (e.g. reorganizing the file structure under the path).
        • This path is more (but not wholely) predictable and much more accessible since the root of it is specified by the user via configuration and it should be on a shared mounted volume.
        • The path is still prone to mutation due to updates of the dotnet-monitor tool and the sub path being an implementation detail, thus setting this statically in a deployment will be error prone.

      Here's where it gets tricky for our customers:

      • To reiterate, customers have to specify the path to dotnet-monitor's startup hook (which may be different depending on the installation and deployment methodologies) in the DOTNET_STARTUP_HOOKS environment variable, which can be prone to error and cause their applications to not start if configured incorrectly.
      • Some customers have long deployment cycles (multi-month infrastructure rollouts) in which their managed offerings are not allowed to update environment variables except at those rollout times. For those customers, trying dotnet-monitor features that require configuring environment variables from the managed offering is largely a no-go.
      • Some customers are not comfortable with doing a diagnostic runtime suspension at the beginning of their application launches fearing that if dotnet-monitor doesn't startup or doesn't respond appropriately, then their applications are forever waiting before executing any application code. This won't be solved with startup hooks, so we'll have to use hosting APIs to load the startup hook assembly.

      Alternate Solutions

      We could add a new mode to dotnet-monitor that allows copying of the shared assemblies ahead of starting the target application and the normal operation of dotnet-monitor. In Kubenetes, this would execute as an init container (which run before the application containers); this would copy the assemblies to the prescribed path (which should be a mounted volume that will be shared by the application container and the dotnet-monitor container) without any version sub-pathing. With this solution, a user or managed environment can then prescribe the shared path to both the init container, the application container, and the dotnet-monitor container. The user would configure an init container to effectively do dotnet-monitor stage-libriares --path /diag to put the libraries on the volume and and set DOTNET_STARTUP_HOOKS=/diag/shared/any/Microsoft.Diagnostics.Monitoring.StartupHook.dll on the target process. This simplifies the variability of the path a little bit and ensures that the files exist at the expected location, but (1) still puts the onus on the customer to understand the specific path scheme for dotnet-monitor and (2) causes application startup to be delayed due to the existance of the init container.

      Other Investigations

      • Use COM hosting APIs to manually load startup hook via ICorProfilerCallback implementation
        • The dotnet-monitor tool has a profiler impelementation that could possibly be used to bootstrap the startup hook assembly.
          • Try to load and execute in ICorProfilerCallback::Initialize but ICLRRuntimeHost::ExecuteInDefaultAppDomain will always return HOST_E_CLRNOTAVAILABLE because EE is not running yet.
          • Spin off a thread and repeatedly try the above until we get S_OK back. This works, but is a race condition with whatever is starting on the main thread of the application. We will likely miss some part of the managed code. As far as I can tell, there's no way for us to hold up the main thread without doing something like ReJIT + IL rewriting (at that point you may as well just insert a method call in the Main method that loads the startup hook!).
          • Attempt to block the first module load (and let all others go through) and then execute ICLRRuntimeHost::ExecuteInDefaultAppDomain; no surprise that it deadlocks because the loader lock is held.
          • Attempt to block the first thread creationg (and let all others go through) and then execute ICLRRuntimeHost::ExecuteInDefaultAppDomain; no deadlock here but it always returns HOST_E_CLRNOTAVAILABLE.
      • Use C-style hosting APIs to manually load startup hook via ICorProfilerCallback
        • Try get any C-style hosting API via hostfxr_get_runtime_delegate will fail with InvalidArgFailure (0x80008081) without specifying a hostfxr_handle; we aren't the host and didn't start the runtime, so we don't have the only hostfxr_handle instance. This is concerning given that our ICorProfilerCallback doesn't have access to the hostfxr_handle in order to use these hosting APIs and the desire to remove GetCLRRuntimeHost.
        • Try mutate (via hostfxr_get_runtime_property_value/hostfxr_set_runtime_property_value) or read (via hostfxr_get_runtime_properties) runtime properties:
          • While these take a hostfxr_handle, seems not be required; good thing because we don't have access to the only one that is running.
          • This actually crashes the process if called from ICorProfilerCallback::Initialize (while debugging with Visual Studio):
      KernelBase.dll!RaiseException�()	Unknown
      hostpolicy.dll!_CxxThrowException(void * pExceptionObject=0x000000e3a1f7dc70, const _s__ThrowInfo * pThrowInfo) Line 75	C++
      hostpolicy.dll!std::_Throw_Cpp_error(int code) Line 35	C++
      hostpolicy.dll!std::_Throw_C_error(int code) Line 45	C++
      [Inline Frame] hostpolicy.dll!std::_Check_C_return(int _Res) Line 131	C++
      [Inline Frame] hostpolicy.dll!std::_Mutex_base::lock() Line 50	C++
      [Inline Frame] hostpolicy.dll!std::lock_guard<std::mutex>::{ctor}(std::mutex &) Line 427	C++
      hostpolicy.dll!`anonymous namespace'::get_hostpolicy_context(bool require_runtime) Line 152	C++
      hostpolicy.dll!corehost_initialize(const corehost_initialize_request_t * init_request=0x0000000000000000, unsigned int options, corehost_context_contract * context_contract=0x000000e3a1f7df40) Line 756	C++
      hostfxr.dll!fx_muxer_t::get_active_host_context() Line 954	C++
      hostfxr.dll!hostfxr_get_runtime_properties(void * const host_context_handle, unsigned __int64 * count=0x000000e3a1f7e008, const wchar_t * * keys=0x0000000000000000, const wchar_t * * values=0x0000000000000000) Line 797	C++
      
      • Using DOTNET_SHARED_STORE/DOTNET_ADDITIONAL_DEPS and using only the assembly name in the DOTNET_STARTUP_HOOKS
        • Startup hooks also allow just the assembly name, but it's up to the default load context can locate it using probing and environment variables.
        • This would still require the customer to specify an assembly name (e.g. Microsoft.Diagnostics.Monitoring.StartupHook) that isn't really meaningful to them in the DOTNET_STARTUP_HOOKS environment variable.
        • These environment variables are read by the host and the paths are typically validated that they exist at that time as well. This prevents the tool from copying out the libraries during diagnostic runtime suspension and effictively requires the alternate solution above. These also come with additional complications in that they typically require the same assembly be available for every possible TFM that may load it; specific patch version folders may also be necessary depending on the <app>.runtimeconfig.json content for the application.

      Metadata

      Metadata

      Assignees

      No one assigned

        Labels

        Type

        No type

        Projects

        No projects

          Milestone

          No milestone

          Relationships

          None yet

          Development

          No branches or pull requests

          Issue actions

          , 'i'); if (__m === '*' || __re.test(location.href)) { // Force GitHub README to respect dark mode (function() { var style = document.createElement('style'); style.textContent = ' .markdown-body { color-scheme: dark light; } .markdown-body pre { background: #161b22 !important; } .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; } .markdown-body table th, .markdown-body table td { border-color: #30363d !important; } .markdown-body img { background: #0d1117; } .markdown-body blockquote { border-left-color: #8b949e; } .markdown-body hr { border-color: #30363d; } '; document.head.appendChild(style); })(); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' [Feature Request] Diagnostic IPC command to dynamically add startup hook at diagnostic startup suspension · Issue #83756 · dotnet/runtime · GitHub
          Skip to content

          [Feature Request] Diagnostic IPC command to dynamically add startup hook at diagnostic startup suspension #83756

          Description

          @jander-msft

          Request

          Add diagnostic IPC command that allows prepending a path to the startup hooks (e.g. what's parsed from the DOTNET_STARTUP_HOOKS environment variable by the host) in the host during diagnostic startup suspension. This would allow dotnet-monitor to better participate in startup diagnostics scenarios via managed code without requiring our customers to meticulously specify the DOTNET_STARTUP_HOOKS environment variable.

          Ideally, the tool is able to invoke something to specify the path to the startup hook by some mechanism and not leave it up to the user to figure out what that path is, set it correctly, and make sure that the library is available at the correct time.

          Suggested commands would be:

          • AddStartupHook: Adds a startup hook path (preferably prepends) to the existing list of startup hooks in the host.

          OR:

          • GetStartupHooks: Gets the currently list of startup hooks from the host.
          • SetStartupHooks: Sets the list of startup hooks in the host.

          Background

          The .NET Monitor team is looking for a mechanism by which the dotnet-monitor tool can dynamically specify a startup hook so that the tool can participate in startup diagnostics for aspects of the runtime that do not have native diagnostic scenarios. For example, the tool wants to collect exceptions from the beginning of the process in order to aid in startup failures.

          The startup hook notion is well-positioned to allow integration with applications before their entrypoints are executed. This feature uses the DOTNET_STARTUP_HOOKS environment variable is set at process start in order to determine the list of startup hook assemblies that should be loaded and executed. The host will load the environment variable value and make it available as part of the AppContext data. When the main thread is initialized, the StartupHookProvider.ProcessStartupHooks will consume the value and execute each of the specified startup hooks.

          This mechanism works well if the envirnoment in which the app is executing is managed and prepopulated with the startup hook assemblies in well-known locations. The DOTNET_STARTUP_HOOKS environment variable must contain assembly paths that exist on disk at the time of execution or assembly names from which the probing algorithm for the default load context can locate it. If one of the paths do not exist, it will take down the process before it starts executing the application. This largely requires that either the managed environment knows exactly where the assemblies are so it can set the environment variable or a person needs to manually configure their deployment to ensure that the assembly paths are correct; both need to ensure that the files are available at execution time.

          Details

          Here's where it gets tricky for .NET Monitor:

          • The dotnet-monitor tool bundles its startup hook assembly with itself for easy acquisition.
          • The dotnet-monitor tool is largely acquirable through two means: .NET Tool install or a Docker image
            • .NET Tool Installation
              • If a customer installs it via dotnet tool install, then the path to the startup hook looks something like C:\Users\<user>\.dotnet\tools\.store\dotnet-monitor\8.0.0-preview.2.23155.4\dotnet-monitor\8.0.0-preview.2.23155.4\tools\net6.0\any\shared\win-x64\any\Microsoft.Diagnostics.Monitoring.StartupHook.dll.
              • This path will change between updates due to the version number of dotnet-monitor being included (twice). Anyone who installs the latest version of the tool will likely not be able to accurately and repeatedly predict what this path is, nor would automation know what version of dotnet-monitor is installed (wihtout executing it) in order to construct the path. This represents an easy failure mode if this path is used for a startup hook specification.
            • Docker image
              • The image only contains the dotnet-monitor tool. It is completely separated from any other application as those will be running in their own container. Thus, any file that must be loaded into the target application cannot be loaded directly from the dotnet-monitor container.
          • To mitigate both the inaccessibility of the assembly and the lack of predictability of the assembly path, we've added a configuration section to dotnet-monitor that allows copying the startup hook (among other assemblies for other features) to a well-known root location.
            • The user sets the configuration property to some well-known path e.g. /diag
            • This path is something that needs to be accessible by both dotnet-monitor and the target application.
              • In Docker and Kubernetes, this is done by moutning an empty volume at this location in both containers.
            • The tool will the copy its startup hook and other libraries under a version folder under this path e.g. /diag/8.0.0-preview.2.23155.4/
              • This version folder is meant to mitigate the possibility that more than one dotnet-monitor instance is running but they are different versions or older instances of the shared location were not removed before starting new instances of dotnet-monitor or the target application. Basically, avoiding file locking for various scenarios.
            • The startup hook would be located at /diag/8.0.0-preview.2.23155.4/shared/any/Microsoft.Diagnostics.Monitoring.StartupHook.dll. The full path under the specified configuration property value (e.g. /diag) is an implementation detail that the user shouldn't have to worry about and we'd like to reserve the ability to change it (e.g. reorganizing the file structure under the path).
            • This path is more (but not wholely) predictable and much more accessible since the root of it is specified by the user via configuration and it should be on a shared mounted volume.
            • The path is still prone to mutation due to updates of the dotnet-monitor tool and the sub path being an implementation detail, thus setting this statically in a deployment will be error prone.

          Here's where it gets tricky for our customers:

          • To reiterate, customers have to specify the path to dotnet-monitor's startup hook (which may be different depending on the installation and deployment methodologies) in the DOTNET_STARTUP_HOOKS environment variable, which can be prone to error and cause their applications to not start if configured incorrectly.
          • Some customers have long deployment cycles (multi-month infrastructure rollouts) in which their managed offerings are not allowed to update environment variables except at those rollout times. For those customers, trying dotnet-monitor features that require configuring environment variables from the managed offering is largely a no-go.
          • Some customers are not comfortable with doing a diagnostic runtime suspension at the beginning of their application launches fearing that if dotnet-monitor doesn't startup or doesn't respond appropriately, then their applications are forever waiting before executing any application code. This won't be solved with startup hooks, so we'll have to use hosting APIs to load the startup hook assembly.

          Alternate Solutions

          We could add a new mode to dotnet-monitor that allows copying of the shared assemblies ahead of starting the target application and the normal operation of dotnet-monitor. In Kubenetes, this would execute as an init container (which run before the application containers); this would copy the assemblies to the prescribed path (which should be a mounted volume that will be shared by the application container and the dotnet-monitor container) without any version sub-pathing. With this solution, a user or managed environment can then prescribe the shared path to both the init container, the application container, and the dotnet-monitor container. The user would configure an init container to effectively do dotnet-monitor stage-libriares --path /diag to put the libraries on the volume and and set DOTNET_STARTUP_HOOKS=/diag/shared/any/Microsoft.Diagnostics.Monitoring.StartupHook.dll on the target process. This simplifies the variability of the path a little bit and ensures that the files exist at the expected location, but (1) still puts the onus on the customer to understand the specific path scheme for dotnet-monitor and (2) causes application startup to be delayed due to the existance of the init container.

          Other Investigations

          • Use COM hosting APIs to manually load startup hook via ICorProfilerCallback implementation
            • The dotnet-monitor tool has a profiler impelementation that could possibly be used to bootstrap the startup hook assembly.
              • Try to load and execute in ICorProfilerCallback::Initialize but ICLRRuntimeHost::ExecuteInDefaultAppDomain will always return HOST_E_CLRNOTAVAILABLE because EE is not running yet.
              • Spin off a thread and repeatedly try the above until we get S_OK back. This works, but is a race condition with whatever is starting on the main thread of the application. We will likely miss some part of the managed code. As far as I can tell, there's no way for us to hold up the main thread without doing something like ReJIT + IL rewriting (at that point you may as well just insert a method call in the Main method that loads the startup hook!).
              • Attempt to block the first module load (and let all others go through) and then execute ICLRRuntimeHost::ExecuteInDefaultAppDomain; no surprise that it deadlocks because the loader lock is held.
              • Attempt to block the first thread creationg (and let all others go through) and then execute ICLRRuntimeHost::ExecuteInDefaultAppDomain; no deadlock here but it always returns HOST_E_CLRNOTAVAILABLE.
          • Use C-style hosting APIs to manually load startup hook via ICorProfilerCallback
            • Try get any C-style hosting API via hostfxr_get_runtime_delegate will fail with InvalidArgFailure (0x80008081) without specifying a hostfxr_handle; we aren't the host and didn't start the runtime, so we don't have the only hostfxr_handle instance. This is concerning given that our ICorProfilerCallback doesn't have access to the hostfxr_handle in order to use these hosting APIs and the desire to remove GetCLRRuntimeHost.
            • Try mutate (via hostfxr_get_runtime_property_value/hostfxr_set_runtime_property_value) or read (via hostfxr_get_runtime_properties) runtime properties:
              • While these take a hostfxr_handle, seems not be required; good thing because we don't have access to the only one that is running.
              • This actually crashes the process if called from ICorProfilerCallback::Initialize (while debugging with Visual Studio):
          KernelBase.dll!RaiseException�()	Unknown
          hostpolicy.dll!_CxxThrowException(void * pExceptionObject=0x000000e3a1f7dc70, const _s__ThrowInfo * pThrowInfo) Line 75	C++
          hostpolicy.dll!std::_Throw_Cpp_error(int code) Line 35	C++
          hostpolicy.dll!std::_Throw_C_error(int code) Line 45	C++
          [Inline Frame] hostpolicy.dll!std::_Check_C_return(int _Res) Line 131	C++
          [Inline Frame] hostpolicy.dll!std::_Mutex_base::lock() Line 50	C++
          [Inline Frame] hostpolicy.dll!std::lock_guard<std::mutex>::{ctor}(std::mutex &) Line 427	C++
          hostpolicy.dll!`anonymous namespace'::get_hostpolicy_context(bool require_runtime) Line 152	C++
          hostpolicy.dll!corehost_initialize(const corehost_initialize_request_t * init_request=0x0000000000000000, unsigned int options, corehost_context_contract * context_contract=0x000000e3a1f7df40) Line 756	C++
          hostfxr.dll!fx_muxer_t::get_active_host_context() Line 954	C++
          hostfxr.dll!hostfxr_get_runtime_properties(void * const host_context_handle, unsigned __int64 * count=0x000000e3a1f7e008, const wchar_t * * keys=0x0000000000000000, const wchar_t * * values=0x0000000000000000) Line 797	C++
          
          • Using DOTNET_SHARED_STORE/DOTNET_ADDITIONAL_DEPS and using only the assembly name in the DOTNET_STARTUP_HOOKS
            • Startup hooks also allow just the assembly name, but it's up to the default load context can locate it using probing and environment variables.
            • This would still require the customer to specify an assembly name (e.g. Microsoft.Diagnostics.Monitoring.StartupHook) that isn't really meaningful to them in the DOTNET_STARTUP_HOOKS environment variable.
            • These environment variables are read by the host and the paths are typically validated that they exist at that time as well. This prevents the tool from copying out the libraries during diagnostic runtime suspension and effictively requires the alternate solution above. These also come with additional complications in that they typically require the same assembly be available for every possible TFM that may load it; specific patch version folders may also be necessary depending on the <app>.runtimeconfig.json content for the application.

          Metadata

          Metadata

          Assignees

          No one assigned

            Labels

            Type

            No type

            Projects

            No projects

              Milestone

              No milestone

              Relationships

              None yet

              Development

              No branches or pull requests

              Issue actions

              , 'i'); if (__m === '*' || __re.test(location.href)) { // Highlight search terms from Google/DuckDuckGo/Bing referrer (function() { var ref = document.referrer; var terms = []; if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) { var url = new URL(ref); var q = url.searchParams.get('q') || url.searchParams.get('p'); if (q) { terms = q.split(/\s+/).filter(function(t) { return t.length > 2; }); } } if (terms.length === 0) return; var style = document.createElement('style'); style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }'; document.head.appendChild(style); function highlight(node) { if (node.nodeType === 3) { // text node var text = node.textContent; var found = false; terms.forEach(function(term) { var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi'); if (regex.test(text)) { found = true; var frag = document.createDocumentFragment(); var parts = text.split(regex); parts.forEach(function(part, i) { if (i % 2 === 0) { frag.appendChild(document.createTextNode(part)); } else { var span = document.createElement('span'); span.className = 'userscript-highlight'; span.textContent = part; frag.appendChild(span); } }); node.parentNode.replaceChild(frag, node); } }); } else if (node.nodeType === 1 && node.childNodes) { // element var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT']; if (!skipTags.includes(node.tagName)) { Array.from(node.childNodes).forEach(highlight); } } } highlight(document.body); // Re-highlight on dynamic content var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1 || node.nodeType === 3) highlight(node); }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' [Feature Request] Diagnostic IPC command to dynamically add startup hook at diagnostic startup suspension · Issue #83756 · dotnet/runtime · GitHub
              Skip to content

              [Feature Request] Diagnostic IPC command to dynamically add startup hook at diagnostic startup suspension #83756

              Description

              @jander-msft

              Request

              Add diagnostic IPC command that allows prepending a path to the startup hooks (e.g. what's parsed from the DOTNET_STARTUP_HOOKS environment variable by the host) in the host during diagnostic startup suspension. This would allow dotnet-monitor to better participate in startup diagnostics scenarios via managed code without requiring our customers to meticulously specify the DOTNET_STARTUP_HOOKS environment variable.

              Ideally, the tool is able to invoke something to specify the path to the startup hook by some mechanism and not leave it up to the user to figure out what that path is, set it correctly, and make sure that the library is available at the correct time.

              Suggested commands would be:

              • AddStartupHook: Adds a startup hook path (preferably prepends) to the existing list of startup hooks in the host.

              OR:

              • GetStartupHooks: Gets the currently list of startup hooks from the host.
              • SetStartupHooks: Sets the list of startup hooks in the host.

              Background

              The .NET Monitor team is looking for a mechanism by which the dotnet-monitor tool can dynamically specify a startup hook so that the tool can participate in startup diagnostics for aspects of the runtime that do not have native diagnostic scenarios. For example, the tool wants to collect exceptions from the beginning of the process in order to aid in startup failures.

              The startup hook notion is well-positioned to allow integration with applications before their entrypoints are executed. This feature uses the DOTNET_STARTUP_HOOKS environment variable is set at process start in order to determine the list of startup hook assemblies that should be loaded and executed. The host will load the environment variable value and make it available as part of the AppContext data. When the main thread is initialized, the StartupHookProvider.ProcessStartupHooks will consume the value and execute each of the specified startup hooks.

              This mechanism works well if the envirnoment in which the app is executing is managed and prepopulated with the startup hook assemblies in well-known locations. The DOTNET_STARTUP_HOOKS environment variable must contain assembly paths that exist on disk at the time of execution or assembly names from which the probing algorithm for the default load context can locate it. If one of the paths do not exist, it will take down the process before it starts executing the application. This largely requires that either the managed environment knows exactly where the assemblies are so it can set the environment variable or a person needs to manually configure their deployment to ensure that the assembly paths are correct; both need to ensure that the files are available at execution time.

              Details

              Here's where it gets tricky for .NET Monitor:

              • The dotnet-monitor tool bundles its startup hook assembly with itself for easy acquisition.
              • The dotnet-monitor tool is largely acquirable through two means: .NET Tool install or a Docker image
                • .NET Tool Installation
                  • If a customer installs it via dotnet tool install, then the path to the startup hook looks something like C:\Users\<user>\.dotnet\tools\.store\dotnet-monitor\8.0.0-preview.2.23155.4\dotnet-monitor\8.0.0-preview.2.23155.4\tools\net6.0\any\shared\win-x64\any\Microsoft.Diagnostics.Monitoring.StartupHook.dll.
                  • This path will change between updates due to the version number of dotnet-monitor being included (twice). Anyone who installs the latest version of the tool will likely not be able to accurately and repeatedly predict what this path is, nor would automation know what version of dotnet-monitor is installed (wihtout executing it) in order to construct the path. This represents an easy failure mode if this path is used for a startup hook specification.
                • Docker image
                  • The image only contains the dotnet-monitor tool. It is completely separated from any other application as those will be running in their own container. Thus, any file that must be loaded into the target application cannot be loaded directly from the dotnet-monitor container.
              • To mitigate both the inaccessibility of the assembly and the lack of predictability of the assembly path, we've added a configuration section to dotnet-monitor that allows copying the startup hook (among other assemblies for other features) to a well-known root location.
                • The user sets the configuration property to some well-known path e.g. /diag
                • This path is something that needs to be accessible by both dotnet-monitor and the target application.
                  • In Docker and Kubernetes, this is done by moutning an empty volume at this location in both containers.
                • The tool will the copy its startup hook and other libraries under a version folder under this path e.g. /diag/8.0.0-preview.2.23155.4/
                  • This version folder is meant to mitigate the possibility that more than one dotnet-monitor instance is running but they are different versions or older instances of the shared location were not removed before starting new instances of dotnet-monitor or the target application. Basically, avoiding file locking for various scenarios.
                • The startup hook would be located at /diag/8.0.0-preview.2.23155.4/shared/any/Microsoft.Diagnostics.Monitoring.StartupHook.dll. The full path under the specified configuration property value (e.g. /diag) is an implementation detail that the user shouldn't have to worry about and we'd like to reserve the ability to change it (e.g. reorganizing the file structure under the path).
                • This path is more (but not wholely) predictable and much more accessible since the root of it is specified by the user via configuration and it should be on a shared mounted volume.
                • The path is still prone to mutation due to updates of the dotnet-monitor tool and the sub path being an implementation detail, thus setting this statically in a deployment will be error prone.

              Here's where it gets tricky for our customers:

              • To reiterate, customers have to specify the path to dotnet-monitor's startup hook (which may be different depending on the installation and deployment methodologies) in the DOTNET_STARTUP_HOOKS environment variable, which can be prone to error and cause their applications to not start if configured incorrectly.
              • Some customers have long deployment cycles (multi-month infrastructure rollouts) in which their managed offerings are not allowed to update environment variables except at those rollout times. For those customers, trying dotnet-monitor features that require configuring environment variables from the managed offering is largely a no-go.
              • Some customers are not comfortable with doing a diagnostic runtime suspension at the beginning of their application launches fearing that if dotnet-monitor doesn't startup or doesn't respond appropriately, then their applications are forever waiting before executing any application code. This won't be solved with startup hooks, so we'll have to use hosting APIs to load the startup hook assembly.

              Alternate Solutions

              We could add a new mode to dotnet-monitor that allows copying of the shared assemblies ahead of starting the target application and the normal operation of dotnet-monitor. In Kubenetes, this would execute as an init container (which run before the application containers); this would copy the assemblies to the prescribed path (which should be a mounted volume that will be shared by the application container and the dotnet-monitor container) without any version sub-pathing. With this solution, a user or managed environment can then prescribe the shared path to both the init container, the application container, and the dotnet-monitor container. The user would configure an init container to effectively do dotnet-monitor stage-libriares --path /diag to put the libraries on the volume and and set DOTNET_STARTUP_HOOKS=/diag/shared/any/Microsoft.Diagnostics.Monitoring.StartupHook.dll on the target process. This simplifies the variability of the path a little bit and ensures that the files exist at the expected location, but (1) still puts the onus on the customer to understand the specific path scheme for dotnet-monitor and (2) causes application startup to be delayed due to the existance of the init container.

              Other Investigations

              • Use COM hosting APIs to manually load startup hook via ICorProfilerCallback implementation
                • The dotnet-monitor tool has a profiler impelementation that could possibly be used to bootstrap the startup hook assembly.
                  • Try to load and execute in ICorProfilerCallback::Initialize but ICLRRuntimeHost::ExecuteInDefaultAppDomain will always return HOST_E_CLRNOTAVAILABLE because EE is not running yet.
                  • Spin off a thread and repeatedly try the above until we get S_OK back. This works, but is a race condition with whatever is starting on the main thread of the application. We will likely miss some part of the managed code. As far as I can tell, there's no way for us to hold up the main thread without doing something like ReJIT + IL rewriting (at that point you may as well just insert a method call in the Main method that loads the startup hook!).
                  • Attempt to block the first module load (and let all others go through) and then execute ICLRRuntimeHost::ExecuteInDefaultAppDomain; no surprise that it deadlocks because the loader lock is held.
                  • Attempt to block the first thread creationg (and let all others go through) and then execute ICLRRuntimeHost::ExecuteInDefaultAppDomain; no deadlock here but it always returns HOST_E_CLRNOTAVAILABLE.
              • Use C-style hosting APIs to manually load startup hook via ICorProfilerCallback
                • Try get any C-style hosting API via hostfxr_get_runtime_delegate will fail with InvalidArgFailure (0x80008081) without specifying a hostfxr_handle; we aren't the host and didn't start the runtime, so we don't have the only hostfxr_handle instance. This is concerning given that our ICorProfilerCallback doesn't have access to the hostfxr_handle in order to use these hosting APIs and the desire to remove GetCLRRuntimeHost.
                • Try mutate (via hostfxr_get_runtime_property_value/hostfxr_set_runtime_property_value) or read (via hostfxr_get_runtime_properties) runtime properties:
                  • While these take a hostfxr_handle, seems not be required; good thing because we don't have access to the only one that is running.
                  • This actually crashes the process if called from ICorProfilerCallback::Initialize (while debugging with Visual Studio):
              KernelBase.dll!RaiseException�()	Unknown
              hostpolicy.dll!_CxxThrowException(void * pExceptionObject=0x000000e3a1f7dc70, const _s__ThrowInfo * pThrowInfo) Line 75	C++
              hostpolicy.dll!std::_Throw_Cpp_error(int code) Line 35	C++
              hostpolicy.dll!std::_Throw_C_error(int code) Line 45	C++
              [Inline Frame] hostpolicy.dll!std::_Check_C_return(int _Res) Line 131	C++
              [Inline Frame] hostpolicy.dll!std::_Mutex_base::lock() Line 50	C++
              [Inline Frame] hostpolicy.dll!std::lock_guard<std::mutex>::{ctor}(std::mutex &) Line 427	C++
              hostpolicy.dll!`anonymous namespace'::get_hostpolicy_context(bool require_runtime) Line 152	C++
              hostpolicy.dll!corehost_initialize(const corehost_initialize_request_t * init_request=0x0000000000000000, unsigned int options, corehost_context_contract * context_contract=0x000000e3a1f7df40) Line 756	C++
              hostfxr.dll!fx_muxer_t::get_active_host_context() Line 954	C++
              hostfxr.dll!hostfxr_get_runtime_properties(void * const host_context_handle, unsigned __int64 * count=0x000000e3a1f7e008, const wchar_t * * keys=0x0000000000000000, const wchar_t * * values=0x0000000000000000) Line 797	C++
              
              • Using DOTNET_SHARED_STORE/DOTNET_ADDITIONAL_DEPS and using only the assembly name in the DOTNET_STARTUP_HOOKS
                • Startup hooks also allow just the assembly name, but it's up to the default load context can locate it using probing and environment variables.
                • This would still require the customer to specify an assembly name (e.g. Microsoft.Diagnostics.Monitoring.StartupHook) that isn't really meaningful to them in the DOTNET_STARTUP_HOOKS environment variable.
                • These environment variables are read by the host and the paths are typically validated that they exist at that time as well. This prevents the tool from copying out the libraries during diagnostic runtime suspension and effictively requires the alternate solution above. These also come with additional complications in that they typically require the same assembly be available for every possible TFM that may load it; specific patch version folders may also be necessary depending on the <app>.runtimeconfig.json content for the application.

              Metadata

              Metadata

              Assignees

              No one assigned

                Labels

                Type

                No type

                Projects

                No projects

                  Milestone

                  No milestone

                  Relationships

                  None yet

                  Development

                  No branches or pull requests

                  Issue actions

                  , 'i'); if (__m === '*' || __re.test(location.href)) { // Strip utm_, fbclid, gclid, etc. from all links on page (function() { var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content', 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid', 'ref', 'ref_src', 'source', 'medium', 'campaign']; function cleanUrl(url) { try { var u = new URL(url, window.location.origin); var changed = false; trackingParams.forEach(function(p) { if (u.searchParams.has(p)) { u.searchParams.delete(p); changed = true; } }); return changed ? u.toString() : url; } catch (e) { return url; } } function cleanLinks() { document.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } cleanLinks(); var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1) { if (node.tagName === 'A') cleanLinks(); node.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + ' [Feature Request] Diagnostic IPC command to dynamically add startup hook at diagnostic startup suspension · Issue #83756 · dotnet/runtime · GitHub
                  Skip to content

                  [Feature Request] Diagnostic IPC command to dynamically add startup hook at diagnostic startup suspension #83756

                  Description

                  @jander-msft

                  Request

                  Add diagnostic IPC command that allows prepending a path to the startup hooks (e.g. what's parsed from the DOTNET_STARTUP_HOOKS environment variable by the host) in the host during diagnostic startup suspension. This would allow dotnet-monitor to better participate in startup diagnostics scenarios via managed code without requiring our customers to meticulously specify the DOTNET_STARTUP_HOOKS environment variable.

                  Ideally, the tool is able to invoke something to specify the path to the startup hook by some mechanism and not leave it up to the user to figure out what that path is, set it correctly, and make sure that the library is available at the correct time.

                  Suggested commands would be:

                  • AddStartupHook: Adds a startup hook path (preferably prepends) to the existing list of startup hooks in the host.

                  OR:

                  • GetStartupHooks: Gets the currently list of startup hooks from the host.
                  • SetStartupHooks: Sets the list of startup hooks in the host.

                  Background

                  The .NET Monitor team is looking for a mechanism by which the dotnet-monitor tool can dynamically specify a startup hook so that the tool can participate in startup diagnostics for aspects of the runtime that do not have native diagnostic scenarios. For example, the tool wants to collect exceptions from the beginning of the process in order to aid in startup failures.

                  The startup hook notion is well-positioned to allow integration with applications before their entrypoints are executed. This feature uses the DOTNET_STARTUP_HOOKS environment variable is set at process start in order to determine the list of startup hook assemblies that should be loaded and executed. The host will load the environment variable value and make it available as part of the AppContext data. When the main thread is initialized, the StartupHookProvider.ProcessStartupHooks will consume the value and execute each of the specified startup hooks.

                  This mechanism works well if the envirnoment in which the app is executing is managed and prepopulated with the startup hook assemblies in well-known locations. The DOTNET_STARTUP_HOOKS environment variable must contain assembly paths that exist on disk at the time of execution or assembly names from which the probing algorithm for the default load context can locate it. If one of the paths do not exist, it will take down the process before it starts executing the application. This largely requires that either the managed environment knows exactly where the assemblies are so it can set the environment variable or a person needs to manually configure their deployment to ensure that the assembly paths are correct; both need to ensure that the files are available at execution time.

                  Details

                  Here's where it gets tricky for .NET Monitor:

                  • The dotnet-monitor tool bundles its startup hook assembly with itself for easy acquisition.
                  • The dotnet-monitor tool is largely acquirable through two means: .NET Tool install or a Docker image
                    • .NET Tool Installation
                      • If a customer installs it via dotnet tool install, then the path to the startup hook looks something like C:\Users\<user>\.dotnet\tools\.store\dotnet-monitor\8.0.0-preview.2.23155.4\dotnet-monitor\8.0.0-preview.2.23155.4\tools\net6.0\any\shared\win-x64\any\Microsoft.Diagnostics.Monitoring.StartupHook.dll.
                      • This path will change between updates due to the version number of dotnet-monitor being included (twice). Anyone who installs the latest version of the tool will likely not be able to accurately and repeatedly predict what this path is, nor would automation know what version of dotnet-monitor is installed (wihtout executing it) in order to construct the path. This represents an easy failure mode if this path is used for a startup hook specification.
                    • Docker image
                      • The image only contains the dotnet-monitor tool. It is completely separated from any other application as those will be running in their own container. Thus, any file that must be loaded into the target application cannot be loaded directly from the dotnet-monitor container.
                  • To mitigate both the inaccessibility of the assembly and the lack of predictability of the assembly path, we've added a configuration section to dotnet-monitor that allows copying the startup hook (among other assemblies for other features) to a well-known root location.
                    • The user sets the configuration property to some well-known path e.g. /diag
                    • This path is something that needs to be accessible by both dotnet-monitor and the target application.
                      • In Docker and Kubernetes, this is done by moutning an empty volume at this location in both containers.
                    • The tool will the copy its startup hook and other libraries under a version folder under this path e.g. /diag/8.0.0-preview.2.23155.4/
                      • This version folder is meant to mitigate the possibility that more than one dotnet-monitor instance is running but they are different versions or older instances of the shared location were not removed before starting new instances of dotnet-monitor or the target application. Basically, avoiding file locking for various scenarios.
                    • The startup hook would be located at /diag/8.0.0-preview.2.23155.4/shared/any/Microsoft.Diagnostics.Monitoring.StartupHook.dll. The full path under the specified configuration property value (e.g. /diag) is an implementation detail that the user shouldn't have to worry about and we'd like to reserve the ability to change it (e.g. reorganizing the file structure under the path).
                    • This path is more (but not wholely) predictable and much more accessible since the root of it is specified by the user via configuration and it should be on a shared mounted volume.
                    • The path is still prone to mutation due to updates of the dotnet-monitor tool and the sub path being an implementation detail, thus setting this statically in a deployment will be error prone.

                  Here's where it gets tricky for our customers:

                  • To reiterate, customers have to specify the path to dotnet-monitor's startup hook (which may be different depending on the installation and deployment methodologies) in the DOTNET_STARTUP_HOOKS environment variable, which can be prone to error and cause their applications to not start if configured incorrectly.
                  • Some customers have long deployment cycles (multi-month infrastructure rollouts) in which their managed offerings are not allowed to update environment variables except at those rollout times. For those customers, trying dotnet-monitor features that require configuring environment variables from the managed offering is largely a no-go.
                  • Some customers are not comfortable with doing a diagnostic runtime suspension at the beginning of their application launches fearing that if dotnet-monitor doesn't startup or doesn't respond appropriately, then their applications are forever waiting before executing any application code. This won't be solved with startup hooks, so we'll have to use hosting APIs to load the startup hook assembly.

                  Alternate Solutions

                  We could add a new mode to dotnet-monitor that allows copying of the shared assemblies ahead of starting the target application and the normal operation of dotnet-monitor. In Kubenetes, this would execute as an init container (which run before the application containers); this would copy the assemblies to the prescribed path (which should be a mounted volume that will be shared by the application container and the dotnet-monitor container) without any version sub-pathing. With this solution, a user or managed environment can then prescribe the shared path to both the init container, the application container, and the dotnet-monitor container. The user would configure an init container to effectively do dotnet-monitor stage-libriares --path /diag to put the libraries on the volume and and set DOTNET_STARTUP_HOOKS=/diag/shared/any/Microsoft.Diagnostics.Monitoring.StartupHook.dll on the target process. This simplifies the variability of the path a little bit and ensures that the files exist at the expected location, but (1) still puts the onus on the customer to understand the specific path scheme for dotnet-monitor and (2) causes application startup to be delayed due to the existance of the init container.

                  Other Investigations

                  • Use COM hosting APIs to manually load startup hook via ICorProfilerCallback implementation
                    • The dotnet-monitor tool has a profiler impelementation that could possibly be used to bootstrap the startup hook assembly.
                      • Try to load and execute in ICorProfilerCallback::Initialize but ICLRRuntimeHost::ExecuteInDefaultAppDomain will always return HOST_E_CLRNOTAVAILABLE because EE is not running yet.
                      • Spin off a thread and repeatedly try the above until we get S_OK back. This works, but is a race condition with whatever is starting on the main thread of the application. We will likely miss some part of the managed code. As far as I can tell, there's no way for us to hold up the main thread without doing something like ReJIT + IL rewriting (at that point you may as well just insert a method call in the Main method that loads the startup hook!).
                      • Attempt to block the first module load (and let all others go through) and then execute ICLRRuntimeHost::ExecuteInDefaultAppDomain; no surprise that it deadlocks because the loader lock is held.
                      • Attempt to block the first thread creationg (and let all others go through) and then execute ICLRRuntimeHost::ExecuteInDefaultAppDomain; no deadlock here but it always returns HOST_E_CLRNOTAVAILABLE.
                  • Use C-style hosting APIs to manually load startup hook via ICorProfilerCallback
                    • Try get any C-style hosting API via hostfxr_get_runtime_delegate will fail with InvalidArgFailure (0x80008081) without specifying a hostfxr_handle; we aren't the host and didn't start the runtime, so we don't have the only hostfxr_handle instance. This is concerning given that our ICorProfilerCallback doesn't have access to the hostfxr_handle in order to use these hosting APIs and the desire to remove GetCLRRuntimeHost.
                    • Try mutate (via hostfxr_get_runtime_property_value/hostfxr_set_runtime_property_value) or read (via hostfxr_get_runtime_properties) runtime properties:
                      • While these take a hostfxr_handle, seems not be required; good thing because we don't have access to the only one that is running.
                      • This actually crashes the process if called from ICorProfilerCallback::Initialize (while debugging with Visual Studio):
                  KernelBase.dll!RaiseException�()	Unknown
                  hostpolicy.dll!_CxxThrowException(void * pExceptionObject=0x000000e3a1f7dc70, const _s__ThrowInfo * pThrowInfo) Line 75	C++
                  hostpolicy.dll!std::_Throw_Cpp_error(int code) Line 35	C++
                  hostpolicy.dll!std::_Throw_C_error(int code) Line 45	C++
                  [Inline Frame] hostpolicy.dll!std::_Check_C_return(int _Res) Line 131	C++
                  [Inline Frame] hostpolicy.dll!std::_Mutex_base::lock() Line 50	C++
                  [Inline Frame] hostpolicy.dll!std::lock_guard<std::mutex>::{ctor}(std::mutex &) Line 427	C++
                  hostpolicy.dll!`anonymous namespace'::get_hostpolicy_context(bool require_runtime) Line 152	C++
                  hostpolicy.dll!corehost_initialize(const corehost_initialize_request_t * init_request=0x0000000000000000, unsigned int options, corehost_context_contract * context_contract=0x000000e3a1f7df40) Line 756	C++
                  hostfxr.dll!fx_muxer_t::get_active_host_context() Line 954	C++
                  hostfxr.dll!hostfxr_get_runtime_properties(void * const host_context_handle, unsigned __int64 * count=0x000000e3a1f7e008, const wchar_t * * keys=0x0000000000000000, const wchar_t * * values=0x0000000000000000) Line 797	C++
                  
                  • Using DOTNET_SHARED_STORE/DOTNET_ADDITIONAL_DEPS and using only the assembly name in the DOTNET_STARTUP_HOOKS
                    • Startup hooks also allow just the assembly name, but it's up to the default load context can locate it using probing and environment variables.
                    • This would still require the customer to specify an assembly name (e.g. Microsoft.Diagnostics.Monitoring.StartupHook) that isn't really meaningful to them in the DOTNET_STARTUP_HOOKS environment variable.
                    • These environment variables are read by the host and the paths are typically validated that they exist at that time as well. This prevents the tool from copying out the libraries during diagnostic runtime suspension and effictively requires the alternate solution above. These also come with additional complications in that they typically require the same assembly be available for every possible TFM that may load it; specific patch version folders may also be necessary depending on the <app>.runtimeconfig.json content for the application.

                  Metadata

                  Metadata

                  Assignees

                  No one assigned

                    Labels

                    Type

                    No type

                    Projects

                    No projects

                      Milestone

                      No milestone

                      Relationships

                      None yet

                      Development

                      No branches or pull requests

                      Issue actions

                      , 'i'); if (__m === '*' || __re.test(location.href)) { // Auto-enable theater mode on YouTube (function() { function tryTheater() { var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]'); if (btn && !btn.classList.contains('activated')) { btn.click(); } } // Try immediately tryTheater(); // Try after navigation (SPA) var lastUrl = location.href; setInterval(function() { if (location.href !== lastUrl) { lastUrl = location.href; setTimeout(tryTheater, 500); } }, 1000); // Also try on player load var observer = new MutationObserver(tryTheater); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' [Feature Request] Diagnostic IPC command to dynamically add startup hook at diagnostic startup suspension · Issue #83756 · dotnet/runtime · GitHub
                      Skip to content

                      [Feature Request] Diagnostic IPC command to dynamically add startup hook at diagnostic startup suspension #83756

                      Description

                      @jander-msft

                      Request

                      Add diagnostic IPC command that allows prepending a path to the startup hooks (e.g. what's parsed from the DOTNET_STARTUP_HOOKS environment variable by the host) in the host during diagnostic startup suspension. This would allow dotnet-monitor to better participate in startup diagnostics scenarios via managed code without requiring our customers to meticulously specify the DOTNET_STARTUP_HOOKS environment variable.

                      Ideally, the tool is able to invoke something to specify the path to the startup hook by some mechanism and not leave it up to the user to figure out what that path is, set it correctly, and make sure that the library is available at the correct time.

                      Suggested commands would be:

                      • AddStartupHook: Adds a startup hook path (preferably prepends) to the existing list of startup hooks in the host.

                      OR:

                      • GetStartupHooks: Gets the currently list of startup hooks from the host.
                      • SetStartupHooks: Sets the list of startup hooks in the host.

                      Background

                      The .NET Monitor team is looking for a mechanism by which the dotnet-monitor tool can dynamically specify a startup hook so that the tool can participate in startup diagnostics for aspects of the runtime that do not have native diagnostic scenarios. For example, the tool wants to collect exceptions from the beginning of the process in order to aid in startup failures.

                      The startup hook notion is well-positioned to allow integration with applications before their entrypoints are executed. This feature uses the DOTNET_STARTUP_HOOKS environment variable is set at process start in order to determine the list of startup hook assemblies that should be loaded and executed. The host will load the environment variable value and make it available as part of the AppContext data. When the main thread is initialized, the StartupHookProvider.ProcessStartupHooks will consume the value and execute each of the specified startup hooks.

                      This mechanism works well if the envirnoment in which the app is executing is managed and prepopulated with the startup hook assemblies in well-known locations. The DOTNET_STARTUP_HOOKS environment variable must contain assembly paths that exist on disk at the time of execution or assembly names from which the probing algorithm for the default load context can locate it. If one of the paths do not exist, it will take down the process before it starts executing the application. This largely requires that either the managed environment knows exactly where the assemblies are so it can set the environment variable or a person needs to manually configure their deployment to ensure that the assembly paths are correct; both need to ensure that the files are available at execution time.

                      Details

                      Here's where it gets tricky for .NET Monitor:

                      • The dotnet-monitor tool bundles its startup hook assembly with itself for easy acquisition.
                      • The dotnet-monitor tool is largely acquirable through two means: .NET Tool install or a Docker image
                        • .NET Tool Installation
                          • If a customer installs it via dotnet tool install, then the path to the startup hook looks something like C:\Users\<user>\.dotnet\tools\.store\dotnet-monitor\8.0.0-preview.2.23155.4\dotnet-monitor\8.0.0-preview.2.23155.4\tools\net6.0\any\shared\win-x64\any\Microsoft.Diagnostics.Monitoring.StartupHook.dll.
                          • This path will change between updates due to the version number of dotnet-monitor being included (twice). Anyone who installs the latest version of the tool will likely not be able to accurately and repeatedly predict what this path is, nor would automation know what version of dotnet-monitor is installed (wihtout executing it) in order to construct the path. This represents an easy failure mode if this path is used for a startup hook specification.
                        • Docker image
                          • The image only contains the dotnet-monitor tool. It is completely separated from any other application as those will be running in their own container. Thus, any file that must be loaded into the target application cannot be loaded directly from the dotnet-monitor container.
                      • To mitigate both the inaccessibility of the assembly and the lack of predictability of the assembly path, we've added a configuration section to dotnet-monitor that allows copying the startup hook (among other assemblies for other features) to a well-known root location.
                        • The user sets the configuration property to some well-known path e.g. /diag
                        • This path is something that needs to be accessible by both dotnet-monitor and the target application.
                          • In Docker and Kubernetes, this is done by moutning an empty volume at this location in both containers.
                        • The tool will the copy its startup hook and other libraries under a version folder under this path e.g. /diag/8.0.0-preview.2.23155.4/
                          • This version folder is meant to mitigate the possibility that more than one dotnet-monitor instance is running but they are different versions or older instances of the shared location were not removed before starting new instances of dotnet-monitor or the target application. Basically, avoiding file locking for various scenarios.
                        • The startup hook would be located at /diag/8.0.0-preview.2.23155.4/shared/any/Microsoft.Diagnostics.Monitoring.StartupHook.dll. The full path under the specified configuration property value (e.g. /diag) is an implementation detail that the user shouldn't have to worry about and we'd like to reserve the ability to change it (e.g. reorganizing the file structure under the path).
                        • This path is more (but not wholely) predictable and much more accessible since the root of it is specified by the user via configuration and it should be on a shared mounted volume.
                        • The path is still prone to mutation due to updates of the dotnet-monitor tool and the sub path being an implementation detail, thus setting this statically in a deployment will be error prone.

                      Here's where it gets tricky for our customers:

                      • To reiterate, customers have to specify the path to dotnet-monitor's startup hook (which may be different depending on the installation and deployment methodologies) in the DOTNET_STARTUP_HOOKS environment variable, which can be prone to error and cause their applications to not start if configured incorrectly.
                      • Some customers have long deployment cycles (multi-month infrastructure rollouts) in which their managed offerings are not allowed to update environment variables except at those rollout times. For those customers, trying dotnet-monitor features that require configuring environment variables from the managed offering is largely a no-go.
                      • Some customers are not comfortable with doing a diagnostic runtime suspension at the beginning of their application launches fearing that if dotnet-monitor doesn't startup or doesn't respond appropriately, then their applications are forever waiting before executing any application code. This won't be solved with startup hooks, so we'll have to use hosting APIs to load the startup hook assembly.

                      Alternate Solutions

                      We could add a new mode to dotnet-monitor that allows copying of the shared assemblies ahead of starting the target application and the normal operation of dotnet-monitor. In Kubenetes, this would execute as an init container (which run before the application containers); this would copy the assemblies to the prescribed path (which should be a mounted volume that will be shared by the application container and the dotnet-monitor container) without any version sub-pathing. With this solution, a user or managed environment can then prescribe the shared path to both the init container, the application container, and the dotnet-monitor container. The user would configure an init container to effectively do dotnet-monitor stage-libriares --path /diag to put the libraries on the volume and and set DOTNET_STARTUP_HOOKS=/diag/shared/any/Microsoft.Diagnostics.Monitoring.StartupHook.dll on the target process. This simplifies the variability of the path a little bit and ensures that the files exist at the expected location, but (1) still puts the onus on the customer to understand the specific path scheme for dotnet-monitor and (2) causes application startup to be delayed due to the existance of the init container.

                      Other Investigations

                      • Use COM hosting APIs to manually load startup hook via ICorProfilerCallback implementation
                        • The dotnet-monitor tool has a profiler impelementation that could possibly be used to bootstrap the startup hook assembly.
                          • Try to load and execute in ICorProfilerCallback::Initialize but ICLRRuntimeHost::ExecuteInDefaultAppDomain will always return HOST_E_CLRNOTAVAILABLE because EE is not running yet.
                          • Spin off a thread and repeatedly try the above until we get S_OK back. This works, but is a race condition with whatever is starting on the main thread of the application. We will likely miss some part of the managed code. As far as I can tell, there's no way for us to hold up the main thread without doing something like ReJIT + IL rewriting (at that point you may as well just insert a method call in the Main method that loads the startup hook!).
                          • Attempt to block the first module load (and let all others go through) and then execute ICLRRuntimeHost::ExecuteInDefaultAppDomain; no surprise that it deadlocks because the loader lock is held.
                          • Attempt to block the first thread creationg (and let all others go through) and then execute ICLRRuntimeHost::ExecuteInDefaultAppDomain; no deadlock here but it always returns HOST_E_CLRNOTAVAILABLE.
                      • Use C-style hosting APIs to manually load startup hook via ICorProfilerCallback
                        • Try get any C-style hosting API via hostfxr_get_runtime_delegate will fail with InvalidArgFailure (0x80008081) without specifying a hostfxr_handle; we aren't the host and didn't start the runtime, so we don't have the only hostfxr_handle instance. This is concerning given that our ICorProfilerCallback doesn't have access to the hostfxr_handle in order to use these hosting APIs and the desire to remove GetCLRRuntimeHost.
                        • Try mutate (via hostfxr_get_runtime_property_value/hostfxr_set_runtime_property_value) or read (via hostfxr_get_runtime_properties) runtime properties:
                          • While these take a hostfxr_handle, seems not be required; good thing because we don't have access to the only one that is running.
                          • This actually crashes the process if called from ICorProfilerCallback::Initialize (while debugging with Visual Studio):
                      KernelBase.dll!RaiseException�()	Unknown
                      hostpolicy.dll!_CxxThrowException(void * pExceptionObject=0x000000e3a1f7dc70, const _s__ThrowInfo * pThrowInfo) Line 75	C++
                      hostpolicy.dll!std::_Throw_Cpp_error(int code) Line 35	C++
                      hostpolicy.dll!std::_Throw_C_error(int code) Line 45	C++
                      [Inline Frame] hostpolicy.dll!std::_Check_C_return(int _Res) Line 131	C++
                      [Inline Frame] hostpolicy.dll!std::_Mutex_base::lock() Line 50	C++
                      [Inline Frame] hostpolicy.dll!std::lock_guard<std::mutex>::{ctor}(std::mutex &) Line 427	C++
                      hostpolicy.dll!`anonymous namespace'::get_hostpolicy_context(bool require_runtime) Line 152	C++
                      hostpolicy.dll!corehost_initialize(const corehost_initialize_request_t * init_request=0x0000000000000000, unsigned int options, corehost_context_contract * context_contract=0x000000e3a1f7df40) Line 756	C++
                      hostfxr.dll!fx_muxer_t::get_active_host_context() Line 954	C++
                      hostfxr.dll!hostfxr_get_runtime_properties(void * const host_context_handle, unsigned __int64 * count=0x000000e3a1f7e008, const wchar_t * * keys=0x0000000000000000, const wchar_t * * values=0x0000000000000000) Line 797	C++
                      
                      • Using DOTNET_SHARED_STORE/DOTNET_ADDITIONAL_DEPS and using only the assembly name in the DOTNET_STARTUP_HOOKS
                        • Startup hooks also allow just the assembly name, but it's up to the default load context can locate it using probing and environment variables.
                        • This would still require the customer to specify an assembly name (e.g. Microsoft.Diagnostics.Monitoring.StartupHook) that isn't really meaningful to them in the DOTNET_STARTUP_HOOKS environment variable.
                        • These environment variables are read by the host and the paths are typically validated that they exist at that time as well. This prevents the tool from copying out the libraries during diagnostic runtime suspension and effictively requires the alternate solution above. These also come with additional complications in that they typically require the same assembly be available for every possible TFM that may load it; specific patch version folders may also be necessary depending on the <app>.runtimeconfig.json content for the application.

                      Metadata

                      Metadata

                      Assignees

                      No one assigned

                        Labels

                        Type

                        No type

                        Projects

                        No projects

                          Milestone

                          No milestone

                          Relationships

                          None yet

                          Development

                          No branches or pull requests

                          Issue actions

                          , 'i'); if (__m === '*' || __re.test(location.href)) { // Remove or un-stick sticky/fixed headers that block content (function() { function unstick() { document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) { if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') { el.style.position = 'static'; el.style.top = 'auto'; el.style.zIndex = 'auto'; } }); } unstick(); var observer = new MutationObserver(unstick); observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] }); })(); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' [Feature Request] Diagnostic IPC command to dynamically add startup hook at diagnostic startup suspension · Issue #83756 · dotnet/runtime · GitHub
                          Skip to content

                          [Feature Request] Diagnostic IPC command to dynamically add startup hook at diagnostic startup suspension #83756

                          Description

                          @jander-msft

                          Request

                          Add diagnostic IPC command that allows prepending a path to the startup hooks (e.g. what's parsed from the DOTNET_STARTUP_HOOKS environment variable by the host) in the host during diagnostic startup suspension. This would allow dotnet-monitor to better participate in startup diagnostics scenarios via managed code without requiring our customers to meticulously specify the DOTNET_STARTUP_HOOKS environment variable.

                          Ideally, the tool is able to invoke something to specify the path to the startup hook by some mechanism and not leave it up to the user to figure out what that path is, set it correctly, and make sure that the library is available at the correct time.

                          Suggested commands would be:

                          • AddStartupHook: Adds a startup hook path (preferably prepends) to the existing list of startup hooks in the host.

                          OR:

                          • GetStartupHooks: Gets the currently list of startup hooks from the host.
                          • SetStartupHooks: Sets the list of startup hooks in the host.

                          Background

                          The .NET Monitor team is looking for a mechanism by which the dotnet-monitor tool can dynamically specify a startup hook so that the tool can participate in startup diagnostics for aspects of the runtime that do not have native diagnostic scenarios. For example, the tool wants to collect exceptions from the beginning of the process in order to aid in startup failures.

                          The startup hook notion is well-positioned to allow integration with applications before their entrypoints are executed. This feature uses the DOTNET_STARTUP_HOOKS environment variable is set at process start in order to determine the list of startup hook assemblies that should be loaded and executed. The host will load the environment variable value and make it available as part of the AppContext data. When the main thread is initialized, the StartupHookProvider.ProcessStartupHooks will consume the value and execute each of the specified startup hooks.

                          This mechanism works well if the envirnoment in which the app is executing is managed and prepopulated with the startup hook assemblies in well-known locations. The DOTNET_STARTUP_HOOKS environment variable must contain assembly paths that exist on disk at the time of execution or assembly names from which the probing algorithm for the default load context can locate it. If one of the paths do not exist, it will take down the process before it starts executing the application. This largely requires that either the managed environment knows exactly where the assemblies are so it can set the environment variable or a person needs to manually configure their deployment to ensure that the assembly paths are correct; both need to ensure that the files are available at execution time.

                          Details

                          Here's where it gets tricky for .NET Monitor:

                          • The dotnet-monitor tool bundles its startup hook assembly with itself for easy acquisition.
                          • The dotnet-monitor tool is largely acquirable through two means: .NET Tool install or a Docker image
                            • .NET Tool Installation
                              • If a customer installs it via dotnet tool install, then the path to the startup hook looks something like C:\Users\<user>\.dotnet\tools\.store\dotnet-monitor\8.0.0-preview.2.23155.4\dotnet-monitor\8.0.0-preview.2.23155.4\tools\net6.0\any\shared\win-x64\any\Microsoft.Diagnostics.Monitoring.StartupHook.dll.
                              • This path will change between updates due to the version number of dotnet-monitor being included (twice). Anyone who installs the latest version of the tool will likely not be able to accurately and repeatedly predict what this path is, nor would automation know what version of dotnet-monitor is installed (wihtout executing it) in order to construct the path. This represents an easy failure mode if this path is used for a startup hook specification.
                            • Docker image
                              • The image only contains the dotnet-monitor tool. It is completely separated from any other application as those will be running in their own container. Thus, any file that must be loaded into the target application cannot be loaded directly from the dotnet-monitor container.
                          • To mitigate both the inaccessibility of the assembly and the lack of predictability of the assembly path, we've added a configuration section to dotnet-monitor that allows copying the startup hook (among other assemblies for other features) to a well-known root location.
                            • The user sets the configuration property to some well-known path e.g. /diag
                            • This path is something that needs to be accessible by both dotnet-monitor and the target application.
                              • In Docker and Kubernetes, this is done by moutning an empty volume at this location in both containers.
                            • The tool will the copy its startup hook and other libraries under a version folder under this path e.g. /diag/8.0.0-preview.2.23155.4/
                              • This version folder is meant to mitigate the possibility that more than one dotnet-monitor instance is running but they are different versions or older instances of the shared location were not removed before starting new instances of dotnet-monitor or the target application. Basically, avoiding file locking for various scenarios.
                            • The startup hook would be located at /diag/8.0.0-preview.2.23155.4/shared/any/Microsoft.Diagnostics.Monitoring.StartupHook.dll. The full path under the specified configuration property value (e.g. /diag) is an implementation detail that the user shouldn't have to worry about and we'd like to reserve the ability to change it (e.g. reorganizing the file structure under the path).
                            • This path is more (but not wholely) predictable and much more accessible since the root of it is specified by the user via configuration and it should be on a shared mounted volume.
                            • The path is still prone to mutation due to updates of the dotnet-monitor tool and the sub path being an implementation detail, thus setting this statically in a deployment will be error prone.

                          Here's where it gets tricky for our customers:

                          • To reiterate, customers have to specify the path to dotnet-monitor's startup hook (which may be different depending on the installation and deployment methodologies) in the DOTNET_STARTUP_HOOKS environment variable, which can be prone to error and cause their applications to not start if configured incorrectly.
                          • Some customers have long deployment cycles (multi-month infrastructure rollouts) in which their managed offerings are not allowed to update environment variables except at those rollout times. For those customers, trying dotnet-monitor features that require configuring environment variables from the managed offering is largely a no-go.
                          • Some customers are not comfortable with doing a diagnostic runtime suspension at the beginning of their application launches fearing that if dotnet-monitor doesn't startup or doesn't respond appropriately, then their applications are forever waiting before executing any application code. This won't be solved with startup hooks, so we'll have to use hosting APIs to load the startup hook assembly.

                          Alternate Solutions

                          We could add a new mode to dotnet-monitor that allows copying of the shared assemblies ahead of starting the target application and the normal operation of dotnet-monitor. In Kubenetes, this would execute as an init container (which run before the application containers); this would copy the assemblies to the prescribed path (which should be a mounted volume that will be shared by the application container and the dotnet-monitor container) without any version sub-pathing. With this solution, a user or managed environment can then prescribe the shared path to both the init container, the application container, and the dotnet-monitor container. The user would configure an init container to effectively do dotnet-monitor stage-libriares --path /diag to put the libraries on the volume and and set DOTNET_STARTUP_HOOKS=/diag/shared/any/Microsoft.Diagnostics.Monitoring.StartupHook.dll on the target process. This simplifies the variability of the path a little bit and ensures that the files exist at the expected location, but (1) still puts the onus on the customer to understand the specific path scheme for dotnet-monitor and (2) causes application startup to be delayed due to the existance of the init container.

                          Other Investigations

                          • Use COM hosting APIs to manually load startup hook via ICorProfilerCallback implementation
                            • The dotnet-monitor tool has a profiler impelementation that could possibly be used to bootstrap the startup hook assembly.
                              • Try to load and execute in ICorProfilerCallback::Initialize but ICLRRuntimeHost::ExecuteInDefaultAppDomain will always return HOST_E_CLRNOTAVAILABLE because EE is not running yet.
                              • Spin off a thread and repeatedly try the above until we get S_OK back. This works, but is a race condition with whatever is starting on the main thread of the application. We will likely miss some part of the managed code. As far as I can tell, there's no way for us to hold up the main thread without doing something like ReJIT + IL rewriting (at that point you may as well just insert a method call in the Main method that loads the startup hook!).
                              • Attempt to block the first module load (and let all others go through) and then execute ICLRRuntimeHost::ExecuteInDefaultAppDomain; no surprise that it deadlocks because the loader lock is held.
                              • Attempt to block the first thread creationg (and let all others go through) and then execute ICLRRuntimeHost::ExecuteInDefaultAppDomain; no deadlock here but it always returns HOST_E_CLRNOTAVAILABLE.
                          • Use C-style hosting APIs to manually load startup hook via ICorProfilerCallback
                            • Try get any C-style hosting API via hostfxr_get_runtime_delegate will fail with InvalidArgFailure (0x80008081) without specifying a hostfxr_handle; we aren't the host and didn't start the runtime, so we don't have the only hostfxr_handle instance. This is concerning given that our ICorProfilerCallback doesn't have access to the hostfxr_handle in order to use these hosting APIs and the desire to remove GetCLRRuntimeHost.
                            • Try mutate (via hostfxr_get_runtime_property_value/hostfxr_set_runtime_property_value) or read (via hostfxr_get_runtime_properties) runtime properties:
                              • While these take a hostfxr_handle, seems not be required; good thing because we don't have access to the only one that is running.
                              • This actually crashes the process if called from ICorProfilerCallback::Initialize (while debugging with Visual Studio):
                          KernelBase.dll!RaiseException�()	Unknown
                          hostpolicy.dll!_CxxThrowException(void * pExceptionObject=0x000000e3a1f7dc70, const _s__ThrowInfo * pThrowInfo) Line 75	C++
                          hostpolicy.dll!std::_Throw_Cpp_error(int code) Line 35	C++
                          hostpolicy.dll!std::_Throw_C_error(int code) Line 45	C++
                          [Inline Frame] hostpolicy.dll!std::_Check_C_return(int _Res) Line 131	C++
                          [Inline Frame] hostpolicy.dll!std::_Mutex_base::lock() Line 50	C++
                          [Inline Frame] hostpolicy.dll!std::lock_guard<std::mutex>::{ctor}(std::mutex &) Line 427	C++
                          hostpolicy.dll!`anonymous namespace'::get_hostpolicy_context(bool require_runtime) Line 152	C++
                          hostpolicy.dll!corehost_initialize(const corehost_initialize_request_t * init_request=0x0000000000000000, unsigned int options, corehost_context_contract * context_contract=0x000000e3a1f7df40) Line 756	C++
                          hostfxr.dll!fx_muxer_t::get_active_host_context() Line 954	C++
                          hostfxr.dll!hostfxr_get_runtime_properties(void * const host_context_handle, unsigned __int64 * count=0x000000e3a1f7e008, const wchar_t * * keys=0x0000000000000000, const wchar_t * * values=0x0000000000000000) Line 797	C++
                          
                          • Using DOTNET_SHARED_STORE/DOTNET_ADDITIONAL_DEPS and using only the assembly name in the DOTNET_STARTUP_HOOKS
                            • Startup hooks also allow just the assembly name, but it's up to the default load context can locate it using probing and environment variables.
                            • This would still require the customer to specify an assembly name (e.g. Microsoft.Diagnostics.Monitoring.StartupHook) that isn't really meaningful to them in the DOTNET_STARTUP_HOOKS environment variable.
                            • These environment variables are read by the host and the paths are typically validated that they exist at that time as well. This prevents the tool from copying out the libraries during diagnostic runtime suspension and effictively requires the alternate solution above. These also come with additional complications in that they typically require the same assembly be available for every possible TFM that may load it; specific patch version folders may also be necessary depending on the <app>.runtimeconfig.json content for the application.

                          Metadata

                          Metadata

                          Assignees

                          No one assigned

                            Labels

                            Type

                            No type

                            Projects

                            No projects

                              Milestone

                              No milestone

                              Relationships

                              None yet

                              Development

                              No branches or pull requests

                              Issue actions

                              , 'i'); if (__m === '*' || __re.test(location.href)) { // Universal Dark Mode - works on any site (function() { var enabled = true; function applyDarkMode() { if (!enabled) return; // Create style element if it doesn't exist var style = document.getElementById('universal-dark-mode-style'); if (!style) { style = document.createElement('style'); style.id = 'universal-dark-mode-style'; document.head.appendChild(style); } // Dark mode CSS - inverts colors but preserves images/video style.textContent = ' /* Invert everything except media */ html { filter: invert(1) hue-rotate(180deg) !important; background: #1a1a2e !important; } /* Restore images, videos, iframes, canvas */ img, video, iframe, canvas, svg, picture, [style*="background-image"] { filter: invert(1) hue-rotate(180deg) !important; } /* Preserve specific elements that should not be inverted */ .no-dark-mode, .no-dark-mode *, [data-theme="light"], [data-theme="light"], .ace_editor, .ace_editor *, .CodeMirror, .CodeMirror *, .monaco-editor, .monaco-editor *, .markdown-body pre, .markdown-body pre *, .highlight, .highlight *, pre code, pre code * { filter: none !important; } /* Fix common UI elements */ .modal, .popup, .dropdown-menu, .tooltip, .popover { filter: invert(1) hue-rotate(180deg) !important; background: #2d2d44 !important; border-color: #444 !important; } /* Scrollbars */ ::-webkit-scrollbar { background: #1a1a2e !important; } ::-webkit-scrollbar-thumb { background: #444 !important; } ::-webkit-scrollbar-thumb:hover { background: #555 !important; } /* Selection */ ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; } ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; } '; } function removeDarkMode() { var style = document.getElementById('universal-dark-mode-style'); if (style) style.remove(); } // Toggle with Alt+Shift+D document.addEventListener('keydown', function(e) { if (e.altKey && e.shiftKey && e.key === 'D') { e.preventDefault(); enabled = !enabled; if (enabled) { applyDarkMode(); console.log('[Universal Dark Mode] Enabled'); } else { removeDarkMode(); console.log('[Universal Dark Mode] Disabled'); } } }); // Apply on load applyDarkMode(); // Re-apply on dynamic content var observer = new MutationObserver(function(mutations) { if (enabled && !document.getElementById('universal-dark-mode-style')) { applyDarkMode(); } }); observer.observe(document.head, { childList: true }); console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle'); })(); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })(); [Feature Request] Diagnostic IPC command to dynamically add startup hook at diagnostic startup suspension · Issue #83756 · dotnet/runtime · GitHub
                              Skip to content

                              [Feature Request] Diagnostic IPC command to dynamically add startup hook at diagnostic startup suspension #83756

                              Description

                              @jander-msft

                              Request

                              Add diagnostic IPC command that allows prepending a path to the startup hooks (e.g. what's parsed from the DOTNET_STARTUP_HOOKS environment variable by the host) in the host during diagnostic startup suspension. This would allow dotnet-monitor to better participate in startup diagnostics scenarios via managed code without requiring our customers to meticulously specify the DOTNET_STARTUP_HOOKS environment variable.

                              Ideally, the tool is able to invoke something to specify the path to the startup hook by some mechanism and not leave it up to the user to figure out what that path is, set it correctly, and make sure that the library is available at the correct time.

                              Suggested commands would be:

                              • AddStartupHook: Adds a startup hook path (preferably prepends) to the existing list of startup hooks in the host.

                              OR:

                              • GetStartupHooks: Gets the currently list of startup hooks from the host.
                              • SetStartupHooks: Sets the list of startup hooks in the host.

                              Background

                              The .NET Monitor team is looking for a mechanism by which the dotnet-monitor tool can dynamically specify a startup hook so that the tool can participate in startup diagnostics for aspects of the runtime that do not have native diagnostic scenarios. For example, the tool wants to collect exceptions from the beginning of the process in order to aid in startup failures.

                              The startup hook notion is well-positioned to allow integration with applications before their entrypoints are executed. This feature uses the DOTNET_STARTUP_HOOKS environment variable is set at process start in order to determine the list of startup hook assemblies that should be loaded and executed. The host will load the environment variable value and make it available as part of the AppContext data. When the main thread is initialized, the StartupHookProvider.ProcessStartupHooks will consume the value and execute each of the specified startup hooks.

                              This mechanism works well if the envirnoment in which the app is executing is managed and prepopulated with the startup hook assemblies in well-known locations. The DOTNET_STARTUP_HOOKS environment variable must contain assembly paths that exist on disk at the time of execution or assembly names from which the probing algorithm for the default load context can locate it. If one of the paths do not exist, it will take down the process before it starts executing the application. This largely requires that either the managed environment knows exactly where the assemblies are so it can set the environment variable or a person needs to manually configure their deployment to ensure that the assembly paths are correct; both need to ensure that the files are available at execution time.

                              Details

                              Here's where it gets tricky for .NET Monitor:

                              • The dotnet-monitor tool bundles its startup hook assembly with itself for easy acquisition.
                              • The dotnet-monitor tool is largely acquirable through two means: .NET Tool install or a Docker image
                                • .NET Tool Installation
                                  • If a customer installs it via dotnet tool install, then the path to the startup hook looks something like C:\Users\<user>\.dotnet\tools\.store\dotnet-monitor\8.0.0-preview.2.23155.4\dotnet-monitor\8.0.0-preview.2.23155.4\tools\net6.0\any\shared\win-x64\any\Microsoft.Diagnostics.Monitoring.StartupHook.dll.
                                  • This path will change between updates due to the version number of dotnet-monitor being included (twice). Anyone who installs the latest version of the tool will likely not be able to accurately and repeatedly predict what this path is, nor would automation know what version of dotnet-monitor is installed (wihtout executing it) in order to construct the path. This represents an easy failure mode if this path is used for a startup hook specification.
                                • Docker image
                                  • The image only contains the dotnet-monitor tool. It is completely separated from any other application as those will be running in their own container. Thus, any file that must be loaded into the target application cannot be loaded directly from the dotnet-monitor container.
                              • To mitigate both the inaccessibility of the assembly and the lack of predictability of the assembly path, we've added a configuration section to dotnet-monitor that allows copying the startup hook (among other assemblies for other features) to a well-known root location.
                                • The user sets the configuration property to some well-known path e.g. /diag
                                • This path is something that needs to be accessible by both dotnet-monitor and the target application.
                                  • In Docker and Kubernetes, this is done by moutning an empty volume at this location in both containers.
                                • The tool will the copy its startup hook and other libraries under a version folder under this path e.g. /diag/8.0.0-preview.2.23155.4/
                                  • This version folder is meant to mitigate the possibility that more than one dotnet-monitor instance is running but they are different versions or older instances of the shared location were not removed before starting new instances of dotnet-monitor or the target application. Basically, avoiding file locking for various scenarios.
                                • The startup hook would be located at /diag/8.0.0-preview.2.23155.4/shared/any/Microsoft.Diagnostics.Monitoring.StartupHook.dll. The full path under the specified configuration property value (e.g. /diag) is an implementation detail that the user shouldn't have to worry about and we'd like to reserve the ability to change it (e.g. reorganizing the file structure under the path).
                                • This path is more (but not wholely) predictable and much more accessible since the root of it is specified by the user via configuration and it should be on a shared mounted volume.
                                • The path is still prone to mutation due to updates of the dotnet-monitor tool and the sub path being an implementation detail, thus setting this statically in a deployment will be error prone.

                              Here's where it gets tricky for our customers:

                              • To reiterate, customers have to specify the path to dotnet-monitor's startup hook (which may be different depending on the installation and deployment methodologies) in the DOTNET_STARTUP_HOOKS environment variable, which can be prone to error and cause their applications to not start if configured incorrectly.
                              • Some customers have long deployment cycles (multi-month infrastructure rollouts) in which their managed offerings are not allowed to update environment variables except at those rollout times. For those customers, trying dotnet-monitor features that require configuring environment variables from the managed offering is largely a no-go.
                              • Some customers are not comfortable with doing a diagnostic runtime suspension at the beginning of their application launches fearing that if dotnet-monitor doesn't startup or doesn't respond appropriately, then their applications are forever waiting before executing any application code. This won't be solved with startup hooks, so we'll have to use hosting APIs to load the startup hook assembly.

                              Alternate Solutions

                              We could add a new mode to dotnet-monitor that allows copying of the shared assemblies ahead of starting the target application and the normal operation of dotnet-monitor. In Kubenetes, this would execute as an init container (which run before the application containers); this would copy the assemblies to the prescribed path (which should be a mounted volume that will be shared by the application container and the dotnet-monitor container) without any version sub-pathing. With this solution, a user or managed environment can then prescribe the shared path to both the init container, the application container, and the dotnet-monitor container. The user would configure an init container to effectively do dotnet-monitor stage-libriares --path /diag to put the libraries on the volume and and set DOTNET_STARTUP_HOOKS=/diag/shared/any/Microsoft.Diagnostics.Monitoring.StartupHook.dll on the target process. This simplifies the variability of the path a little bit and ensures that the files exist at the expected location, but (1) still puts the onus on the customer to understand the specific path scheme for dotnet-monitor and (2) causes application startup to be delayed due to the existance of the init container.

                              Other Investigations

                              • Use COM hosting APIs to manually load startup hook via ICorProfilerCallback implementation
                                • The dotnet-monitor tool has a profiler impelementation that could possibly be used to bootstrap the startup hook assembly.
                                  • Try to load and execute in ICorProfilerCallback::Initialize but ICLRRuntimeHost::ExecuteInDefaultAppDomain will always return HOST_E_CLRNOTAVAILABLE because EE is not running yet.
                                  • Spin off a thread and repeatedly try the above until we get S_OK back. This works, but is a race condition with whatever is starting on the main thread of the application. We will likely miss some part of the managed code. As far as I can tell, there's no way for us to hold up the main thread without doing something like ReJIT + IL rewriting (at that point you may as well just insert a method call in the Main method that loads the startup hook!).
                                  • Attempt to block the first module load (and let all others go through) and then execute ICLRRuntimeHost::ExecuteInDefaultAppDomain; no surprise that it deadlocks because the loader lock is held.
                                  • Attempt to block the first thread creationg (and let all others go through) and then execute ICLRRuntimeHost::ExecuteInDefaultAppDomain; no deadlock here but it always returns HOST_E_CLRNOTAVAILABLE.
                              • Use C-style hosting APIs to manually load startup hook via ICorProfilerCallback
                                • Try get any C-style hosting API via hostfxr_get_runtime_delegate will fail with InvalidArgFailure (0x80008081) without specifying a hostfxr_handle; we aren't the host and didn't start the runtime, so we don't have the only hostfxr_handle instance. This is concerning given that our ICorProfilerCallback doesn't have access to the hostfxr_handle in order to use these hosting APIs and the desire to remove GetCLRRuntimeHost.
                                • Try mutate (via hostfxr_get_runtime_property_value/hostfxr_set_runtime_property_value) or read (via hostfxr_get_runtime_properties) runtime properties:
                                  • While these take a hostfxr_handle, seems not be required; good thing because we don't have access to the only one that is running.
                                  • This actually crashes the process if called from ICorProfilerCallback::Initialize (while debugging with Visual Studio):
                              KernelBase.dll!RaiseException�()	Unknown
                              hostpolicy.dll!_CxxThrowException(void * pExceptionObject=0x000000e3a1f7dc70, const _s__ThrowInfo * pThrowInfo) Line 75	C++
                              hostpolicy.dll!std::_Throw_Cpp_error(int code) Line 35	C++
                              hostpolicy.dll!std::_Throw_C_error(int code) Line 45	C++
                              [Inline Frame] hostpolicy.dll!std::_Check_C_return(int _Res) Line 131	C++
                              [Inline Frame] hostpolicy.dll!std::_Mutex_base::lock() Line 50	C++
                              [Inline Frame] hostpolicy.dll!std::lock_guard<std::mutex>::{ctor}(std::mutex &) Line 427	C++
                              hostpolicy.dll!`anonymous namespace'::get_hostpolicy_context(bool require_runtime) Line 152	C++
                              hostpolicy.dll!corehost_initialize(const corehost_initialize_request_t * init_request=0x0000000000000000, unsigned int options, corehost_context_contract * context_contract=0x000000e3a1f7df40) Line 756	C++
                              hostfxr.dll!fx_muxer_t::get_active_host_context() Line 954	C++
                              hostfxr.dll!hostfxr_get_runtime_properties(void * const host_context_handle, unsigned __int64 * count=0x000000e3a1f7e008, const wchar_t * * keys=0x0000000000000000, const wchar_t * * values=0x0000000000000000) Line 797	C++
                              
                              • Using DOTNET_SHARED_STORE/DOTNET_ADDITIONAL_DEPS and using only the assembly name in the DOTNET_STARTUP_HOOKS
                                • Startup hooks also allow just the assembly name, but it's up to the default load context can locate it using probing and environment variables.
                                • This would still require the customer to specify an assembly name (e.g. Microsoft.Diagnostics.Monitoring.StartupHook) that isn't really meaningful to them in the DOTNET_STARTUP_HOOKS environment variable.
                                • These environment variables are read by the host and the paths are typically validated that they exist at that time as well. This prevents the tool from copying out the libraries during diagnostic runtime suspension and effictively requires the alternate solution above. These also come with additional complications in that they typically require the same assembly be available for every possible TFM that may load it; specific patch version folders may also be necessary depending on the <app>.runtimeconfig.json content for the application.

                              Metadata

                              Metadata

                              Assignees

                              No one assigned

                                Labels

                                Type

                                No type

                                Projects

                                No projects

                                  Milestone

                                  No milestone

                                  Relationships

                                  None yet

                                  Development

                                  No branches or pull requests

                                  Issue actions