HyperparameterTuner.attach() does not get use_spot_instances setting #1817

Description

@dz902

Describe the bug

When using the following:

PARENT_TUNER = HyperparameterTuner.attach(
tuning_job_name = PARENT_TUNING_JOB_NAME
)

...on a tuning job where its job definition has:

...
"StoppingCondition": {
"MaxRuntimeInSeconds": 3600,
"MaxWaitTimeInSeconds": 7200
},
"EnableNetworkIsolation": false,
"EnableInterContainerTrafficEncryption": false,
"EnableManagedSpotTraining": true
...

The max_wait and use_spot_instances setting are both None. I traced back to:

def_prepare_init_params_from_job_description(cls, job_details, model_channel_name=None):
"""Convert the job description to init params that can be handled by the
class constructor
Args:
job_details: the returned job details from a describe_training_job
API call.
model_channel_name (str): Name of the channel where pre-trained
model data will be downloaded.
Returns:
dictionary: The transformed init_params
"""
init_params=dict()
init_params["role"] =job_details["RoleArn"]
init_params["instance_count"] =job_details["ResourceConfig"]["InstanceCount"]
init_params["instance_type"] =job_details["ResourceConfig"]["InstanceType"]
init_params["volume_size"] =job_details["ResourceConfig"]["VolumeSizeInGB"]
init_params["max_run"] =job_details["StoppingCondition"]["MaxRuntimeInSeconds"]
init_params["input_mode"] =job_details["AlgorithmSpecification"]["TrainingInputMode"]
init_params["base_job_name"] =base_from_name(job_details["TrainingJobName"])
init_params["output_path"] =job_details["OutputDataConfig"]["S3OutputPath"]
init_params["output_kms_key"] =job_details["OutputDataConfig"]["KmsKeyId"]
if"EnableNetworkIsolation"injob_details:
init_params["enable_network_isolation"] =job_details["EnableNetworkIsolation"]
has_hps="HyperParameters"injob_details
init_params["hyperparameters"] =job_details["HyperParameters"] ifhas_hpselse {}
if"AlgorithmName"injob_details["AlgorithmSpecification"]:
init_params["algorithm_arn"] =job_details["AlgorithmSpecification"]["AlgorithmName"]
elif"TrainingImage"injob_details["AlgorithmSpecification"]:
init_params["image_uri"] =job_details["AlgorithmSpecification"]["TrainingImage"]
else:
raiseRuntimeError(
"Invalid AlgorithmSpecification. Either TrainingImage or "
"AlgorithmName is expected. None was found."
)
if"MetricDefinitons"injob_details["AlgorithmSpecification"]:
init_params["metric_definitions"] =job_details["AlgorithmSpecification"][
"MetricsDefinition"
]
if"EnableInterContainerTrafficEncryption"injob_details:
init_params["encrypt_inter_container_traffic"] =job_details[
"EnableInterContainerTrafficEncryption"
]
subnets, security_group_ids=vpc_utils.from_dict(job_details.get(vpc_utils.VPC_CONFIG_KEY))
ifsubnets:
init_params["subnets"] =subnets
ifsecurity_group_ids:
init_params["security_group_ids"] =security_group_ids
if"InputDataConfig"injob_detailsandmodel_channel_name:
forchannelinjob_details["InputDataConfig"]:
ifchannel["ChannelName"] ==model_channel_name:
init_params["model_channel_name"] =model_channel_name
init_params["model_uri"] =channel["DataSource"]["S3DataSource"]["S3Uri"]
break
returninit_params

It seems use_spot_instances and max_wait do not get carried over to newly created estimator.

To reproduce

See above.

Expected behavior

use_spot_instances and max_wait etc. should all be carried over to newly attach()ed tuner. This also affects warm start helpers like identical_data_and_algorithm().

If applicable, add screenshots or logs to help explain your problem.
**System information**
A description of your system. Please provide:
- **SageMaker Python SDK version**: v2.0.0
- **Framework name (eg. PyTorch) or algorithm (eg. KMeans)**:
- **Framework version**:
- **Python version**:
- **CPU or GPU**:
- **Custom Docker image (Y/N)**: N, official image classification image
**Additional context**
Add any other context about the problem here.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions

      , 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
       blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
      }
      } catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
      })();
      (function(){
      try {
      var __m = "github.com";
      var __re = new RegExp('^' + "github\\.com" + '
      
      Skip to content

      HyperparameterTuner.attach() does not get use_spot_instances setting #1817

      Description

      @dz902

      Describe the bug

      When using the following:

      PARENT_TUNER = HyperparameterTuner.attach(
      tuning_job_name = PARENT_TUNING_JOB_NAME
      )
      

      ...on a tuning job where its job definition has:

      ...
      "StoppingCondition": {
      "MaxRuntimeInSeconds": 3600,
      "MaxWaitTimeInSeconds": 7200
      },
      "EnableNetworkIsolation": false,
      "EnableInterContainerTrafficEncryption": false,
      "EnableManagedSpotTraining": true
      ...
      

      The max_wait and use_spot_instances setting are both None. I traced back to:

      def_prepare_init_params_from_job_description(cls, job_details, model_channel_name=None):
      """Convert the job description to init params that can be handled by the
      class constructor
      Args:
      job_details: the returned job details from a describe_training_job
      API call.
      model_channel_name (str): Name of the channel where pre-trained
      model data will be downloaded.
      Returns:
      dictionary: The transformed init_params
      """
      init_params=dict()
      init_params["role"] =job_details["RoleArn"]
      init_params["instance_count"] =job_details["ResourceConfig"]["InstanceCount"]
      init_params["instance_type"] =job_details["ResourceConfig"]["InstanceType"]
      init_params["volume_size"] =job_details["ResourceConfig"]["VolumeSizeInGB"]
      init_params["max_run"] =job_details["StoppingCondition"]["MaxRuntimeInSeconds"]
      init_params["input_mode"] =job_details["AlgorithmSpecification"]["TrainingInputMode"]
      init_params["base_job_name"] =base_from_name(job_details["TrainingJobName"])
      init_params["output_path"] =job_details["OutputDataConfig"]["S3OutputPath"]
      init_params["output_kms_key"] =job_details["OutputDataConfig"]["KmsKeyId"]
      if"EnableNetworkIsolation"injob_details:
      init_params["enable_network_isolation"] =job_details["EnableNetworkIsolation"]
      has_hps="HyperParameters"injob_details
      init_params["hyperparameters"] =job_details["HyperParameters"] ifhas_hpselse {}
      if"AlgorithmName"injob_details["AlgorithmSpecification"]:
      init_params["algorithm_arn"] =job_details["AlgorithmSpecification"]["AlgorithmName"]
      elif"TrainingImage"injob_details["AlgorithmSpecification"]:
      init_params["image_uri"] =job_details["AlgorithmSpecification"]["TrainingImage"]
      else:
      raiseRuntimeError(
      "Invalid AlgorithmSpecification. Either TrainingImage or "
      "AlgorithmName is expected. None was found."
      )
      if"MetricDefinitons"injob_details["AlgorithmSpecification"]:
      init_params["metric_definitions"] =job_details["AlgorithmSpecification"][
      "MetricsDefinition"
      ]
      if"EnableInterContainerTrafficEncryption"injob_details:
      init_params["encrypt_inter_container_traffic"] =job_details[
      "EnableInterContainerTrafficEncryption"
      ]
      subnets, security_group_ids=vpc_utils.from_dict(job_details.get(vpc_utils.VPC_CONFIG_KEY))
      ifsubnets:
      init_params["subnets"] =subnets
      ifsecurity_group_ids:
      init_params["security_group_ids"] =security_group_ids
      if"InputDataConfig"injob_detailsandmodel_channel_name:
      forchannelinjob_details["InputDataConfig"]:
      ifchannel["ChannelName"] ==model_channel_name:
      init_params["model_channel_name"] =model_channel_name
      init_params["model_uri"] =channel["DataSource"]["S3DataSource"]["S3Uri"]
      break
      returninit_params

      It seems use_spot_instances and max_wait do not get carried over to newly created estimator.

      To reproduce

      See above.

      Expected behavior

      use_spot_instances and max_wait etc. should all be carried over to newly attach()ed tuner. This also affects warm start helpers like identical_data_and_algorithm().

      If applicable, add screenshots or logs to help explain your problem.
      **System information**
      A description of your system. Please provide:
      - **SageMaker Python SDK version**: v2.0.0
      - **Framework name (eg. PyTorch) or algorithm (eg. KMeans)**:
      - **Framework version**:
      - **Python version**:
      - **CPU or GPU**:
      - **Custom Docker image (Y/N)**: N, official image classification image
      **Additional context**
      Add any other context about the problem here.
      

      Metadata

      Metadata

      Assignees

      No one assigned

        Type

        No type

        Projects

        No projects

          Milestone

          No milestone

          Relationships

          None yet

          Development

          No branches or pull requests

          Issue actions

          , 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
          Skip to content

          HyperparameterTuner.attach() does not get use_spot_instances setting #1817

          Description

          @dz902

          Describe the bug

          When using the following:

          PARENT_TUNER = HyperparameterTuner.attach(
          tuning_job_name = PARENT_TUNING_JOB_NAME
          )
          

          ...on a tuning job where its job definition has:

          ...
          "StoppingCondition": {
          "MaxRuntimeInSeconds": 3600,
          "MaxWaitTimeInSeconds": 7200
          },
          "EnableNetworkIsolation": false,
          "EnableInterContainerTrafficEncryption": false,
          "EnableManagedSpotTraining": true
          ...
          

          The max_wait and use_spot_instances setting are both None. I traced back to:

          def_prepare_init_params_from_job_description(cls, job_details, model_channel_name=None):
          """Convert the job description to init params that can be handled by the
          class constructor
          Args:
          job_details: the returned job details from a describe_training_job
          API call.
          model_channel_name (str): Name of the channel where pre-trained
          model data will be downloaded.
          Returns:
          dictionary: The transformed init_params
          """
          init_params=dict()
          init_params["role"] =job_details["RoleArn"]
          init_params["instance_count"] =job_details["ResourceConfig"]["InstanceCount"]
          init_params["instance_type"] =job_details["ResourceConfig"]["InstanceType"]
          init_params["volume_size"] =job_details["ResourceConfig"]["VolumeSizeInGB"]
          init_params["max_run"] =job_details["StoppingCondition"]["MaxRuntimeInSeconds"]
          init_params["input_mode"] =job_details["AlgorithmSpecification"]["TrainingInputMode"]
          init_params["base_job_name"] =base_from_name(job_details["TrainingJobName"])
          init_params["output_path"] =job_details["OutputDataConfig"]["S3OutputPath"]
          init_params["output_kms_key"] =job_details["OutputDataConfig"]["KmsKeyId"]
          if"EnableNetworkIsolation"injob_details:
          init_params["enable_network_isolation"] =job_details["EnableNetworkIsolation"]
          has_hps="HyperParameters"injob_details
          init_params["hyperparameters"] =job_details["HyperParameters"] ifhas_hpselse {}
          if"AlgorithmName"injob_details["AlgorithmSpecification"]:
          init_params["algorithm_arn"] =job_details["AlgorithmSpecification"]["AlgorithmName"]
          elif"TrainingImage"injob_details["AlgorithmSpecification"]:
          init_params["image_uri"] =job_details["AlgorithmSpecification"]["TrainingImage"]
          else:
          raiseRuntimeError(
          "Invalid AlgorithmSpecification. Either TrainingImage or "
          "AlgorithmName is expected. None was found."
          )
          if"MetricDefinitons"injob_details["AlgorithmSpecification"]:
          init_params["metric_definitions"] =job_details["AlgorithmSpecification"][
          "MetricsDefinition"
          ]
          if"EnableInterContainerTrafficEncryption"injob_details:
          init_params["encrypt_inter_container_traffic"] =job_details[
          "EnableInterContainerTrafficEncryption"
          ]
          subnets, security_group_ids=vpc_utils.from_dict(job_details.get(vpc_utils.VPC_CONFIG_KEY))
          ifsubnets:
          init_params["subnets"] =subnets
          ifsecurity_group_ids:
          init_params["security_group_ids"] =security_group_ids
          if"InputDataConfig"injob_detailsandmodel_channel_name:
          forchannelinjob_details["InputDataConfig"]:
          ifchannel["ChannelName"] ==model_channel_name:
          init_params["model_channel_name"] =model_channel_name
          init_params["model_uri"] =channel["DataSource"]["S3DataSource"]["S3Uri"]
          break
          returninit_params

          It seems use_spot_instances and max_wait do not get carried over to newly created estimator.

          To reproduce

          See above.

          Expected behavior

          use_spot_instances and max_wait etc. should all be carried over to newly attach()ed tuner. This also affects warm start helpers like identical_data_and_algorithm().

          If applicable, add screenshots or logs to help explain your problem.
          **System information**
          A description of your system. Please provide:
          - **SageMaker Python SDK version**: v2.0.0
          - **Framework name (eg. PyTorch) or algorithm (eg. KMeans)**:
          - **Framework version**:
          - **Python version**:
          - **CPU or GPU**:
          - **Custom Docker image (Y/N)**: N, official image classification image
          **Additional context**
          Add any other context about the problem here.
          

          Metadata

          Metadata

          Assignees

          No one assigned

            Type

            No type

            Projects

            No projects

              Milestone

              No milestone

              Relationships

              None yet

              Development

              No branches or pull requests

              Issue actions

              , 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
              Skip to content

              HyperparameterTuner.attach() does not get use_spot_instances setting #1817

              Description

              @dz902

              Describe the bug

              When using the following:

              PARENT_TUNER = HyperparameterTuner.attach(
              tuning_job_name = PARENT_TUNING_JOB_NAME
              )
              

              ...on a tuning job where its job definition has:

              ...
              "StoppingCondition": {
              "MaxRuntimeInSeconds": 3600,
              "MaxWaitTimeInSeconds": 7200
              },
              "EnableNetworkIsolation": false,
              "EnableInterContainerTrafficEncryption": false,
              "EnableManagedSpotTraining": true
              ...
              

              The max_wait and use_spot_instances setting are both None. I traced back to:

              def_prepare_init_params_from_job_description(cls, job_details, model_channel_name=None):
              """Convert the job description to init params that can be handled by the
              class constructor
              Args:
              job_details: the returned job details from a describe_training_job
              API call.
              model_channel_name (str): Name of the channel where pre-trained
              model data will be downloaded.
              Returns:
              dictionary: The transformed init_params
              """
              init_params=dict()
              init_params["role"] =job_details["RoleArn"]
              init_params["instance_count"] =job_details["ResourceConfig"]["InstanceCount"]
              init_params["instance_type"] =job_details["ResourceConfig"]["InstanceType"]
              init_params["volume_size"] =job_details["ResourceConfig"]["VolumeSizeInGB"]
              init_params["max_run"] =job_details["StoppingCondition"]["MaxRuntimeInSeconds"]
              init_params["input_mode"] =job_details["AlgorithmSpecification"]["TrainingInputMode"]
              init_params["base_job_name"] =base_from_name(job_details["TrainingJobName"])
              init_params["output_path"] =job_details["OutputDataConfig"]["S3OutputPath"]
              init_params["output_kms_key"] =job_details["OutputDataConfig"]["KmsKeyId"]
              if"EnableNetworkIsolation"injob_details:
              init_params["enable_network_isolation"] =job_details["EnableNetworkIsolation"]
              has_hps="HyperParameters"injob_details
              init_params["hyperparameters"] =job_details["HyperParameters"] ifhas_hpselse {}
              if"AlgorithmName"injob_details["AlgorithmSpecification"]:
              init_params["algorithm_arn"] =job_details["AlgorithmSpecification"]["AlgorithmName"]
              elif"TrainingImage"injob_details["AlgorithmSpecification"]:
              init_params["image_uri"] =job_details["AlgorithmSpecification"]["TrainingImage"]
              else:
              raiseRuntimeError(
              "Invalid AlgorithmSpecification. Either TrainingImage or "
              "AlgorithmName is expected. None was found."
              )
              if"MetricDefinitons"injob_details["AlgorithmSpecification"]:
              init_params["metric_definitions"] =job_details["AlgorithmSpecification"][
              "MetricsDefinition"
              ]
              if"EnableInterContainerTrafficEncryption"injob_details:
              init_params["encrypt_inter_container_traffic"] =job_details[
              "EnableInterContainerTrafficEncryption"
              ]
              subnets, security_group_ids=vpc_utils.from_dict(job_details.get(vpc_utils.VPC_CONFIG_KEY))
              ifsubnets:
              init_params["subnets"] =subnets
              ifsecurity_group_ids:
              init_params["security_group_ids"] =security_group_ids
              if"InputDataConfig"injob_detailsandmodel_channel_name:
              forchannelinjob_details["InputDataConfig"]:
              ifchannel["ChannelName"] ==model_channel_name:
              init_params["model_channel_name"] =model_channel_name
              init_params["model_uri"] =channel["DataSource"]["S3DataSource"]["S3Uri"]
              break
              returninit_params

              It seems use_spot_instances and max_wait do not get carried over to newly created estimator.

              To reproduce

              See above.

              Expected behavior

              use_spot_instances and max_wait etc. should all be carried over to newly attach()ed tuner. This also affects warm start helpers like identical_data_and_algorithm().

              If applicable, add screenshots or logs to help explain your problem.
              **System information**
              A description of your system. Please provide:
              - **SageMaker Python SDK version**: v2.0.0
              - **Framework name (eg. PyTorch) or algorithm (eg. KMeans)**:
              - **Framework version**:
              - **Python version**:
              - **CPU or GPU**:
              - **Custom Docker image (Y/N)**: N, official image classification image
              **Additional context**
              Add any other context about the problem here.
              

              Metadata

              Metadata

              Assignees

              No one assigned

                Type

                No type

                Projects

                No projects

                  Milestone

                  No milestone

                  Relationships

                  None yet

                  Development

                  No branches or pull requests

                  Issue actions

                  , 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
                  Skip to content

                  HyperparameterTuner.attach() does not get use_spot_instances setting #1817

                  Description

                  @dz902

                  Describe the bug

                  When using the following:

                  PARENT_TUNER = HyperparameterTuner.attach(
                  tuning_job_name = PARENT_TUNING_JOB_NAME
                  )
                  

                  ...on a tuning job where its job definition has:

                  ...
                  "StoppingCondition": {
                  "MaxRuntimeInSeconds": 3600,
                  "MaxWaitTimeInSeconds": 7200
                  },
                  "EnableNetworkIsolation": false,
                  "EnableInterContainerTrafficEncryption": false,
                  "EnableManagedSpotTraining": true
                  ...
                  

                  The max_wait and use_spot_instances setting are both None. I traced back to:

                  def_prepare_init_params_from_job_description(cls, job_details, model_channel_name=None):
                  """Convert the job description to init params that can be handled by the
                  class constructor
                  Args:
                  job_details: the returned job details from a describe_training_job
                  API call.
                  model_channel_name (str): Name of the channel where pre-trained
                  model data will be downloaded.
                  Returns:
                  dictionary: The transformed init_params
                  """
                  init_params=dict()
                  init_params["role"] =job_details["RoleArn"]
                  init_params["instance_count"] =job_details["ResourceConfig"]["InstanceCount"]
                  init_params["instance_type"] =job_details["ResourceConfig"]["InstanceType"]
                  init_params["volume_size"] =job_details["ResourceConfig"]["VolumeSizeInGB"]
                  init_params["max_run"] =job_details["StoppingCondition"]["MaxRuntimeInSeconds"]
                  init_params["input_mode"] =job_details["AlgorithmSpecification"]["TrainingInputMode"]
                  init_params["base_job_name"] =base_from_name(job_details["TrainingJobName"])
                  init_params["output_path"] =job_details["OutputDataConfig"]["S3OutputPath"]
                  init_params["output_kms_key"] =job_details["OutputDataConfig"]["KmsKeyId"]
                  if"EnableNetworkIsolation"injob_details:
                  init_params["enable_network_isolation"] =job_details["EnableNetworkIsolation"]
                  has_hps="HyperParameters"injob_details
                  init_params["hyperparameters"] =job_details["HyperParameters"] ifhas_hpselse {}
                  if"AlgorithmName"injob_details["AlgorithmSpecification"]:
                  init_params["algorithm_arn"] =job_details["AlgorithmSpecification"]["AlgorithmName"]
                  elif"TrainingImage"injob_details["AlgorithmSpecification"]:
                  init_params["image_uri"] =job_details["AlgorithmSpecification"]["TrainingImage"]
                  else:
                  raiseRuntimeError(
                  "Invalid AlgorithmSpecification. Either TrainingImage or "
                  "AlgorithmName is expected. None was found."
                  )
                  if"MetricDefinitons"injob_details["AlgorithmSpecification"]:
                  init_params["metric_definitions"] =job_details["AlgorithmSpecification"][
                  "MetricsDefinition"
                  ]
                  if"EnableInterContainerTrafficEncryption"injob_details:
                  init_params["encrypt_inter_container_traffic"] =job_details[
                  "EnableInterContainerTrafficEncryption"
                  ]
                  subnets, security_group_ids=vpc_utils.from_dict(job_details.get(vpc_utils.VPC_CONFIG_KEY))
                  ifsubnets:
                  init_params["subnets"] =subnets
                  ifsecurity_group_ids:
                  init_params["security_group_ids"] =security_group_ids
                  if"InputDataConfig"injob_detailsandmodel_channel_name:
                  forchannelinjob_details["InputDataConfig"]:
                  ifchannel["ChannelName"] ==model_channel_name:
                  init_params["model_channel_name"] =model_channel_name
                  init_params["model_uri"] =channel["DataSource"]["S3DataSource"]["S3Uri"]
                  break
                  returninit_params

                  It seems use_spot_instances and max_wait do not get carried over to newly created estimator.

                  To reproduce

                  See above.

                  Expected behavior

                  use_spot_instances and max_wait etc. should all be carried over to newly attach()ed tuner. This also affects warm start helpers like identical_data_and_algorithm().

                  If applicable, add screenshots or logs to help explain your problem.
                  **System information**
                  A description of your system. Please provide:
                  - **SageMaker Python SDK version**: v2.0.0
                  - **Framework name (eg. PyTorch) or algorithm (eg. KMeans)**:
                  - **Framework version**:
                  - **Python version**:
                  - **CPU or GPU**:
                  - **Custom Docker image (Y/N)**: N, official image classification image
                  **Additional context**
                  Add any other context about the problem here.
                  

                  Metadata

                  Metadata

                  Assignees

                  No one assigned

                    Type

                    No type

                    Projects

                    No projects

                      Milestone

                      No milestone

                      Relationships

                      None yet

                      Development

                      No branches or pull requests

                      Issue actions

                      , 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
                      Skip to content

                      HyperparameterTuner.attach() does not get use_spot_instances setting #1817

                      Description

                      @dz902

                      Describe the bug

                      When using the following:

                      PARENT_TUNER = HyperparameterTuner.attach(
                      tuning_job_name = PARENT_TUNING_JOB_NAME
                      )
                      

                      ...on a tuning job where its job definition has:

                      ...
                      "StoppingCondition": {
                      "MaxRuntimeInSeconds": 3600,
                      "MaxWaitTimeInSeconds": 7200
                      },
                      "EnableNetworkIsolation": false,
                      "EnableInterContainerTrafficEncryption": false,
                      "EnableManagedSpotTraining": true
                      ...
                      

                      The max_wait and use_spot_instances setting are both None. I traced back to:

                      def_prepare_init_params_from_job_description(cls, job_details, model_channel_name=None):
                      """Convert the job description to init params that can be handled by the
                      class constructor
                      Args:
                      job_details: the returned job details from a describe_training_job
                      API call.
                      model_channel_name (str): Name of the channel where pre-trained
                      model data will be downloaded.
                      Returns:
                      dictionary: The transformed init_params
                      """
                      init_params=dict()
                      init_params["role"] =job_details["RoleArn"]
                      init_params["instance_count"] =job_details["ResourceConfig"]["InstanceCount"]
                      init_params["instance_type"] =job_details["ResourceConfig"]["InstanceType"]
                      init_params["volume_size"] =job_details["ResourceConfig"]["VolumeSizeInGB"]
                      init_params["max_run"] =job_details["StoppingCondition"]["MaxRuntimeInSeconds"]
                      init_params["input_mode"] =job_details["AlgorithmSpecification"]["TrainingInputMode"]
                      init_params["base_job_name"] =base_from_name(job_details["TrainingJobName"])
                      init_params["output_path"] =job_details["OutputDataConfig"]["S3OutputPath"]
                      init_params["output_kms_key"] =job_details["OutputDataConfig"]["KmsKeyId"]
                      if"EnableNetworkIsolation"injob_details:
                      init_params["enable_network_isolation"] =job_details["EnableNetworkIsolation"]
                      has_hps="HyperParameters"injob_details
                      init_params["hyperparameters"] =job_details["HyperParameters"] ifhas_hpselse {}
                      if"AlgorithmName"injob_details["AlgorithmSpecification"]:
                      init_params["algorithm_arn"] =job_details["AlgorithmSpecification"]["AlgorithmName"]
                      elif"TrainingImage"injob_details["AlgorithmSpecification"]:
                      init_params["image_uri"] =job_details["AlgorithmSpecification"]["TrainingImage"]
                      else:
                      raiseRuntimeError(
                      "Invalid AlgorithmSpecification. Either TrainingImage or "
                      "AlgorithmName is expected. None was found."
                      )
                      if"MetricDefinitons"injob_details["AlgorithmSpecification"]:
                      init_params["metric_definitions"] =job_details["AlgorithmSpecification"][
                      "MetricsDefinition"
                      ]
                      if"EnableInterContainerTrafficEncryption"injob_details:
                      init_params["encrypt_inter_container_traffic"] =job_details[
                      "EnableInterContainerTrafficEncryption"
                      ]
                      subnets, security_group_ids=vpc_utils.from_dict(job_details.get(vpc_utils.VPC_CONFIG_KEY))
                      ifsubnets:
                      init_params["subnets"] =subnets
                      ifsecurity_group_ids:
                      init_params["security_group_ids"] =security_group_ids
                      if"InputDataConfig"injob_detailsandmodel_channel_name:
                      forchannelinjob_details["InputDataConfig"]:
                      ifchannel["ChannelName"] ==model_channel_name:
                      init_params["model_channel_name"] =model_channel_name
                      init_params["model_uri"] =channel["DataSource"]["S3DataSource"]["S3Uri"]
                      break
                      returninit_params

                      It seems use_spot_instances and max_wait do not get carried over to newly created estimator.

                      To reproduce

                      See above.

                      Expected behavior

                      use_spot_instances and max_wait etc. should all be carried over to newly attach()ed tuner. This also affects warm start helpers like identical_data_and_algorithm().

                      If applicable, add screenshots or logs to help explain your problem.
                      **System information**
                      A description of your system. Please provide:
                      - **SageMaker Python SDK version**: v2.0.0
                      - **Framework name (eg. PyTorch) or algorithm (eg. KMeans)**:
                      - **Framework version**:
                      - **Python version**:
                      - **CPU or GPU**:
                      - **Custom Docker image (Y/N)**: N, official image classification image
                      **Additional context**
                      Add any other context about the problem here.
                      

                      Metadata

                      Metadata

                      Assignees

                      No one assigned

                        Type

                        No type

                        Projects

                        No projects

                          Milestone

                          No milestone

                          Relationships

                          None yet

                          Development

                          No branches or pull requests

                          Issue actions

                          , 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
                          Skip to content

                          HyperparameterTuner.attach() does not get use_spot_instances setting #1817

                          Description

                          @dz902

                          Describe the bug

                          When using the following:

                          PARENT_TUNER = HyperparameterTuner.attach(
                          tuning_job_name = PARENT_TUNING_JOB_NAME
                          )
                          

                          ...on a tuning job where its job definition has:

                          ...
                          "StoppingCondition": {
                          "MaxRuntimeInSeconds": 3600,
                          "MaxWaitTimeInSeconds": 7200
                          },
                          "EnableNetworkIsolation": false,
                          "EnableInterContainerTrafficEncryption": false,
                          "EnableManagedSpotTraining": true
                          ...
                          

                          The max_wait and use_spot_instances setting are both None. I traced back to:

                          def_prepare_init_params_from_job_description(cls, job_details, model_channel_name=None):
                          """Convert the job description to init params that can be handled by the
                          class constructor
                          Args:
                          job_details: the returned job details from a describe_training_job
                          API call.
                          model_channel_name (str): Name of the channel where pre-trained
                          model data will be downloaded.
                          Returns:
                          dictionary: The transformed init_params
                          """
                          init_params=dict()
                          init_params["role"] =job_details["RoleArn"]
                          init_params["instance_count"] =job_details["ResourceConfig"]["InstanceCount"]
                          init_params["instance_type"] =job_details["ResourceConfig"]["InstanceType"]
                          init_params["volume_size"] =job_details["ResourceConfig"]["VolumeSizeInGB"]
                          init_params["max_run"] =job_details["StoppingCondition"]["MaxRuntimeInSeconds"]
                          init_params["input_mode"] =job_details["AlgorithmSpecification"]["TrainingInputMode"]
                          init_params["base_job_name"] =base_from_name(job_details["TrainingJobName"])
                          init_params["output_path"] =job_details["OutputDataConfig"]["S3OutputPath"]
                          init_params["output_kms_key"] =job_details["OutputDataConfig"]["KmsKeyId"]
                          if"EnableNetworkIsolation"injob_details:
                          init_params["enable_network_isolation"] =job_details["EnableNetworkIsolation"]
                          has_hps="HyperParameters"injob_details
                          init_params["hyperparameters"] =job_details["HyperParameters"] ifhas_hpselse {}
                          if"AlgorithmName"injob_details["AlgorithmSpecification"]:
                          init_params["algorithm_arn"] =job_details["AlgorithmSpecification"]["AlgorithmName"]
                          elif"TrainingImage"injob_details["AlgorithmSpecification"]:
                          init_params["image_uri"] =job_details["AlgorithmSpecification"]["TrainingImage"]
                          else:
                          raiseRuntimeError(
                          "Invalid AlgorithmSpecification. Either TrainingImage or "
                          "AlgorithmName is expected. None was found."
                          )
                          if"MetricDefinitons"injob_details["AlgorithmSpecification"]:
                          init_params["metric_definitions"] =job_details["AlgorithmSpecification"][
                          "MetricsDefinition"
                          ]
                          if"EnableInterContainerTrafficEncryption"injob_details:
                          init_params["encrypt_inter_container_traffic"] =job_details[
                          "EnableInterContainerTrafficEncryption"
                          ]
                          subnets, security_group_ids=vpc_utils.from_dict(job_details.get(vpc_utils.VPC_CONFIG_KEY))
                          ifsubnets:
                          init_params["subnets"] =subnets
                          ifsecurity_group_ids:
                          init_params["security_group_ids"] =security_group_ids
                          if"InputDataConfig"injob_detailsandmodel_channel_name:
                          forchannelinjob_details["InputDataConfig"]:
                          ifchannel["ChannelName"] ==model_channel_name:
                          init_params["model_channel_name"] =model_channel_name
                          init_params["model_uri"] =channel["DataSource"]["S3DataSource"]["S3Uri"]
                          break
                          returninit_params

                          It seems use_spot_instances and max_wait do not get carried over to newly created estimator.

                          To reproduce

                          See above.

                          Expected behavior

                          use_spot_instances and max_wait etc. should all be carried over to newly attach()ed tuner. This also affects warm start helpers like identical_data_and_algorithm().

                          If applicable, add screenshots or logs to help explain your problem.
                          **System information**
                          A description of your system. Please provide:
                          - **SageMaker Python SDK version**: v2.0.0
                          - **Framework name (eg. PyTorch) or algorithm (eg. KMeans)**:
                          - **Framework version**:
                          - **Python version**:
                          - **CPU or GPU**:
                          - **Custom Docker image (Y/N)**: N, official image classification image
                          **Additional context**
                          Add any other context about the problem here.
                          

                          Metadata

                          Metadata

                          Assignees

                          No one assigned

                            Type

                            No type

                            Projects

                            No projects

                              Milestone

                              No milestone

                              Relationships

                              None yet

                              Development

                              No branches or pull requests

                              Issue actions

                              , 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
                              Skip to content

                              HyperparameterTuner.attach() does not get use_spot_instances setting #1817

                              Description

                              @dz902

                              Describe the bug

                              When using the following:

                              PARENT_TUNER = HyperparameterTuner.attach(
                              tuning_job_name = PARENT_TUNING_JOB_NAME
                              )
                              

                              ...on a tuning job where its job definition has:

                              ...
                              "StoppingCondition": {
                              "MaxRuntimeInSeconds": 3600,
                              "MaxWaitTimeInSeconds": 7200
                              },
                              "EnableNetworkIsolation": false,
                              "EnableInterContainerTrafficEncryption": false,
                              "EnableManagedSpotTraining": true
                              ...
                              

                              The max_wait and use_spot_instances setting are both None. I traced back to:

                              def_prepare_init_params_from_job_description(cls, job_details, model_channel_name=None):
                              """Convert the job description to init params that can be handled by the
                              class constructor
                              Args:
                              job_details: the returned job details from a describe_training_job
                              API call.
                              model_channel_name (str): Name of the channel where pre-trained
                              model data will be downloaded.
                              Returns:
                              dictionary: The transformed init_params
                              """
                              init_params=dict()
                              init_params["role"] =job_details["RoleArn"]
                              init_params["instance_count"] =job_details["ResourceConfig"]["InstanceCount"]
                              init_params["instance_type"] =job_details["ResourceConfig"]["InstanceType"]
                              init_params["volume_size"] =job_details["ResourceConfig"]["VolumeSizeInGB"]
                              init_params["max_run"] =job_details["StoppingCondition"]["MaxRuntimeInSeconds"]
                              init_params["input_mode"] =job_details["AlgorithmSpecification"]["TrainingInputMode"]
                              init_params["base_job_name"] =base_from_name(job_details["TrainingJobName"])
                              init_params["output_path"] =job_details["OutputDataConfig"]["S3OutputPath"]
                              init_params["output_kms_key"] =job_details["OutputDataConfig"]["KmsKeyId"]
                              if"EnableNetworkIsolation"injob_details:
                              init_params["enable_network_isolation"] =job_details["EnableNetworkIsolation"]
                              has_hps="HyperParameters"injob_details
                              init_params["hyperparameters"] =job_details["HyperParameters"] ifhas_hpselse {}
                              if"AlgorithmName"injob_details["AlgorithmSpecification"]:
                              init_params["algorithm_arn"] =job_details["AlgorithmSpecification"]["AlgorithmName"]
                              elif"TrainingImage"injob_details["AlgorithmSpecification"]:
                              init_params["image_uri"] =job_details["AlgorithmSpecification"]["TrainingImage"]
                              else:
                              raiseRuntimeError(
                              "Invalid AlgorithmSpecification. Either TrainingImage or "
                              "AlgorithmName is expected. None was found."
                              )
                              if"MetricDefinitons"injob_details["AlgorithmSpecification"]:
                              init_params["metric_definitions"] =job_details["AlgorithmSpecification"][
                              "MetricsDefinition"
                              ]
                              if"EnableInterContainerTrafficEncryption"injob_details:
                              init_params["encrypt_inter_container_traffic"] =job_details[
                              "EnableInterContainerTrafficEncryption"
                              ]
                              subnets, security_group_ids=vpc_utils.from_dict(job_details.get(vpc_utils.VPC_CONFIG_KEY))
                              ifsubnets:
                              init_params["subnets"] =subnets
                              ifsecurity_group_ids:
                              init_params["security_group_ids"] =security_group_ids
                              if"InputDataConfig"injob_detailsandmodel_channel_name:
                              forchannelinjob_details["InputDataConfig"]:
                              ifchannel["ChannelName"] ==model_channel_name:
                              init_params["model_channel_name"] =model_channel_name
                              init_params["model_uri"] =channel["DataSource"]["S3DataSource"]["S3Uri"]
                              break
                              returninit_params

                              It seems use_spot_instances and max_wait do not get carried over to newly created estimator.

                              To reproduce

                              See above.

                              Expected behavior

                              use_spot_instances and max_wait etc. should all be carried over to newly attach()ed tuner. This also affects warm start helpers like identical_data_and_algorithm().

                              If applicable, add screenshots or logs to help explain your problem.
                              **System information**
                              A description of your system. Please provide:
                              - **SageMaker Python SDK version**: v2.0.0
                              - **Framework name (eg. PyTorch) or algorithm (eg. KMeans)**:
                              - **Framework version**:
                              - **Python version**:
                              - **CPU or GPU**:
                              - **Custom Docker image (Y/N)**: N, official image classification image
                              **Additional context**
                              Add any other context about the problem here.
                              

                              Metadata

                              Metadata

                              Assignees

                              No one assigned

                                Type

                                No type

                                Projects

                                No projects

                                  Milestone

                                  No milestone

                                  Relationships

                                  None yet

                                  Development

                                  No branches or pull requests

                                  Issue actions