fix: resolve MTRL eval base-model ARN against the configured hub (not always SageMakerPublicHub)

[jam-jee]

Issue

MTRL evaluation of a fine-tuned model via MultiTurnRLTrainer.attach(...) (or a ModelPackage) fails server-side inside the evaluation pipeline:

StepDetail(name='EvaluateFineTunedModel', status='Failed',
failure_reason="ClientError: Failed to invoke sagemaker:CreateJob.
Error Details: ResourceNotFound: Hub content with name mock-oss-test does not exist.")

Root cause

When the base model is resolved from a model package, its hub-content ARN is reconstructed from the package's BaseModel metadata. The backend populates only HubContentName + HubContentVersion (no HubContentArn), so the SDK rebuilds the ARN in _resolve_model_package_object — and that reconstruction was hardcoded to SageMakerPublicHub with an account-less (aws) owner:

# Base model always lives in SageMakerPublicHub (SAGEMAKER_HUB_NAME is for training recipes only)base_model_arn=f"arn:aws:sagemaker:{region}:aws:hub-content/SageMakerPublicHub/Model/{name}/{version}"

For a model customized against a private/custom hub (SAGEMAKER_HUB_NAME), this yields a public-hub ARN that doesn't exist, and the eval pipeline's CreateJob fails with ResourceNotFound.

The bare-string / JumpStart-ID path does not have this bug — it already resolves via _resolve_jumpstart_model(..., get_sagemaker_hub_name()), which honors SAGEMAKER_HUB_NAME. Only the model-package / attached-job path ignored it.

This affects any customer who fine-tunes an MTRL model whose base model lives in a private/custom hub and then evaluates it via attach() or a ModelPackage — not just integration tests.

Fix

In _resolve_model_package_object, reconstruct the base-model ARN in the hub the model was actually customized against:

• Use get_sagemaker_hub_name() (honors SAGEMAKER_HUB_NAME, still defaults to SageMakerPublicHub) instead of a hardcoded string.
• Use the correct ARN owner: account-less aws for the public hub, the model package's own account for a private hub.

Verification

• Reproduced against a live account: resolver now yields
  arn:aws:sagemaker:us-west-2:<acct>:hub-content/sdktest/Model/mock-oss-test/0.0.1,
  and describe-hub-content confirms that ARN is Available — so the pipeline's CreateJob finds the base model.
• Unit tests: tests/unit/train/common_utils/test_model_resolution.py — 38 passed.
  • Added test_resolve_arn_construct_hub_content_arn_private_hub (private-hub case).
  • Pinned the existing test_resolve_arn_construct_hub_content_arn to the default public hub.

Notes

• This unblocks the gpu_intensive nightly integ test test_evaluate_with_attached_trainer, which currently fails because mock-oss-test lives only in the private sdktest hub.
• A complementary backend improvement (populate HubContentArn in the model package BaseModel block at registration time) would let the SDK skip reconstruction entirely — the SDK already prefers hub_content_arn when present. That is out of scope for this SDK change.

[jam-jee]

Issue

MTRL evaluation of a fine-tuned model via MultiTurnRLTrainer.attach(...) (or a ModelPackage) fails server-side inside the evaluation pipeline:

StepDetail(name='EvaluateFineTunedModel', status='Failed',
failure_reason="ClientError: Failed to invoke sagemaker:CreateJob.
Error Details: ResourceNotFound: Hub content with name mock-oss-test does not exist.")

Root cause

When the base model is resolved from a model package, its hub-content ARN is reconstructed from the package's BaseModel metadata. The backend populates only HubContentName + HubContentVersion (no HubContentArn), so the SDK rebuilds the ARN in _resolve_model_package_object — and that reconstruction was hardcoded to SageMakerPublicHub with an account-less (aws) owner:

# Base model always lives in SageMakerPublicHub (SAGEMAKER_HUB_NAME is for training recipes only)base_model_arn=f"arn:aws:sagemaker:{region}:aws:hub-content/SageMakerPublicHub/Model/{name}/{version}"

For a model customized against a private/custom hub (SAGEMAKER_HUB_NAME), this yields a public-hub ARN that doesn't exist, and the eval pipeline's CreateJob fails with ResourceNotFound.

The bare-string / JumpStart-ID path does not have this bug — it already resolves via _resolve_jumpstart_model(..., get_sagemaker_hub_name()), which honors SAGEMAKER_HUB_NAME. Only the model-package / attached-job path ignored it.

This affects any customer who fine-tunes an MTRL model whose base model lives in a private/custom hub and then evaluates it via attach() or a ModelPackage — not just integration tests.

Fix

In _resolve_model_package_object, reconstruct the base-model ARN in the hub the model was actually customized against:

• Use get_sagemaker_hub_name() (honors SAGEMAKER_HUB_NAME, still defaults to SageMakerPublicHub) instead of a hardcoded string.
• Use the correct ARN owner: account-less aws for the public hub, the model package's own account for a private hub.

Verification

• Reproduced against a live account: resolver now yields
  arn:aws:sagemaker:us-west-2:<acct>:hub-content/sdktest/Model/mock-oss-test/0.0.1,
  and describe-hub-content confirms that ARN is Available — so the pipeline's CreateJob finds the base model.
• Unit tests: tests/unit/train/common_utils/test_model_resolution.py — 38 passed.
  • Added test_resolve_arn_construct_hub_content_arn_private_hub (private-hub case).
  • Pinned the existing test_resolve_arn_construct_hub_content_arn to the default public hub.

Notes

• This unblocks the gpu_intensive nightly integ test test_evaluate_with_attached_trainer, which currently fails because mock-oss-test lives only in the private sdktest hub.
• A complementary backend improvement (populate HubContentArn in the model package BaseModel block at registration time) would let the SDK skip reconstruction entirely — the SDK already prefers hub_content_arn when present. That is out of scope for this SDK change.

[jam-jee]

Issue

MTRL evaluation of a fine-tuned model via MultiTurnRLTrainer.attach(...) (or a ModelPackage) fails server-side inside the evaluation pipeline:

StepDetail(name='EvaluateFineTunedModel', status='Failed',
failure_reason="ClientError: Failed to invoke sagemaker:CreateJob.
Error Details: ResourceNotFound: Hub content with name mock-oss-test does not exist.")

Root cause

When the base model is resolved from a model package, its hub-content ARN is reconstructed from the package's BaseModel metadata. The backend populates only HubContentName + HubContentVersion (no HubContentArn), so the SDK rebuilds the ARN in _resolve_model_package_object — and that reconstruction was hardcoded to SageMakerPublicHub with an account-less (aws) owner:

# Base model always lives in SageMakerPublicHub (SAGEMAKER_HUB_NAME is for training recipes only)base_model_arn=f"arn:aws:sagemaker:{region}:aws:hub-content/SageMakerPublicHub/Model/{name}/{version}"

For a model customized against a private/custom hub (SAGEMAKER_HUB_NAME), this yields a public-hub ARN that doesn't exist, and the eval pipeline's CreateJob fails with ResourceNotFound.

The bare-string / JumpStart-ID path does not have this bug — it already resolves via _resolve_jumpstart_model(..., get_sagemaker_hub_name()), which honors SAGEMAKER_HUB_NAME. Only the model-package / attached-job path ignored it.

This affects any customer who fine-tunes an MTRL model whose base model lives in a private/custom hub and then evaluates it via attach() or a ModelPackage — not just integration tests.

Fix

In _resolve_model_package_object, reconstruct the base-model ARN in the hub the model was actually customized against:

• Use get_sagemaker_hub_name() (honors SAGEMAKER_HUB_NAME, still defaults to SageMakerPublicHub) instead of a hardcoded string.
• Use the correct ARN owner: account-less aws for the public hub, the model package's own account for a private hub.

Verification

• Reproduced against a live account: resolver now yields
  arn:aws:sagemaker:us-west-2:<acct>:hub-content/sdktest/Model/mock-oss-test/0.0.1,
  and describe-hub-content confirms that ARN is Available — so the pipeline's CreateJob finds the base model.
• Unit tests: tests/unit/train/common_utils/test_model_resolution.py — 38 passed.
  • Added test_resolve_arn_construct_hub_content_arn_private_hub (private-hub case).
  • Pinned the existing test_resolve_arn_construct_hub_content_arn to the default public hub.

Notes

• This unblocks the gpu_intensive nightly integ test test_evaluate_with_attached_trainer, which currently fails because mock-oss-test lives only in the private sdktest hub.
• A complementary backend improvement (populate HubContentArn in the model package BaseModel block at registration time) would let the SDK skip reconstruction entirely — the SDK already prefers hub_content_arn when present. That is out of scope for this SDK change.

[jam-jee]

Issue

MTRL evaluation of a fine-tuned model via MultiTurnRLTrainer.attach(...) (or a ModelPackage) fails server-side inside the evaluation pipeline:

StepDetail(name='EvaluateFineTunedModel', status='Failed',
failure_reason="ClientError: Failed to invoke sagemaker:CreateJob.
Error Details: ResourceNotFound: Hub content with name mock-oss-test does not exist.")

Root cause

When the base model is resolved from a model package, its hub-content ARN is reconstructed from the package's BaseModel metadata. The backend populates only HubContentName + HubContentVersion (no HubContentArn), so the SDK rebuilds the ARN in _resolve_model_package_object — and that reconstruction was hardcoded to SageMakerPublicHub with an account-less (aws) owner:

# Base model always lives in SageMakerPublicHub (SAGEMAKER_HUB_NAME is for training recipes only)base_model_arn=f"arn:aws:sagemaker:{region}:aws:hub-content/SageMakerPublicHub/Model/{name}/{version}"

For a model customized against a private/custom hub (SAGEMAKER_HUB_NAME), this yields a public-hub ARN that doesn't exist, and the eval pipeline's CreateJob fails with ResourceNotFound.

The bare-string / JumpStart-ID path does not have this bug — it already resolves via _resolve_jumpstart_model(..., get_sagemaker_hub_name()), which honors SAGEMAKER_HUB_NAME. Only the model-package / attached-job path ignored it.

This affects any customer who fine-tunes an MTRL model whose base model lives in a private/custom hub and then evaluates it via attach() or a ModelPackage — not just integration tests.

Fix

In _resolve_model_package_object, reconstruct the base-model ARN in the hub the model was actually customized against:

• Use get_sagemaker_hub_name() (honors SAGEMAKER_HUB_NAME, still defaults to SageMakerPublicHub) instead of a hardcoded string.
• Use the correct ARN owner: account-less aws for the public hub, the model package's own account for a private hub.

Verification

• Reproduced against a live account: resolver now yields
  arn:aws:sagemaker:us-west-2:<acct>:hub-content/sdktest/Model/mock-oss-test/0.0.1,
  and describe-hub-content confirms that ARN is Available — so the pipeline's CreateJob finds the base model.
• Unit tests: tests/unit/train/common_utils/test_model_resolution.py — 38 passed.
  • Added test_resolve_arn_construct_hub_content_arn_private_hub (private-hub case).
  • Pinned the existing test_resolve_arn_construct_hub_content_arn to the default public hub.

Notes

• This unblocks the gpu_intensive nightly integ test test_evaluate_with_attached_trainer, which currently fails because mock-oss-test lives only in the private sdktest hub.
• A complementary backend improvement (populate HubContentArn in the model package BaseModel block at registration time) would let the SDK skip reconstruction entirely — the SDK already prefers hub_content_arn when present. That is out of scope for this SDK change.

[jam-jee]

Issue

MTRL evaluation of a fine-tuned model via MultiTurnRLTrainer.attach(...) (or a ModelPackage) fails server-side inside the evaluation pipeline:

StepDetail(name='EvaluateFineTunedModel', status='Failed',
failure_reason="ClientError: Failed to invoke sagemaker:CreateJob.
Error Details: ResourceNotFound: Hub content with name mock-oss-test does not exist.")

Root cause

When the base model is resolved from a model package, its hub-content ARN is reconstructed from the package's BaseModel metadata. The backend populates only HubContentName + HubContentVersion (no HubContentArn), so the SDK rebuilds the ARN in _resolve_model_package_object — and that reconstruction was hardcoded to SageMakerPublicHub with an account-less (aws) owner:

# Base model always lives in SageMakerPublicHub (SAGEMAKER_HUB_NAME is for training recipes only)base_model_arn=f"arn:aws:sagemaker:{region}:aws:hub-content/SageMakerPublicHub/Model/{name}/{version}"

For a model customized against a private/custom hub (SAGEMAKER_HUB_NAME), this yields a public-hub ARN that doesn't exist, and the eval pipeline's CreateJob fails with ResourceNotFound.

The bare-string / JumpStart-ID path does not have this bug — it already resolves via _resolve_jumpstart_model(..., get_sagemaker_hub_name()), which honors SAGEMAKER_HUB_NAME. Only the model-package / attached-job path ignored it.

This affects any customer who fine-tunes an MTRL model whose base model lives in a private/custom hub and then evaluates it via attach() or a ModelPackage — not just integration tests.

Fix

In _resolve_model_package_object, reconstruct the base-model ARN in the hub the model was actually customized against:

• Use get_sagemaker_hub_name() (honors SAGEMAKER_HUB_NAME, still defaults to SageMakerPublicHub) instead of a hardcoded string.
• Use the correct ARN owner: account-less aws for the public hub, the model package's own account for a private hub.

Verification

• Reproduced against a live account: resolver now yields
  arn:aws:sagemaker:us-west-2:<acct>:hub-content/sdktest/Model/mock-oss-test/0.0.1,
  and describe-hub-content confirms that ARN is Available — so the pipeline's CreateJob finds the base model.
• Unit tests: tests/unit/train/common_utils/test_model_resolution.py — 38 passed.
  • Added test_resolve_arn_construct_hub_content_arn_private_hub (private-hub case).
  • Pinned the existing test_resolve_arn_construct_hub_content_arn to the default public hub.

Notes

• This unblocks the gpu_intensive nightly integ test test_evaluate_with_attached_trainer, which currently fails because mock-oss-test lives only in the private sdktest hub.
• A complementary backend improvement (populate HubContentArn in the model package BaseModel block at registration time) would let the SDK skip reconstruction entirely — the SDK already prefers hub_content_arn when present. That is out of scope for this SDK change.

[jam-jee]

Issue

MTRL evaluation of a fine-tuned model via MultiTurnRLTrainer.attach(...) (or a ModelPackage) fails server-side inside the evaluation pipeline:

StepDetail(name='EvaluateFineTunedModel', status='Failed',
failure_reason="ClientError: Failed to invoke sagemaker:CreateJob.
Error Details: ResourceNotFound: Hub content with name mock-oss-test does not exist.")

Root cause

When the base model is resolved from a model package, its hub-content ARN is reconstructed from the package's BaseModel metadata. The backend populates only HubContentName + HubContentVersion (no HubContentArn), so the SDK rebuilds the ARN in _resolve_model_package_object — and that reconstruction was hardcoded to SageMakerPublicHub with an account-less (aws) owner:

# Base model always lives in SageMakerPublicHub (SAGEMAKER_HUB_NAME is for training recipes only)base_model_arn=f"arn:aws:sagemaker:{region}:aws:hub-content/SageMakerPublicHub/Model/{name}/{version}"

For a model customized against a private/custom hub (SAGEMAKER_HUB_NAME), this yields a public-hub ARN that doesn't exist, and the eval pipeline's CreateJob fails with ResourceNotFound.

The bare-string / JumpStart-ID path does not have this bug — it already resolves via _resolve_jumpstart_model(..., get_sagemaker_hub_name()), which honors SAGEMAKER_HUB_NAME. Only the model-package / attached-job path ignored it.

This affects any customer who fine-tunes an MTRL model whose base model lives in a private/custom hub and then evaluates it via attach() or a ModelPackage — not just integration tests.

Fix

In _resolve_model_package_object, reconstruct the base-model ARN in the hub the model was actually customized against:

• Use get_sagemaker_hub_name() (honors SAGEMAKER_HUB_NAME, still defaults to SageMakerPublicHub) instead of a hardcoded string.
• Use the correct ARN owner: account-less aws for the public hub, the model package's own account for a private hub.

Verification

• Reproduced against a live account: resolver now yields
  arn:aws:sagemaker:us-west-2:<acct>:hub-content/sdktest/Model/mock-oss-test/0.0.1,
  and describe-hub-content confirms that ARN is Available — so the pipeline's CreateJob finds the base model.
• Unit tests: tests/unit/train/common_utils/test_model_resolution.py — 38 passed.
  • Added test_resolve_arn_construct_hub_content_arn_private_hub (private-hub case).
  • Pinned the existing test_resolve_arn_construct_hub_content_arn to the default public hub.

Notes

• This unblocks the gpu_intensive nightly integ test test_evaluate_with_attached_trainer, which currently fails because mock-oss-test lives only in the private sdktest hub.
• A complementary backend improvement (populate HubContentArn in the model package BaseModel block at registration time) would let the SDK skip reconstruction entirely — the SDK already prefers hub_content_arn when present. That is out of scope for this SDK change.

[jam-jee]

Issue

MTRL evaluation of a fine-tuned model via MultiTurnRLTrainer.attach(...) (or a ModelPackage) fails server-side inside the evaluation pipeline:

StepDetail(name='EvaluateFineTunedModel', status='Failed',
failure_reason="ClientError: Failed to invoke sagemaker:CreateJob.
Error Details: ResourceNotFound: Hub content with name mock-oss-test does not exist.")

Root cause

When the base model is resolved from a model package, its hub-content ARN is reconstructed from the package's BaseModel metadata. The backend populates only HubContentName + HubContentVersion (no HubContentArn), so the SDK rebuilds the ARN in _resolve_model_package_object — and that reconstruction was hardcoded to SageMakerPublicHub with an account-less (aws) owner:

# Base model always lives in SageMakerPublicHub (SAGEMAKER_HUB_NAME is for training recipes only)base_model_arn=f"arn:aws:sagemaker:{region}:aws:hub-content/SageMakerPublicHub/Model/{name}/{version}"

For a model customized against a private/custom hub (SAGEMAKER_HUB_NAME), this yields a public-hub ARN that doesn't exist, and the eval pipeline's CreateJob fails with ResourceNotFound.

The bare-string / JumpStart-ID path does not have this bug — it already resolves via _resolve_jumpstart_model(..., get_sagemaker_hub_name()), which honors SAGEMAKER_HUB_NAME. Only the model-package / attached-job path ignored it.

This affects any customer who fine-tunes an MTRL model whose base model lives in a private/custom hub and then evaluates it via attach() or a ModelPackage — not just integration tests.

Fix

In _resolve_model_package_object, reconstruct the base-model ARN in the hub the model was actually customized against:

• Use get_sagemaker_hub_name() (honors SAGEMAKER_HUB_NAME, still defaults to SageMakerPublicHub) instead of a hardcoded string.
• Use the correct ARN owner: account-less aws for the public hub, the model package's own account for a private hub.

Verification

• Reproduced against a live account: resolver now yields
  arn:aws:sagemaker:us-west-2:<acct>:hub-content/sdktest/Model/mock-oss-test/0.0.1,
  and describe-hub-content confirms that ARN is Available — so the pipeline's CreateJob finds the base model.
• Unit tests: tests/unit/train/common_utils/test_model_resolution.py — 38 passed.
  • Added test_resolve_arn_construct_hub_content_arn_private_hub (private-hub case).
  • Pinned the existing test_resolve_arn_construct_hub_content_arn to the default public hub.

Notes

• This unblocks the gpu_intensive nightly integ test test_evaluate_with_attached_trainer, which currently fails because mock-oss-test lives only in the private sdktest hub.
• A complementary backend improvement (populate HubContentArn in the model package BaseModel block at registration time) would let the SDK skip reconstruction entirely — the SDK already prefers hub_content_arn when present. That is out of scope for this SDK change.

[jam-jee]

Issue

MTRL evaluation of a fine-tuned model via MultiTurnRLTrainer.attach(...) (or a ModelPackage) fails server-side inside the evaluation pipeline:

StepDetail(name='EvaluateFineTunedModel', status='Failed',
failure_reason="ClientError: Failed to invoke sagemaker:CreateJob.
Error Details: ResourceNotFound: Hub content with name mock-oss-test does not exist.")

Root cause

When the base model is resolved from a model package, its hub-content ARN is reconstructed from the package's BaseModel metadata. The backend populates only HubContentName + HubContentVersion (no HubContentArn), so the SDK rebuilds the ARN in _resolve_model_package_object — and that reconstruction was hardcoded to SageMakerPublicHub with an account-less (aws) owner:

# Base model always lives in SageMakerPublicHub (SAGEMAKER_HUB_NAME is for training recipes only)base_model_arn=f"arn:aws:sagemaker:{region}:aws:hub-content/SageMakerPublicHub/Model/{name}/{version}"

For a model customized against a private/custom hub (SAGEMAKER_HUB_NAME), this yields a public-hub ARN that doesn't exist, and the eval pipeline's CreateJob fails with ResourceNotFound.

The bare-string / JumpStart-ID path does not have this bug — it already resolves via _resolve_jumpstart_model(..., get_sagemaker_hub_name()), which honors SAGEMAKER_HUB_NAME. Only the model-package / attached-job path ignored it.

This affects any customer who fine-tunes an MTRL model whose base model lives in a private/custom hub and then evaluates it via attach() or a ModelPackage — not just integration tests.

Fix

In _resolve_model_package_object, reconstruct the base-model ARN in the hub the model was actually customized against:

• Use get_sagemaker_hub_name() (honors SAGEMAKER_HUB_NAME, still defaults to SageMakerPublicHub) instead of a hardcoded string.
• Use the correct ARN owner: account-less aws for the public hub, the model package's own account for a private hub.

Verification

• Reproduced against a live account: resolver now yields
  arn:aws:sagemaker:us-west-2:<acct>:hub-content/sdktest/Model/mock-oss-test/0.0.1,
  and describe-hub-content confirms that ARN is Available — so the pipeline's CreateJob finds the base model.
• Unit tests: tests/unit/train/common_utils/test_model_resolution.py — 38 passed.
  • Added test_resolve_arn_construct_hub_content_arn_private_hub (private-hub case).
  • Pinned the existing test_resolve_arn_construct_hub_content_arn to the default public hub.

Notes

• This unblocks the gpu_intensive nightly integ test test_evaluate_with_attached_trainer, which currently fails because mock-oss-test lives only in the private sdktest hub.
• A complementary backend improvement (populate HubContentArn in the model package BaseModel block at registration time) would let the SDK skip reconstruction entirely — the SDK already prefers hub_content_arn when present. That is out of scope for this SDK change.