fix(serve): private hub deploys attach HubAccessConfig and support aliased content names by e-davidson · Pull Request #6035 · aws/sagemaker-python-sdk

e-davidson · 2026-07-16T17:33:31Z

Scope update (2026-07-16)

#6036 (merged after this PR opened) independently fixed the model_reference_arn propagation defect with the same change this PR originally carried. This branch has been merged with master: the duplicate propagation block was dropped in favor of the upstream one. This PR now provides the remaining fix plus the test coverage that would have caught the original regression.

Issue

Deploying a JumpStart model from a private hub via ModelBuilder.from_jumpstart_config(hub_name=...) was broken in two ways on v3.16.0:

Defect A (fixed upstream in #6036): HubAccessConfig never attached to CreateModel.
get_init_kwargs resolved model_reference_arn, but the builder never copied it, so _prepare_container_def_base omitted S3DataSource.HubAccessConfig.HubContentArn and the execution role needed s3:GetObject on the public jumpstart-cache-prod-* bucket.

Defect B (fixed here): aliased hub content names unsupported.
CreateHubContentReference accepts a custom HubContentName, but the SDK always looks up hub content by model_id. A reference named differently from the public model id fails with ResourceNotFound.

Changes in this PR

Fix: optional hub_content_name on JumpStartConfig / from_jumpstart_config; when set, hub content is resolved by its actual name.
Unit tests (tests/unit/test_private_hub_artifact_resolution.py, extends the file from fix: ModelBuilder resolves private hub artifacts correctly #5985):
- model_reference_arn propagation regression test (guards the Fix private hub #6036 fix; Fix private hub #6036's own tests assert a subset)
- CreateModel payload chain test: drives the real unmocked path _build_for_jumpstart -> _prepare_container_def_base and asserts the exact container definition dict carries HubAccessConfig (present for hub builds, absent for public). This is the assertion that would have caught the regression that shipped in v3.15.1-v3.16.0.
- hub_content_name lookup + config threading tests
- Shared setup via module-level _build_jumpstart_builder() helper, matching package conventions
Integration tests (tests/integ/test_private_hub_artifact_resolution.py, slow_test):
- test_deploy_with_no_s3_execution_role: self-provisions an execution role with zero S3 permissions, deploys from a private hub, asserts the created Model carries HubAccessConfig. Encodes the service-side brokered-access contract that unit tests cannot reach.
- test_deploy_with_aliased_hub_content_name: deploys a reference named differently from the model_id via hub_content_name.
- Both reuse the existing self-provisioning hub fixture; full teardown (hub, references, role, endpoints).

Verification (all red -> green)

Layer	Baseline v3.16.0	With fix
Unit (17 tests)	4 discriminating tests FAIL	17/17 pass
Integ, live us-east-1	2 FAIL (missing HubAccessConfig assertion; pydantic rejection of `hub_content_name`)	2 pass (38s), both with the zero-S3 role
Post-merge with #6036	n/a	17/17 unit tests pass, single propagation block

Live runs used a real private hub + ModelReference and an IAM role with no S3 permissions; endpoints and resources were cleaned up.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

…iased content names Two remaining defects after PR aws#5985: 1. HubAccessConfig missing from CreateModel. _build_for_jumpstart resolved model_reference_arn via get_init_kwargs but never copied it onto the builder, so _prepare_container_def_base passed None to container_def() and the S3DataSource had no HubAccessConfig. The execution role was forced to need s3:GetObject on the public jumpstart-cache-prod bucket. Fix: propagate init_kwargs.model_reference_arn to self.model_reference_arn. 2. Hub content references named differently from the public model_id could not be resolved (ResourceNotFound). Fix: add hub_content_name to JumpStartConfig and use it as the hub lookup identifier in the build path when set. Verified E2E against a live private hub with an execution role that has zero S3 permissions: both matching-name and aliased-name deploys now succeed, with SageMaker brokering artifact access through the hub content reference. Pattern follows master-v2, where Model.prepare_container_def passes model_reference_arn into sagemaker.container_def(). --- X-AI-Prompt: create a new workspace, pull the pysdk, reproduce in our workspace and fix both bugs, the name and that it's not configured correctly. Look at master-v2 to fix this. X-AI-Tool: meshclaw

…ased hub content names Add mock-based unit tests covering the two private hub defects fixed in the previous commit. Verified each discriminating test fails on the v3.16.0 baseline and passes with the fix: - test_model_reference_arn_propagated_from_init_kwargs: the ARN resolved by get_init_kwargs must land on the builder so container_def attaches S3DataSource.HubAccessConfig.HubContentArn (fails on baseline with model_reference_arn=None) - test_hub_content_name_used_as_lookup_model_id: aliased references resolve by hub_content_name (fails on baseline) - test_from_jumpstart_config_threads_hub_content_name: config threading (fails on baseline, JumpStartConfig rejects the field) Plus negative tests (public catalog path unchanged, no HubAccessConfig without a model_reference_arn) and pure-function coverage of container_def HubAccessConfig injection. No AWS calls. --- X-AI-Prompt: Write the mock unit tests, verify they fail without fix + pass with fix, push to PR X-AI-Tool: meshclaw

…sConfig Close the gap between the propagation test and the pure container_def test: drive the real unmocked path from _build_for_jumpstart through _prepare_container_def_base and assert the exact container definition dict sent to the CreateModel API contains S3DataSource.HubAccessConfig.HubContentArn for private hub builds, and omits it for public catalog builds. This is the payload-shape assertion that encodes the customer-visible contract: with HubAccessConfig present, SageMaker brokers model data access through the hub content reference and the execution role needs no s3:GetObject on the public JumpStart cache bucket. Verified: fails on the v3.16.0 baseline (HubAccessConfig absent), passes with the fix. No AWS calls. --- X-AI-Prompt: Did we make sure there are no calls to s3 when using a model reference? what about that we're passing in the intended call to sagemaker create model? X-AI-Tool: meshclaw

Replace cross-TestCase borrowing (instantiating TestModelReferenceArnPropagation inside other test classes to reuse its _build method) with a module-level _build_jumpstart_builder() helper, matching the package convention for shared builder setup (see _create_mock_builder in test_model_builder_servers_hf_model_id.py and _builder in test_bedrock_model_builder.py). No behavior change. Re-verified both directions: 4 discriminating tests fail on the v3.16.0 baseline, all 17 pass with the fix. --- X-AI-Prompt: Refactor to module-level helper, re-verify both directions, push X-AI-Tool: meshclaw

…ntent name Add two slow_test integration tests encoding the service-side contract the unit tests cannot cover: - test_deploy_with_no_s3_execution_role: creates an execution role with NO S3 permissions (ECR + CloudWatch Logs + hub read only), deploys from the private hub, and asserts the created Model resource carries S3DataSource.HubAccessConfig. On v3.16.0 this fails at CreateModel with 'Could not access model data at s3://jumpstart-cache-prod-...'. - test_deploy_with_aliased_hub_content_name: deploys a ModelReference whose HubContentName differs from the public model_id via the new JumpStartConfig.hub_content_name. On v3.16.0 this fails with ResourceNotFound. Both reuse the existing self-provisioning private_hub fixture (create hub + reference, tear down after) and clean up endpoints, endpoint configs, and the IAM role. Mirrors the live E2E verification performed against this branch (both scenarios deployed successfully with the zero-S3 role in us-east-1). --- X-AI-Prompt: Now where's the e2e test? X-AI-Tool: meshclaw

Live verification of both integ tests in both directions: - With fix: 2 passed (38s) - Baseline v3.16.0: 2 failed as intended. no_s3 test failed at the HubAccessConfig assertion (CreateModel accepted the gated gemma-2b request without HubAccessConfig; the endpoint then failed asynchronously). Aliased test failed at JumpStartConfig validation (hub_content_name field does not exist on baseline). Corrections from the live run: the no_s3 docstring claimed a synchronous CreateModel ValidationException, but the deterministic failure signal is the HubAccessConfig assertion. Cleanup except block now logs failures instead of silently swallowing them (three Failed-status endpoints leaked during verification runs and were deleted manually). --- X-AI-Prompt: Run both directions live now X-AI-Tool: meshclaw

Upstream aws#6036 independently fixed the model_reference_arn propagation (Defect A) with the same 6-line fix. Resolution: kept upstream's propagation block as canonical (removed our duplicate at the earlier location), kept our hub_content_name lookup fix (Defect B, not covered upstream), and kept our unit test file as the superset (upstream's 2 tests assert a subset of our TestModelReferenceArnPropagation). Verified post-merge: 17/17 unit tests pass, single propagation block, no conflict markers.

e-davidson · 2026-07-17T00:09:30Z

Superseded by #6039. This PR predated #6036, which independently shipped the same model_reference_arn propagation fix, leaving this branch with a redundant duplicate commit and a merge commit. #6039 is a clean cut from current master carrying only the remaining fix (aliased hub_content_name support) and the full regression test coverage (unit + CreateModel payload chain + live-verified E2E integ tests).

e-davidson · 2026-07-16T17:33:31Z