fix(serve): remove IC data-source collapse hack from recommendation deploy by ZealSV · Pull Request #6101 · aws/sagemaker-python-sdk

ZealSV · 2026-07-24T20:28:57Z

Inference Components now support AdditionalModelDataSources natively (kernel tuning / speculative decoding channels), so _deploy_recommendation no longer needs to collapse an optimized recommendation's base_model + draft channels into a single primary ModelDataSource. Deploy the recommendation's ModelPackage directly and let the hosting stack resolve the channels.

Removes the base_model-promotion / OPTION_SPECULATIVE_DRAFT_MODEL rewiring block and its now-unused shape imports (AdditionalModelDataSource, ModelDataSource, S3ModelDataSource, SPECULATIVE_DRAFT_MODEL). Updates the speculative-decoding unit tests to assert the ModelPackage pass-through.

Issue #, if available:

Description of changes:

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

…eploy Inference Components now support AdditionalModelDataSources natively (kernel tuning / speculative decoding channels), so _deploy_recommendation no longer needs to collapse an optimized recommendation's base_model + draft channels into a single primary ModelDataSource. Deploy the recommendation's ModelPackage directly and let the hosting stack resolve the channels. Removes the base_model-promotion / OPTION_SPECULATIVE_DRAFT_MODEL rewiring block and its now-unused shape imports (AdditionalModelDataSource, ModelDataSource, S3ModelDataSource, SPECULATIVE_DRAFT_MODEL). Updates the speculative-decoding unit tests to assert the ModelPackage pass-through.

…onent Adds an end-to-end integration test that builds a model carrying speculative-decoding / kernel-tuning AdditionalModelDataSources (base + draft channels) and deploys it as an Inference Component via ModelBuilder.deploy(inference_config=ResourceRequirements(...)). Asserts the Inference Component reaches InService and that the deployed model still carries the additional sources (guards against a silent client-side collapse). Marked slow_test + gpu_intensive.

The test built the IC model via ModelBuilder(model_path=<s3_uri>), which raises 'Cannot detect required model or inference spec' — a raw S3 path is not a buildable model spec. Build via ModelBuilder.from_jumpstart_config so the container/framework resolve, then attach additional_model_data_sources (carried onto the model through _prepare_container_def_base) before deploying as an Inference Component. Also switch the IC instance to g4dn.xlarge (a GPU is needed for the vLLM/LMI container, not for the 0.6B size) and trim the module docstring.

…down deploy(wait=True) waits for the endpoint, but the Inference Component is created with wait=False, so the IC can still be Creating when deploy() returns. Add _wait_for_ic_terminal to poll the IC to InService/Failed before the status assertion, and wait it out of Creating before teardown (an IC in Creating cannot be deleted, which would strand its GPU endpoint).

ZealSV · 2026-07-24T20:28:57Z

Inference Components now support AdditionalModelDataSources natively (kernel tuning / speculative decoding channels), so _deploy_recommendation no longer needs to collapse an optimized recommendation's base_model + draft channels into a single primary ModelDataSource. Deploy the recommendation's ModelPackage directly and let the hosting stack resolve the channels.

Removes the base_model-promotion / OPTION_SPECULATIVE_DRAFT_MODEL rewiring block and its now-unused shape imports (AdditionalModelDataSource, ModelDataSource, S3ModelDataSource, SPECULATIVE_DRAFT_MODEL). Updates the speculative-decoding unit tests to assert the ModelPackage pass-through.

Issue #, if available:

Description of changes:

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

…eploy Inference Components now support AdditionalModelDataSources natively (kernel tuning / speculative decoding channels), so _deploy_recommendation no longer needs to collapse an optimized recommendation's base_model + draft channels into a single primary ModelDataSource. Deploy the recommendation's ModelPackage directly and let the hosting stack resolve the channels. Removes the base_model-promotion / OPTION_SPECULATIVE_DRAFT_MODEL rewiring block and its now-unused shape imports (AdditionalModelDataSource, ModelDataSource, S3ModelDataSource, SPECULATIVE_DRAFT_MODEL). Updates the speculative-decoding unit tests to assert the ModelPackage pass-through.

…onent Adds an end-to-end integration test that builds a model carrying speculative-decoding / kernel-tuning AdditionalModelDataSources (base + draft channels) and deploys it as an Inference Component via ModelBuilder.deploy(inference_config=ResourceRequirements(...)). Asserts the Inference Component reaches InService and that the deployed model still carries the additional sources (guards against a silent client-side collapse). Marked slow_test + gpu_intensive.

The test built the IC model via ModelBuilder(model_path=<s3_uri>), which raises 'Cannot detect required model or inference spec' — a raw S3 path is not a buildable model spec. Build via ModelBuilder.from_jumpstart_config so the container/framework resolve, then attach additional_model_data_sources (carried onto the model through _prepare_container_def_base) before deploying as an Inference Component. Also switch the IC instance to g4dn.xlarge (a GPU is needed for the vLLM/LMI container, not for the 0.6B size) and trim the module docstring.

…down deploy(wait=True) waits for the endpoint, but the Inference Component is created with wait=False, so the IC can still be Creating when deploy() returns. Add _wait_for_ic_terminal to poll the IC to InService/Failed before the status assertion, and wait it out of Creating before teardown (an IC in Creating cannot be deleted, which would strand its GPU endpoint).

ZealSV · 2026-07-24T20:28:57Z

Inference Components now support AdditionalModelDataSources natively (kernel tuning / speculative decoding channels), so _deploy_recommendation no longer needs to collapse an optimized recommendation's base_model + draft channels into a single primary ModelDataSource. Deploy the recommendation's ModelPackage directly and let the hosting stack resolve the channels.

Removes the base_model-promotion / OPTION_SPECULATIVE_DRAFT_MODEL rewiring block and its now-unused shape imports (AdditionalModelDataSource, ModelDataSource, S3ModelDataSource, SPECULATIVE_DRAFT_MODEL). Updates the speculative-decoding unit tests to assert the ModelPackage pass-through.

Issue #, if available:

Description of changes:

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

…eploy Inference Components now support AdditionalModelDataSources natively (kernel tuning / speculative decoding channels), so _deploy_recommendation no longer needs to collapse an optimized recommendation's base_model + draft channels into a single primary ModelDataSource. Deploy the recommendation's ModelPackage directly and let the hosting stack resolve the channels. Removes the base_model-promotion / OPTION_SPECULATIVE_DRAFT_MODEL rewiring block and its now-unused shape imports (AdditionalModelDataSource, ModelDataSource, S3ModelDataSource, SPECULATIVE_DRAFT_MODEL). Updates the speculative-decoding unit tests to assert the ModelPackage pass-through.

…onent Adds an end-to-end integration test that builds a model carrying speculative-decoding / kernel-tuning AdditionalModelDataSources (base + draft channels) and deploys it as an Inference Component via ModelBuilder.deploy(inference_config=ResourceRequirements(...)). Asserts the Inference Component reaches InService and that the deployed model still carries the additional sources (guards against a silent client-side collapse). Marked slow_test + gpu_intensive.

The test built the IC model via ModelBuilder(model_path=<s3_uri>), which raises 'Cannot detect required model or inference spec' — a raw S3 path is not a buildable model spec. Build via ModelBuilder.from_jumpstart_config so the container/framework resolve, then attach additional_model_data_sources (carried onto the model through _prepare_container_def_base) before deploying as an Inference Component. Also switch the IC instance to g4dn.xlarge (a GPU is needed for the vLLM/LMI container, not for the 0.6B size) and trim the module docstring.

…down deploy(wait=True) waits for the endpoint, but the Inference Component is created with wait=False, so the IC can still be Creating when deploy() returns. Add _wait_for_ic_terminal to poll the IC to InService/Failed before the status assertion, and wait it out of Creating before teardown (an IC in Creating cannot be deleted, which would strand its GPU endpoint).

ZealSV · 2026-07-24T20:28:57Z

Inference Components now support AdditionalModelDataSources natively (kernel tuning / speculative decoding channels), so _deploy_recommendation no longer needs to collapse an optimized recommendation's base_model + draft channels into a single primary ModelDataSource. Deploy the recommendation's ModelPackage directly and let the hosting stack resolve the channels.

Removes the base_model-promotion / OPTION_SPECULATIVE_DRAFT_MODEL rewiring block and its now-unused shape imports (AdditionalModelDataSource, ModelDataSource, S3ModelDataSource, SPECULATIVE_DRAFT_MODEL). Updates the speculative-decoding unit tests to assert the ModelPackage pass-through.

Issue #, if available:

Description of changes:

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

…eploy Inference Components now support AdditionalModelDataSources natively (kernel tuning / speculative decoding channels), so _deploy_recommendation no longer needs to collapse an optimized recommendation's base_model + draft channels into a single primary ModelDataSource. Deploy the recommendation's ModelPackage directly and let the hosting stack resolve the channels. Removes the base_model-promotion / OPTION_SPECULATIVE_DRAFT_MODEL rewiring block and its now-unused shape imports (AdditionalModelDataSource, ModelDataSource, S3ModelDataSource, SPECULATIVE_DRAFT_MODEL). Updates the speculative-decoding unit tests to assert the ModelPackage pass-through.

…onent Adds an end-to-end integration test that builds a model carrying speculative-decoding / kernel-tuning AdditionalModelDataSources (base + draft channels) and deploys it as an Inference Component via ModelBuilder.deploy(inference_config=ResourceRequirements(...)). Asserts the Inference Component reaches InService and that the deployed model still carries the additional sources (guards against a silent client-side collapse). Marked slow_test + gpu_intensive.

The test built the IC model via ModelBuilder(model_path=<s3_uri>), which raises 'Cannot detect required model or inference spec' — a raw S3 path is not a buildable model spec. Build via ModelBuilder.from_jumpstart_config so the container/framework resolve, then attach additional_model_data_sources (carried onto the model through _prepare_container_def_base) before deploying as an Inference Component. Also switch the IC instance to g4dn.xlarge (a GPU is needed for the vLLM/LMI container, not for the 0.6B size) and trim the module docstring.

…down deploy(wait=True) waits for the endpoint, but the Inference Component is created with wait=False, so the IC can still be Creating when deploy() returns. Add _wait_for_ic_terminal to poll the IC to InService/Failed before the status assertion, and wait it out of Creating before teardown (an IC in Creating cannot be deleted, which would strand its GPU endpoint).

ZealSV · 2026-07-24T20:28:57Z

Inference Components now support AdditionalModelDataSources natively (kernel tuning / speculative decoding channels), so _deploy_recommendation no longer needs to collapse an optimized recommendation's base_model + draft channels into a single primary ModelDataSource. Deploy the recommendation's ModelPackage directly and let the hosting stack resolve the channels.

Removes the base_model-promotion / OPTION_SPECULATIVE_DRAFT_MODEL rewiring block and its now-unused shape imports (AdditionalModelDataSource, ModelDataSource, S3ModelDataSource, SPECULATIVE_DRAFT_MODEL). Updates the speculative-decoding unit tests to assert the ModelPackage pass-through.

Issue #, if available:

Description of changes:

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

…eploy Inference Components now support AdditionalModelDataSources natively (kernel tuning / speculative decoding channels), so _deploy_recommendation no longer needs to collapse an optimized recommendation's base_model + draft channels into a single primary ModelDataSource. Deploy the recommendation's ModelPackage directly and let the hosting stack resolve the channels. Removes the base_model-promotion / OPTION_SPECULATIVE_DRAFT_MODEL rewiring block and its now-unused shape imports (AdditionalModelDataSource, ModelDataSource, S3ModelDataSource, SPECULATIVE_DRAFT_MODEL). Updates the speculative-decoding unit tests to assert the ModelPackage pass-through.

…onent Adds an end-to-end integration test that builds a model carrying speculative-decoding / kernel-tuning AdditionalModelDataSources (base + draft channels) and deploys it as an Inference Component via ModelBuilder.deploy(inference_config=ResourceRequirements(...)). Asserts the Inference Component reaches InService and that the deployed model still carries the additional sources (guards against a silent client-side collapse). Marked slow_test + gpu_intensive.

The test built the IC model via ModelBuilder(model_path=<s3_uri>), which raises 'Cannot detect required model or inference spec' — a raw S3 path is not a buildable model spec. Build via ModelBuilder.from_jumpstart_config so the container/framework resolve, then attach additional_model_data_sources (carried onto the model through _prepare_container_def_base) before deploying as an Inference Component. Also switch the IC instance to g4dn.xlarge (a GPU is needed for the vLLM/LMI container, not for the 0.6B size) and trim the module docstring.

…down deploy(wait=True) waits for the endpoint, but the Inference Component is created with wait=False, so the IC can still be Creating when deploy() returns. Add _wait_for_ic_terminal to poll the IC to InService/Failed before the status assertion, and wait it out of Creating before teardown (an IC in Creating cannot be deleted, which would strand its GPU endpoint).

ZealSV · 2026-07-24T20:28:57Z

Inference Components now support AdditionalModelDataSources natively (kernel tuning / speculative decoding channels), so _deploy_recommendation no longer needs to collapse an optimized recommendation's base_model + draft channels into a single primary ModelDataSource. Deploy the recommendation's ModelPackage directly and let the hosting stack resolve the channels.

Removes the base_model-promotion / OPTION_SPECULATIVE_DRAFT_MODEL rewiring block and its now-unused shape imports (AdditionalModelDataSource, ModelDataSource, S3ModelDataSource, SPECULATIVE_DRAFT_MODEL). Updates the speculative-decoding unit tests to assert the ModelPackage pass-through.

Issue #, if available:

Description of changes:

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

…eploy Inference Components now support AdditionalModelDataSources natively (kernel tuning / speculative decoding channels), so _deploy_recommendation no longer needs to collapse an optimized recommendation's base_model + draft channels into a single primary ModelDataSource. Deploy the recommendation's ModelPackage directly and let the hosting stack resolve the channels. Removes the base_model-promotion / OPTION_SPECULATIVE_DRAFT_MODEL rewiring block and its now-unused shape imports (AdditionalModelDataSource, ModelDataSource, S3ModelDataSource, SPECULATIVE_DRAFT_MODEL). Updates the speculative-decoding unit tests to assert the ModelPackage pass-through.

…onent Adds an end-to-end integration test that builds a model carrying speculative-decoding / kernel-tuning AdditionalModelDataSources (base + draft channels) and deploys it as an Inference Component via ModelBuilder.deploy(inference_config=ResourceRequirements(...)). Asserts the Inference Component reaches InService and that the deployed model still carries the additional sources (guards against a silent client-side collapse). Marked slow_test + gpu_intensive.

The test built the IC model via ModelBuilder(model_path=<s3_uri>), which raises 'Cannot detect required model or inference spec' — a raw S3 path is not a buildable model spec. Build via ModelBuilder.from_jumpstart_config so the container/framework resolve, then attach additional_model_data_sources (carried onto the model through _prepare_container_def_base) before deploying as an Inference Component. Also switch the IC instance to g4dn.xlarge (a GPU is needed for the vLLM/LMI container, not for the 0.6B size) and trim the module docstring.

…down deploy(wait=True) waits for the endpoint, but the Inference Component is created with wait=False, so the IC can still be Creating when deploy() returns. Add _wait_for_ic_terminal to poll the IC to InService/Failed before the status assertion, and wait it out of Creating before teardown (an IC in Creating cannot be deleted, which would strand its GPU endpoint).

ZealSV · 2026-07-24T20:28:57Z

Inference Components now support AdditionalModelDataSources natively (kernel tuning / speculative decoding channels), so _deploy_recommendation no longer needs to collapse an optimized recommendation's base_model + draft channels into a single primary ModelDataSource. Deploy the recommendation's ModelPackage directly and let the hosting stack resolve the channels.

Removes the base_model-promotion / OPTION_SPECULATIVE_DRAFT_MODEL rewiring block and its now-unused shape imports (AdditionalModelDataSource, ModelDataSource, S3ModelDataSource, SPECULATIVE_DRAFT_MODEL). Updates the speculative-decoding unit tests to assert the ModelPackage pass-through.

Issue #, if available:

Description of changes:

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

…eploy Inference Components now support AdditionalModelDataSources natively (kernel tuning / speculative decoding channels), so _deploy_recommendation no longer needs to collapse an optimized recommendation's base_model + draft channels into a single primary ModelDataSource. Deploy the recommendation's ModelPackage directly and let the hosting stack resolve the channels. Removes the base_model-promotion / OPTION_SPECULATIVE_DRAFT_MODEL rewiring block and its now-unused shape imports (AdditionalModelDataSource, ModelDataSource, S3ModelDataSource, SPECULATIVE_DRAFT_MODEL). Updates the speculative-decoding unit tests to assert the ModelPackage pass-through.

…onent Adds an end-to-end integration test that builds a model carrying speculative-decoding / kernel-tuning AdditionalModelDataSources (base + draft channels) and deploys it as an Inference Component via ModelBuilder.deploy(inference_config=ResourceRequirements(...)). Asserts the Inference Component reaches InService and that the deployed model still carries the additional sources (guards against a silent client-side collapse). Marked slow_test + gpu_intensive.

The test built the IC model via ModelBuilder(model_path=<s3_uri>), which raises 'Cannot detect required model or inference spec' — a raw S3 path is not a buildable model spec. Build via ModelBuilder.from_jumpstart_config so the container/framework resolve, then attach additional_model_data_sources (carried onto the model through _prepare_container_def_base) before deploying as an Inference Component. Also switch the IC instance to g4dn.xlarge (a GPU is needed for the vLLM/LMI container, not for the 0.6B size) and trim the module docstring.

…down deploy(wait=True) waits for the endpoint, but the Inference Component is created with wait=False, so the IC can still be Creating when deploy() returns. Add _wait_for_ic_terminal to poll the IC to InService/Failed before the status assertion, and wait it out of Creating before teardown (an IC in Creating cannot be deleted, which would strand its GPU endpoint).

ZealSV · 2026-07-24T20:28:57Z

Inference Components now support AdditionalModelDataSources natively (kernel tuning / speculative decoding channels), so _deploy_recommendation no longer needs to collapse an optimized recommendation's base_model + draft channels into a single primary ModelDataSource. Deploy the recommendation's ModelPackage directly and let the hosting stack resolve the channels.

Removes the base_model-promotion / OPTION_SPECULATIVE_DRAFT_MODEL rewiring block and its now-unused shape imports (AdditionalModelDataSource, ModelDataSource, S3ModelDataSource, SPECULATIVE_DRAFT_MODEL). Updates the speculative-decoding unit tests to assert the ModelPackage pass-through.

Issue #, if available:

Description of changes:

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

…eploy Inference Components now support AdditionalModelDataSources natively (kernel tuning / speculative decoding channels), so _deploy_recommendation no longer needs to collapse an optimized recommendation's base_model + draft channels into a single primary ModelDataSource. Deploy the recommendation's ModelPackage directly and let the hosting stack resolve the channels. Removes the base_model-promotion / OPTION_SPECULATIVE_DRAFT_MODEL rewiring block and its now-unused shape imports (AdditionalModelDataSource, ModelDataSource, S3ModelDataSource, SPECULATIVE_DRAFT_MODEL). Updates the speculative-decoding unit tests to assert the ModelPackage pass-through.

…onent Adds an end-to-end integration test that builds a model carrying speculative-decoding / kernel-tuning AdditionalModelDataSources (base + draft channels) and deploys it as an Inference Component via ModelBuilder.deploy(inference_config=ResourceRequirements(...)). Asserts the Inference Component reaches InService and that the deployed model still carries the additional sources (guards against a silent client-side collapse). Marked slow_test + gpu_intensive.

The test built the IC model via ModelBuilder(model_path=<s3_uri>), which raises 'Cannot detect required model or inference spec' — a raw S3 path is not a buildable model spec. Build via ModelBuilder.from_jumpstart_config so the container/framework resolve, then attach additional_model_data_sources (carried onto the model through _prepare_container_def_base) before deploying as an Inference Component. Also switch the IC instance to g4dn.xlarge (a GPU is needed for the vLLM/LMI container, not for the 0.6B size) and trim the module docstring.

…down deploy(wait=True) waits for the endpoint, but the Inference Component is created with wait=False, so the IC can still be Creating when deploy() returns. Add _wait_for_ic_terminal to poll the IC to InService/Failed before the status assertion, and wait it out of Creating before teardown (an IC in Creating cannot be deleted, which would strand its GPU endpoint).

ZealSV had a problem deploying to manual-approval July 24, 2026 20:29 — with GitHub Actions Error

Lokiiiiii suggested changes Jul 24, 2026
View reviewed changes

Comment threadsagemaker-serve/tests/unit/test_ai_inference_recommender/test_model_builder_recommendations.py
Comment threadsagemaker-serve/src/sagemaker/serve/model_builder.py

ZealSV force-pushed the remove-ic-sdkt-hack branch from a322ca1 to e161c32Compare July 27, 2026 18:32

ZealSV had a problem deploying to manual-approval July 27, 2026 18:33 — with GitHub Actions Error

ZealSV had a problem deploying to manual-approval July 27, 2026 20:35 — with GitHub Actions Error

ZealSV force-pushed the remove-ic-sdkt-hack branch from b68d882 to 81bee88Compare July 27, 2026 20:37

ZealSV had a problem deploying to manual-approval July 27, 2026 20:37 — with GitHub Actions Error

Merge branch 'master' into remove-ic-sdkt-hack
aa6f339

ZealSV had a problem deploying to manual-approval July 27, 2026 20:45 — with GitHub Actions Error

Merge branch 'master' into remove-ic-sdkt-hack
db82399

ZealSV temporarily deployed to manual-approval July 29, 2026 05:58 — with GitHub Actions Inactive

ZealSVand others added 2 commits July 29, 2026 13:17

Merge branch 'master' into remove-ic-sdkt-hack
d632b80

ZealSV temporarily deployed to manual-approval July 29, 2026 20:26 — with GitHub Actions Inactive

ZealSV had a problem deploying to manual-approval July 29, 2026 20:26 — with GitHub Actions Error

ZealSV had a problem deploying to manual-approval July 29, 2026 20:27 — with GitHub Actions Error

ZealSV had a problem deploying to manual-approval July 29, 2026 22:08 — with GitHub Actions Error

ZealSV temporarily deployed to manual-approval July 29, 2026 22:08 — with GitHub Actions Inactive

ZealSV temporarily deployed to manual-approval July 29, 2026 22:30 — with GitHub Actions Inactive

ZealSV temporarily deployed to manual-approval July 30, 2026 17:47 — with GitHub Actions Inactive

lucasjia-aws approved these changes Jul 31, 2026
View reviewed changes

lucasjia-aws merged commit 9ff3e5f into aws:masterJul 31, 2026
25 of 34 checks passed

github-actionsBot mentioned this pull request Aug 10, 2026
Release 3.19.0 #6173
Merged