fix: Add additional dependencies for ModelTrainer (5668) - #5731

Draft
aviruthen wants to merge 2 commits into
aws:masterfrom
aviruthen:fix/add-additional-dependencies-for-modeltrainer-5668
Draft

fix: Add additional dependencies for ModelTrainer (5668)#5731
aviruthen wants to merge 2 commits into
aws:masterfrom
aviruthen:fix/add-additional-dependencies-for-modeltrainer-5668

Conversation

@aviruthen

@aviruthenaviruthen commented Apr 7, 2026

Copy link
Copy Markdown
Collaborator

Description

The V2 Estimator's dependencies parameter allows users to specify a list of local directory/file paths containing additional libraries that are uploaded to S3, mounted in the training container, and added to PYTHONPATH. The V3 ModelTrainer's SourceCode config has no equivalent field. The fix requires: (1) Adding a dependencies field to the SourceCode config class, (2) Adding a new constant for the dependencies channel and container path, (3) In _create_training_job_args(), creating input data channels for each dependency and copying them into the sm_drivers temp dir, (4) Adding an INSTALL_DEPENDENCIES template that adds dependency paths to PYTHONPATH and sys.path, (5) Including the dependencies installation step in the generated train script, and (6) Validating dependency paths exist during _validate_source_code().

Related Issue

Related issue: 5668

Changes Made

  • sagemaker-core/src/sagemaker/core/training/configs.py
  • sagemaker-train/src/sagemaker/train/constants.py
  • sagemaker-train/src/sagemaker/train/templates.py
  • sagemaker-train/src/sagemaker/train/model_trainer.py
  • sagemaker-train/tests/unit/train/test_model_trainer_dependencies.py

AI-Generated PR

This PR was automatically generated by the PySDK Issue Agent.

  • Confidence score: 85%
  • Classification: bug
  • SDK version target: V3

Merge Checklist

  • Changes are backward compatible
  • Commit message follows prefix: description format
  • Unit tests added/updated
  • Integration tests added (if applicable)
  • Documentation updated (if applicable)

sagemaker-bot
sagemaker-bot reviewed Apr 7, 2026
View reviewed changes

@sagemaker-botsagemaker-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 AI Code Review

This PR adds a dependencies field to SourceCode for ModelTrainer, enabling users to specify additional local directories/files to be uploaded and added to PYTHONPATH in the training container. While the feature is well-structured, there are several issues: a resource leak with TemporaryDirectory, the INSTALL_DEPENDENCIES template only handles directories (not files), the PR description mentions sys.path but the implementation doesn't add it, and some tests are weak assertions that don't actually verify the behavior they claim to test.

Comment threadsagemaker-train/src/sagemaker/train/model_trainer.py Outdated
Comment threadsagemaker-train/src/sagemaker/train/templates.py
Comment threadsagemaker-train/src/sagemaker/train/model_trainer.py
Comment threadsagemaker-train/src/sagemaker/train/model_trainer.py Outdated
Comment threadsagemaker-train/src/sagemaker/train/constants.py
Comment threadsagemaker-train/tests/unit/train/test_model_trainer_dependencies.py
Comment threadsagemaker-train/tests/unit/train/test_model_trainer_dependencies.py
Comment threadsagemaker-core/src/sagemaker/core/training/configs.py
Comment threadsagemaker-train/src/sagemaker/train/model_trainer.py
Comment threadsagemaker-train/tests/unit/train/test_model_trainer_dependencies.py
@aviruthen

aviruthen commented Apr 7, 2026

Copy link
Copy Markdown
CollaboratorAuthor

🤖 Iteration #1 — Review Comments Addressed

Add dependencies support for ModelTrainer's SourceCode config

Description

When migrating from SageMaker v2 to v3, the V2 Estimator's dependencies parameter allows users to specify a list of local directory paths containing additional libraries that are uploaded to S3, mounted in the training container, and added to PYTHONPATH. The V3 ModelTrainer's SourceCode config had no equivalent field.

This PR adds dependencies support to SourceCode with the following changes:

Changes Made

sagemaker-core/src/sagemaker/core/training/configs.py

  • Added dependencies field to SourceCode config class (list of local directory paths)
  • Fixed PEP 8 blank line formatting

sagemaker-train/src/sagemaker/train/constants.py

  • Added SM_DEPENDENCIES and SM_DEPENDENCIES_CONTAINER_PATH constants for the dependencies channel
  • Fixed missing newline at end of file

sagemaker-train/src/sagemaker/train/templates.py

  • Added INSTALL_DEPENDENCIES template that:
    • Adds dependency directories to PYTHONPATH
    • Installs .whl and .tar.gz files via pip
    • Handles other files by adding the parent directory to PYTHONPATH

sagemaker-train/src/sagemaker/train/model_trainer.py

  • Added _temp_deps_dir private attribute for deterministic cleanup of the dependencies temp directory
  • In _create_training_job_args(), creates input data channels for dependencies by copying them to a temp dir preserving basenames
  • Includes the INSTALL_DEPENDENCIES step in the generated train script when dependencies are present
  • Validates that dependency paths exist and are directories during _validate_source_code()
  • Cleans up _temp_deps_dir in destructor and after training completes

sagemaker-train/tests/unit/train/test_model_trainer_dependencies.py

  • Updated to use from __future__ import annotations per V3 conventions
  • Changed fixture scope from module to function for proper test isolation
  • Added test for rejecting file dependencies (only directories allowed)
  • Fixed weak tests to actually verify train script content and temp directory structure

Design Decisions

  • Directories only: Dependencies are restricted to directories (matching V2 Estimator behavior), not individual files
  • PYTHONPATH approach: Dependencies are added to PYTHONPATH via the bash train script, which is sufficient since Python processes inherit environment variables
  • Dependencies before requirements: Dependencies are set up before requirements.txt installation, matching the logical ordering where dependencies should be available first

Comments reviewed: 34
Files modified:sagemaker-core/src/sagemaker/core/training/configs.py, sagemaker-train/src/sagemaker/train/constants.py, sagemaker-train/src/sagemaker/train/model_trainer.py, sagemaker-train/src/sagemaker/train/templates.py, sagemaker-train/tests/unit/train/test_model_trainer_dependencies.py

  • sagemaker-core/src/sagemaker/core/training/configs.py: Add blank line before OutputDataConfig class definition per PEP 8
  • sagemaker-train/src/sagemaker/train/constants.py: Ensure file ends with newline after SM_RECIPE_CONTAINER_PATH
  • sagemaker-train/src/sagemaker/train/model_trainer.py: Fix resource leak for deps temp dir, restrict dependencies to directories only, ensure shutil import exists, and add _temp_deps_dir as instance attribute with cleanup
  • sagemaker-train/src/sagemaker/train/templates.py: Improve INSTALL_DEPENDENCIES template to handle both directories and .whl/.tar.gz files via pip install
  • sagemaker-train/tests/unit/train/test_model_trainer_dependencies.py: Fix weak tests, change fixture scope, use proper future import, and add meaningful assertions

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@sagemaker-botsagemaker-botsagemaker-bot left review comments

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@aviruthen@sagemaker-bot
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all 
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '

Skip to content
fix: Add additional dependencies for ModelTrainer (5668) - #5731
Draft
aviruthen wants to merge 2 commits into
aws:masterfrom 
aviruthen:fix/add-additional-dependencies-for-modeltrainer-5668
Draft
fix: Add additional dependencies for ModelTrainer (5668)#5731
aviruthen wants to merge 2 commits into
aws:masterfrom 
aviruthen:fix/add-additional-dependencies-for-modeltrainer-5668
Conversation
@aviruthen
@aviruthenaviruthen
commented
Apr 7, 2026

Copy link


Copy Markdown

Collaborator
Description
The V2 Estimator's dependencies parameter allows users to specify a list of local directory/file paths containing additional libraries that are uploaded to S3, mounted in the training container, and added to PYTHONPATH. The V3 ModelTrainer's SourceCode config has no equivalent field. The fix requires: (1) Adding a dependencies field to the SourceCode config class, (2) Adding a new constant for the dependencies channel and container path, (3) In _create_training_job_args(), creating input data channels for each dependency and copying them into the sm_drivers temp dir, (4) Adding an INSTALL_DEPENDENCIES template that adds dependency paths to PYTHONPATH and sys.path, (5) Including the dependencies installation step in the generated train script, and (6) Validating dependency paths exist during _validate_source_code().
Related Issue
Related issue: 5668
Changes Made
  • sagemaker-core/src/sagemaker/core/training/configs.py
  • sagemaker-train/src/sagemaker/train/constants.py
  • sagemaker-train/src/sagemaker/train/templates.py
  • sagemaker-train/src/sagemaker/train/model_trainer.py
  • sagemaker-train/tests/unit/train/test_model_trainer_dependencies.py
AI-Generated PR
This PR was automatically generated by the PySDK Issue Agent.
  • Confidence score: 85%
  • Classification: bug
  • SDK version target: V3
Merge Checklist
  •  Changes are backward compatible
  •  Commit message follows prefix: description format
  •  Unit tests added/updated
  •  Integration tests added (if applicable)
  •  Documentation updated (if applicable)
sagemaker-bot
sagemaker-bot
reviewed
Apr 7, 2026
View reviewed changes
@sagemaker-botsagemaker-bot
left a comment


Copy link


Copy Markdown

Collaborator
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 AI Code Review
This PR adds a dependencies field to SourceCode for ModelTrainer, enabling users to specify additional local directories/files to be uploaded and added to PYTHONPATH in the training container. While the feature is well-structured, there are several issues: a resource leak with TemporaryDirectory, the INSTALL_DEPENDENCIES template only handles directories (not files), the PR description mentions sys.path but the implementation doesn't add it, and some tests are weak assertions that don't actually verify the behavior they claim to test.
Comment threadsagemaker-train/src/sagemaker/train/model_trainer.py
Outdated

Comment threadsagemaker-train/src/sagemaker/train/templates.py
Comment threadsagemaker-train/src/sagemaker/train/model_trainer.py
Comment threadsagemaker-train/src/sagemaker/train/model_trainer.py
Outdated

Comment threadsagemaker-train/src/sagemaker/train/constants.py
Comment threadsagemaker-train/tests/unit/train/test_model_trainer_dependencies.py
Comment threadsagemaker-train/tests/unit/train/test_model_trainer_dependencies.py
Comment threadsagemaker-core/src/sagemaker/core/training/configs.py
Comment threadsagemaker-train/src/sagemaker/train/model_trainer.py
Comment threadsagemaker-train/tests/unit/train/test_model_trainer_dependencies.py
@aviruthen
aviruthen
commented
Apr 7, 2026

Copy link


Copy Markdown

CollaboratorAuthor
🤖 Iteration #1 — Review Comments Addressed
Add dependencies support for ModelTrainer's SourceCode config
Description
When migrating from SageMaker v2 to v3, the V2 Estimator's dependencies parameter allows users to specify a list of local directory paths containing additional libraries that are uploaded to S3, mounted in the training container, and added to PYTHONPATH. The V3 ModelTrainer's SourceCode config had no equivalent field.
This PR adds dependencies support to SourceCode with the following changes:
Changes Made
sagemaker-core/src/sagemaker/core/training/configs.py
  • Added dependencies field to SourceCode config class (list of local directory paths)
  • Fixed PEP 8 blank line formatting
sagemaker-train/src/sagemaker/train/constants.py
  • Added SM_DEPENDENCIES and SM_DEPENDENCIES_CONTAINER_PATH constants for the dependencies channel
  • Fixed missing newline at end of file
sagemaker-train/src/sagemaker/train/templates.py
  • Added INSTALL_DEPENDENCIES template that:

    • Adds dependency directories to PYTHONPATH
    • Installs .whl and .tar.gz files via pip
    • Handles other files by adding the parent directory to PYTHONPATH
sagemaker-train/src/sagemaker/train/model_trainer.py
  • Added _temp_deps_dir private attribute for deterministic cleanup of the dependencies temp directory
  • In _create_training_job_args(), creates input data channels for dependencies by copying them to a temp dir preserving basenames
  • Includes the INSTALL_DEPENDENCIES step in the generated train script when dependencies are present
  • Validates that dependency paths exist and are directories during _validate_source_code()
  • Cleans up _temp_deps_dir in destructor and after training completes
sagemaker-train/tests/unit/train/test_model_trainer_dependencies.py
  • Updated to use from __future__ import annotations per V3 conventions
  • Changed fixture scope from module to function for proper test isolation
  • Added test for rejecting file dependencies (only directories allowed)
  • Fixed weak tests to actually verify train script content and temp directory structure
Design Decisions
  • Directories only: Dependencies are restricted to directories (matching V2 Estimator behavior), not individual files
  • PYTHONPATH approach: Dependencies are added to PYTHONPATH via the bash train script, which is sufficient since Python processes inherit environment variables
  • Dependencies before requirements: Dependencies are set up before requirements.txt installation, matching the logical ordering where dependencies should be available first
Comments reviewed: 34
Files modified:sagemaker-core/src/sagemaker/core/training/configs.py, sagemaker-train/src/sagemaker/train/constants.py, sagemaker-train/src/sagemaker/train/model_trainer.py, sagemaker-train/src/sagemaker/train/templates.py, sagemaker-train/tests/unit/train/test_model_trainer_dependencies.py
  • sagemaker-core/src/sagemaker/core/training/configs.py: Add blank line before OutputDataConfig class definition per PEP 8
  • sagemaker-train/src/sagemaker/train/constants.py: Ensure file ends with newline after SM_RECIPE_CONTAINER_PATH
  • sagemaker-train/src/sagemaker/train/model_trainer.py: Fix resource leak for deps temp dir, restrict dependencies to directories only, ensure shutil import exists, and add _temp_deps_dir as instance attribute with cleanup
  • sagemaker-train/src/sagemaker/train/templates.py: Improve INSTALL_DEPENDENCIES template to handle both directories and .whl/.tar.gz files via pip install
  • sagemaker-train/tests/unit/train/test_model_trainer_dependencies.py: Fix weak tests, change fixture scope, use proper future import, and add meaningful assertions
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@sagemaker-botsagemaker-botsagemaker-bot left review comments

At least 1 approving review is required to merge this pull request.


Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

2 participants

@aviruthen@sagemaker-bot

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix");
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
fix: Add additional dependencies for ModelTrainer (5668) - #5731
Draft
aviruthen wants to merge 2 commits into
aws:masterfrom 
aviruthen:fix/add-additional-dependencies-for-modeltrainer-5668
Draft
fix: Add additional dependencies for ModelTrainer (5668)#5731
aviruthen wants to merge 2 commits into
aws:masterfrom 
aviruthen:fix/add-additional-dependencies-for-modeltrainer-5668
Conversation
@aviruthen
@aviruthenaviruthen
commented
Apr 7, 2026

Copy link


Copy Markdown

Collaborator
Description
The V2 Estimator's dependencies parameter allows users to specify a list of local directory/file paths containing additional libraries that are uploaded to S3, mounted in the training container, and added to PYTHONPATH. The V3 ModelTrainer's SourceCode config has no equivalent field. The fix requires: (1) Adding a dependencies field to the SourceCode config class, (2) Adding a new constant for the dependencies channel and container path, (3) In _create_training_job_args(), creating input data channels for each dependency and copying them into the sm_drivers temp dir, (4) Adding an INSTALL_DEPENDENCIES template that adds dependency paths to PYTHONPATH and sys.path, (5) Including the dependencies installation step in the generated train script, and (6) Validating dependency paths exist during _validate_source_code().
Related Issue
Related issue: 5668
Changes Made
  • sagemaker-core/src/sagemaker/core/training/configs.py
  • sagemaker-train/src/sagemaker/train/constants.py
  • sagemaker-train/src/sagemaker/train/templates.py
  • sagemaker-train/src/sagemaker/train/model_trainer.py
  • sagemaker-train/tests/unit/train/test_model_trainer_dependencies.py
AI-Generated PR
This PR was automatically generated by the PySDK Issue Agent.
  • Confidence score: 85%
  • Classification: bug
  • SDK version target: V3
Merge Checklist
  •  Changes are backward compatible
  •  Commit message follows prefix: description format
  •  Unit tests added/updated
  •  Integration tests added (if applicable)
  •  Documentation updated (if applicable)
sagemaker-bot
sagemaker-bot
reviewed
Apr 7, 2026
View reviewed changes
@sagemaker-botsagemaker-bot
left a comment


Copy link


Copy Markdown

Collaborator
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 AI Code Review
This PR adds a dependencies field to SourceCode for ModelTrainer, enabling users to specify additional local directories/files to be uploaded and added to PYTHONPATH in the training container. While the feature is well-structured, there are several issues: a resource leak with TemporaryDirectory, the INSTALL_DEPENDENCIES template only handles directories (not files), the PR description mentions sys.path but the implementation doesn't add it, and some tests are weak assertions that don't actually verify the behavior they claim to test.
Comment threadsagemaker-train/src/sagemaker/train/model_trainer.py
Outdated

Comment threadsagemaker-train/src/sagemaker/train/templates.py
Comment threadsagemaker-train/src/sagemaker/train/model_trainer.py
Comment threadsagemaker-train/src/sagemaker/train/model_trainer.py
Outdated

Comment threadsagemaker-train/src/sagemaker/train/constants.py
Comment threadsagemaker-train/tests/unit/train/test_model_trainer_dependencies.py
Comment threadsagemaker-train/tests/unit/train/test_model_trainer_dependencies.py
Comment threadsagemaker-core/src/sagemaker/core/training/configs.py
Comment threadsagemaker-train/src/sagemaker/train/model_trainer.py
Comment threadsagemaker-train/tests/unit/train/test_model_trainer_dependencies.py
@aviruthen
aviruthen
commented
Apr 7, 2026

Copy link


Copy Markdown

CollaboratorAuthor
🤖 Iteration #1 — Review Comments Addressed
Add dependencies support for ModelTrainer's SourceCode config
Description
When migrating from SageMaker v2 to v3, the V2 Estimator's dependencies parameter allows users to specify a list of local directory paths containing additional libraries that are uploaded to S3, mounted in the training container, and added to PYTHONPATH. The V3 ModelTrainer's SourceCode config had no equivalent field.
This PR adds dependencies support to SourceCode with the following changes:
Changes Made
sagemaker-core/src/sagemaker/core/training/configs.py
  • Added dependencies field to SourceCode config class (list of local directory paths)
  • Fixed PEP 8 blank line formatting
sagemaker-train/src/sagemaker/train/constants.py
  • Added SM_DEPENDENCIES and SM_DEPENDENCIES_CONTAINER_PATH constants for the dependencies channel
  • Fixed missing newline at end of file
sagemaker-train/src/sagemaker/train/templates.py
  • Added INSTALL_DEPENDENCIES template that:

    • Adds dependency directories to PYTHONPATH
    • Installs .whl and .tar.gz files via pip
    • Handles other files by adding the parent directory to PYTHONPATH
sagemaker-train/src/sagemaker/train/model_trainer.py
  • Added _temp_deps_dir private attribute for deterministic cleanup of the dependencies temp directory
  • In _create_training_job_args(), creates input data channels for dependencies by copying them to a temp dir preserving basenames
  • Includes the INSTALL_DEPENDENCIES step in the generated train script when dependencies are present
  • Validates that dependency paths exist and are directories during _validate_source_code()
  • Cleans up _temp_deps_dir in destructor and after training completes
sagemaker-train/tests/unit/train/test_model_trainer_dependencies.py
  • Updated to use from __future__ import annotations per V3 conventions
  • Changed fixture scope from module to function for proper test isolation
  • Added test for rejecting file dependencies (only directories allowed)
  • Fixed weak tests to actually verify train script content and temp directory structure
Design Decisions
  • Directories only: Dependencies are restricted to directories (matching V2 Estimator behavior), not individual files
  • PYTHONPATH approach: Dependencies are added to PYTHONPATH via the bash train script, which is sufficient since Python processes inherit environment variables
  • Dependencies before requirements: Dependencies are set up before requirements.txt installation, matching the logical ordering where dependencies should be available first
Comments reviewed: 34
Files modified:sagemaker-core/src/sagemaker/core/training/configs.py, sagemaker-train/src/sagemaker/train/constants.py, sagemaker-train/src/sagemaker/train/model_trainer.py, sagemaker-train/src/sagemaker/train/templates.py, sagemaker-train/tests/unit/train/test_model_trainer_dependencies.py
  • sagemaker-core/src/sagemaker/core/training/configs.py: Add blank line before OutputDataConfig class definition per PEP 8
  • sagemaker-train/src/sagemaker/train/constants.py: Ensure file ends with newline after SM_RECIPE_CONTAINER_PATH
  • sagemaker-train/src/sagemaker/train/model_trainer.py: Fix resource leak for deps temp dir, restrict dependencies to directories only, ensure shutil import exists, and add _temp_deps_dir as instance attribute with cleanup
  • sagemaker-train/src/sagemaker/train/templates.py: Improve INSTALL_DEPENDENCIES template to handle both directories and .whl/.tar.gz files via pip install
  • sagemaker-train/tests/unit/train/test_model_trainer_dependencies.py: Fix weak tests, change fixture scope, use proper future import, and add meaningful assertions
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@sagemaker-botsagemaker-botsagemaker-bot left review comments

At least 1 approving review is required to merge this pull request.


Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

2 participants

@aviruthen@sagemaker-bot

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms");
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
fix: Add additional dependencies for ModelTrainer (5668) - #5731
Draft
aviruthen wants to merge 2 commits into
aws:masterfrom 
aviruthen:fix/add-additional-dependencies-for-modeltrainer-5668
Draft
fix: Add additional dependencies for ModelTrainer (5668)#5731
aviruthen wants to merge 2 commits into
aws:masterfrom 
aviruthen:fix/add-additional-dependencies-for-modeltrainer-5668
Conversation
@aviruthen
@aviruthenaviruthen
commented
Apr 7, 2026

Copy link


Copy Markdown

Collaborator
Description
The V2 Estimator's dependencies parameter allows users to specify a list of local directory/file paths containing additional libraries that are uploaded to S3, mounted in the training container, and added to PYTHONPATH. The V3 ModelTrainer's SourceCode config has no equivalent field. The fix requires: (1) Adding a dependencies field to the SourceCode config class, (2) Adding a new constant for the dependencies channel and container path, (3) In _create_training_job_args(), creating input data channels for each dependency and copying them into the sm_drivers temp dir, (4) Adding an INSTALL_DEPENDENCIES template that adds dependency paths to PYTHONPATH and sys.path, (5) Including the dependencies installation step in the generated train script, and (6) Validating dependency paths exist during _validate_source_code().
Related Issue
Related issue: 5668
Changes Made
  • sagemaker-core/src/sagemaker/core/training/configs.py
  • sagemaker-train/src/sagemaker/train/constants.py
  • sagemaker-train/src/sagemaker/train/templates.py
  • sagemaker-train/src/sagemaker/train/model_trainer.py
  • sagemaker-train/tests/unit/train/test_model_trainer_dependencies.py
AI-Generated PR
This PR was automatically generated by the PySDK Issue Agent.
  • Confidence score: 85%
  • Classification: bug
  • SDK version target: V3
Merge Checklist
  •  Changes are backward compatible
  •  Commit message follows prefix: description format
  •  Unit tests added/updated
  •  Integration tests added (if applicable)
  •  Documentation updated (if applicable)
sagemaker-bot
sagemaker-bot
reviewed
Apr 7, 2026
View reviewed changes
@sagemaker-botsagemaker-bot
left a comment


Copy link


Copy Markdown

Collaborator
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 AI Code Review
This PR adds a dependencies field to SourceCode for ModelTrainer, enabling users to specify additional local directories/files to be uploaded and added to PYTHONPATH in the training container. While the feature is well-structured, there are several issues: a resource leak with TemporaryDirectory, the INSTALL_DEPENDENCIES template only handles directories (not files), the PR description mentions sys.path but the implementation doesn't add it, and some tests are weak assertions that don't actually verify the behavior they claim to test.
Comment threadsagemaker-train/src/sagemaker/train/model_trainer.py
Outdated

Comment threadsagemaker-train/src/sagemaker/train/templates.py
Comment threadsagemaker-train/src/sagemaker/train/model_trainer.py
Comment threadsagemaker-train/src/sagemaker/train/model_trainer.py
Outdated

Comment threadsagemaker-train/src/sagemaker/train/constants.py
Comment threadsagemaker-train/tests/unit/train/test_model_trainer_dependencies.py
Comment threadsagemaker-train/tests/unit/train/test_model_trainer_dependencies.py
Comment threadsagemaker-core/src/sagemaker/core/training/configs.py
Comment threadsagemaker-train/src/sagemaker/train/model_trainer.py
Comment threadsagemaker-train/tests/unit/train/test_model_trainer_dependencies.py
@aviruthen
aviruthen
commented
Apr 7, 2026

Copy link


Copy Markdown

CollaboratorAuthor
🤖 Iteration #1 — Review Comments Addressed
Add dependencies support for ModelTrainer's SourceCode config
Description
When migrating from SageMaker v2 to v3, the V2 Estimator's dependencies parameter allows users to specify a list of local directory paths containing additional libraries that are uploaded to S3, mounted in the training container, and added to PYTHONPATH. The V3 ModelTrainer's SourceCode config had no equivalent field.
This PR adds dependencies support to SourceCode with the following changes:
Changes Made
sagemaker-core/src/sagemaker/core/training/configs.py
  • Added dependencies field to SourceCode config class (list of local directory paths)
  • Fixed PEP 8 blank line formatting
sagemaker-train/src/sagemaker/train/constants.py
  • Added SM_DEPENDENCIES and SM_DEPENDENCIES_CONTAINER_PATH constants for the dependencies channel
  • Fixed missing newline at end of file
sagemaker-train/src/sagemaker/train/templates.py
  • Added INSTALL_DEPENDENCIES template that:

    • Adds dependency directories to PYTHONPATH
    • Installs .whl and .tar.gz files via pip
    • Handles other files by adding the parent directory to PYTHONPATH
sagemaker-train/src/sagemaker/train/model_trainer.py
  • Added _temp_deps_dir private attribute for deterministic cleanup of the dependencies temp directory
  • In _create_training_job_args(), creates input data channels for dependencies by copying them to a temp dir preserving basenames
  • Includes the INSTALL_DEPENDENCIES step in the generated train script when dependencies are present
  • Validates that dependency paths exist and are directories during _validate_source_code()
  • Cleans up _temp_deps_dir in destructor and after training completes
sagemaker-train/tests/unit/train/test_model_trainer_dependencies.py
  • Updated to use from __future__ import annotations per V3 conventions
  • Changed fixture scope from module to function for proper test isolation
  • Added test for rejecting file dependencies (only directories allowed)
  • Fixed weak tests to actually verify train script content and temp directory structure
Design Decisions
  • Directories only: Dependencies are restricted to directories (matching V2 Estimator behavior), not individual files
  • PYTHONPATH approach: Dependencies are added to PYTHONPATH via the bash train script, which is sufficient since Python processes inherit environment variables
  • Dependencies before requirements: Dependencies are set up before requirements.txt installation, matching the logical ordering where dependencies should be available first
Comments reviewed: 34
Files modified:sagemaker-core/src/sagemaker/core/training/configs.py, sagemaker-train/src/sagemaker/train/constants.py, sagemaker-train/src/sagemaker/train/model_trainer.py, sagemaker-train/src/sagemaker/train/templates.py, sagemaker-train/tests/unit/train/test_model_trainer_dependencies.py
  • sagemaker-core/src/sagemaker/core/training/configs.py: Add blank line before OutputDataConfig class definition per PEP 8
  • sagemaker-train/src/sagemaker/train/constants.py: Ensure file ends with newline after SM_RECIPE_CONTAINER_PATH
  • sagemaker-train/src/sagemaker/train/model_trainer.py: Fix resource leak for deps temp dir, restrict dependencies to directories only, ensure shutil import exists, and add _temp_deps_dir as instance attribute with cleanup
  • sagemaker-train/src/sagemaker/train/templates.py: Improve INSTALL_DEPENDENCIES template to handle both directories and .whl/.tar.gz files via pip install
  • sagemaker-train/tests/unit/train/test_model_trainer_dependencies.py: Fix weak tests, change fixture scope, use proper future import, and add meaningful assertions
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@sagemaker-botsagemaker-botsagemaker-bot left review comments

At least 1 approving review is required to merge this pull request.


Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

2 participants

@aviruthen@sagemaker-bot

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links");
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '

Skip to content
fix: Add additional dependencies for ModelTrainer (5668) - #5731
Draft
aviruthen wants to merge 2 commits into
aws:masterfrom 
aviruthen:fix/add-additional-dependencies-for-modeltrainer-5668
Draft
fix: Add additional dependencies for ModelTrainer (5668)#5731
aviruthen wants to merge 2 commits into
aws:masterfrom 
aviruthen:fix/add-additional-dependencies-for-modeltrainer-5668
Conversation
@aviruthen
@aviruthenaviruthen
commented
Apr 7, 2026

Copy link


Copy Markdown

Collaborator
Description
The V2 Estimator's dependencies parameter allows users to specify a list of local directory/file paths containing additional libraries that are uploaded to S3, mounted in the training container, and added to PYTHONPATH. The V3 ModelTrainer's SourceCode config has no equivalent field. The fix requires: (1) Adding a dependencies field to the SourceCode config class, (2) Adding a new constant for the dependencies channel and container path, (3) In _create_training_job_args(), creating input data channels for each dependency and copying them into the sm_drivers temp dir, (4) Adding an INSTALL_DEPENDENCIES template that adds dependency paths to PYTHONPATH and sys.path, (5) Including the dependencies installation step in the generated train script, and (6) Validating dependency paths exist during _validate_source_code().
Related Issue
Related issue: 5668
Changes Made
  • sagemaker-core/src/sagemaker/core/training/configs.py
  • sagemaker-train/src/sagemaker/train/constants.py
  • sagemaker-train/src/sagemaker/train/templates.py
  • sagemaker-train/src/sagemaker/train/model_trainer.py
  • sagemaker-train/tests/unit/train/test_model_trainer_dependencies.py
AI-Generated PR
This PR was automatically generated by the PySDK Issue Agent.
  • Confidence score: 85%
  • Classification: bug
  • SDK version target: V3
Merge Checklist
  •  Changes are backward compatible
  •  Commit message follows prefix: description format
  •  Unit tests added/updated
  •  Integration tests added (if applicable)
  •  Documentation updated (if applicable)
sagemaker-bot
sagemaker-bot
reviewed
Apr 7, 2026
View reviewed changes
@sagemaker-botsagemaker-bot
left a comment


Copy link


Copy Markdown

Collaborator
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 AI Code Review
This PR adds a dependencies field to SourceCode for ModelTrainer, enabling users to specify additional local directories/files to be uploaded and added to PYTHONPATH in the training container. While the feature is well-structured, there are several issues: a resource leak with TemporaryDirectory, the INSTALL_DEPENDENCIES template only handles directories (not files), the PR description mentions sys.path but the implementation doesn't add it, and some tests are weak assertions that don't actually verify the behavior they claim to test.
Comment threadsagemaker-train/src/sagemaker/train/model_trainer.py
Outdated

Comment threadsagemaker-train/src/sagemaker/train/templates.py
Comment threadsagemaker-train/src/sagemaker/train/model_trainer.py
Comment threadsagemaker-train/src/sagemaker/train/model_trainer.py
Outdated

Comment threadsagemaker-train/src/sagemaker/train/constants.py
Comment threadsagemaker-train/tests/unit/train/test_model_trainer_dependencies.py
Comment threadsagemaker-train/tests/unit/train/test_model_trainer_dependencies.py
Comment threadsagemaker-core/src/sagemaker/core/training/configs.py
Comment threadsagemaker-train/src/sagemaker/train/model_trainer.py
Comment threadsagemaker-train/tests/unit/train/test_model_trainer_dependencies.py
@aviruthen
aviruthen
commented
Apr 7, 2026

Copy link


Copy Markdown

CollaboratorAuthor
🤖 Iteration #1 — Review Comments Addressed
Add dependencies support for ModelTrainer's SourceCode config
Description
When migrating from SageMaker v2 to v3, the V2 Estimator's dependencies parameter allows users to specify a list of local directory paths containing additional libraries that are uploaded to S3, mounted in the training container, and added to PYTHONPATH. The V3 ModelTrainer's SourceCode config had no equivalent field.
This PR adds dependencies support to SourceCode with the following changes:
Changes Made
sagemaker-core/src/sagemaker/core/training/configs.py
  • Added dependencies field to SourceCode config class (list of local directory paths)
  • Fixed PEP 8 blank line formatting
sagemaker-train/src/sagemaker/train/constants.py
  • Added SM_DEPENDENCIES and SM_DEPENDENCIES_CONTAINER_PATH constants for the dependencies channel
  • Fixed missing newline at end of file
sagemaker-train/src/sagemaker/train/templates.py
  • Added INSTALL_DEPENDENCIES template that:

    • Adds dependency directories to PYTHONPATH
    • Installs .whl and .tar.gz files via pip
    • Handles other files by adding the parent directory to PYTHONPATH
sagemaker-train/src/sagemaker/train/model_trainer.py
  • Added _temp_deps_dir private attribute for deterministic cleanup of the dependencies temp directory
  • In _create_training_job_args(), creates input data channels for dependencies by copying them to a temp dir preserving basenames
  • Includes the INSTALL_DEPENDENCIES step in the generated train script when dependencies are present
  • Validates that dependency paths exist and are directories during _validate_source_code()
  • Cleans up _temp_deps_dir in destructor and after training completes
sagemaker-train/tests/unit/train/test_model_trainer_dependencies.py
  • Updated to use from __future__ import annotations per V3 conventions
  • Changed fixture scope from module to function for proper test isolation
  • Added test for rejecting file dependencies (only directories allowed)
  • Fixed weak tests to actually verify train script content and temp directory structure
Design Decisions
  • Directories only: Dependencies are restricted to directories (matching V2 Estimator behavior), not individual files
  • PYTHONPATH approach: Dependencies are added to PYTHONPATH via the bash train script, which is sufficient since Python processes inherit environment variables
  • Dependencies before requirements: Dependencies are set up before requirements.txt installation, matching the logical ordering where dependencies should be available first
Comments reviewed: 34
Files modified:sagemaker-core/src/sagemaker/core/training/configs.py, sagemaker-train/src/sagemaker/train/constants.py, sagemaker-train/src/sagemaker/train/model_trainer.py, sagemaker-train/src/sagemaker/train/templates.py, sagemaker-train/tests/unit/train/test_model_trainer_dependencies.py
  • sagemaker-core/src/sagemaker/core/training/configs.py: Add blank line before OutputDataConfig class definition per PEP 8
  • sagemaker-train/src/sagemaker/train/constants.py: Ensure file ends with newline after SM_RECIPE_CONTAINER_PATH
  • sagemaker-train/src/sagemaker/train/model_trainer.py: Fix resource leak for deps temp dir, restrict dependencies to directories only, ensure shutil import exists, and add _temp_deps_dir as instance attribute with cleanup
  • sagemaker-train/src/sagemaker/train/templates.py: Improve INSTALL_DEPENDENCIES template to handle both directories and .whl/.tar.gz files via pip install
  • sagemaker-train/tests/unit/train/test_model_trainer_dependencies.py: Fix weak tests, change fixture scope, use proper future import, and add meaningful assertions
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@sagemaker-botsagemaker-botsagemaker-bot left review comments

At least 1 approving review is required to merge this pull request.


Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

2 participants

@aviruthen@sagemaker-bot

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default");
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
fix: Add additional dependencies for ModelTrainer (5668) - #5731
Draft
aviruthen wants to merge 2 commits into
aws:masterfrom 
aviruthen:fix/add-additional-dependencies-for-modeltrainer-5668
Draft
fix: Add additional dependencies for ModelTrainer (5668)#5731
aviruthen wants to merge 2 commits into
aws:masterfrom 
aviruthen:fix/add-additional-dependencies-for-modeltrainer-5668
Conversation
@aviruthen
@aviruthenaviruthen
commented
Apr 7, 2026

Copy link


Copy Markdown

Collaborator
Description
The V2 Estimator's dependencies parameter allows users to specify a list of local directory/file paths containing additional libraries that are uploaded to S3, mounted in the training container, and added to PYTHONPATH. The V3 ModelTrainer's SourceCode config has no equivalent field. The fix requires: (1) Adding a dependencies field to the SourceCode config class, (2) Adding a new constant for the dependencies channel and container path, (3) In _create_training_job_args(), creating input data channels for each dependency and copying them into the sm_drivers temp dir, (4) Adding an INSTALL_DEPENDENCIES template that adds dependency paths to PYTHONPATH and sys.path, (5) Including the dependencies installation step in the generated train script, and (6) Validating dependency paths exist during _validate_source_code().
Related Issue
Related issue: 5668
Changes Made
  • sagemaker-core/src/sagemaker/core/training/configs.py
  • sagemaker-train/src/sagemaker/train/constants.py
  • sagemaker-train/src/sagemaker/train/templates.py
  • sagemaker-train/src/sagemaker/train/model_trainer.py
  • sagemaker-train/tests/unit/train/test_model_trainer_dependencies.py
AI-Generated PR
This PR was automatically generated by the PySDK Issue Agent.
  • Confidence score: 85%
  • Classification: bug
  • SDK version target: V3
Merge Checklist
  •  Changes are backward compatible
  •  Commit message follows prefix: description format
  •  Unit tests added/updated
  •  Integration tests added (if applicable)
  •  Documentation updated (if applicable)
sagemaker-bot
sagemaker-bot
reviewed
Apr 7, 2026
View reviewed changes
@sagemaker-botsagemaker-bot
left a comment


Copy link


Copy Markdown

Collaborator
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 AI Code Review
This PR adds a dependencies field to SourceCode for ModelTrainer, enabling users to specify additional local directories/files to be uploaded and added to PYTHONPATH in the training container. While the feature is well-structured, there are several issues: a resource leak with TemporaryDirectory, the INSTALL_DEPENDENCIES template only handles directories (not files), the PR description mentions sys.path but the implementation doesn't add it, and some tests are weak assertions that don't actually verify the behavior they claim to test.
Comment threadsagemaker-train/src/sagemaker/train/model_trainer.py
Outdated

Comment threadsagemaker-train/src/sagemaker/train/templates.py
Comment threadsagemaker-train/src/sagemaker/train/model_trainer.py
Comment threadsagemaker-train/src/sagemaker/train/model_trainer.py
Outdated

Comment threadsagemaker-train/src/sagemaker/train/constants.py
Comment threadsagemaker-train/tests/unit/train/test_model_trainer_dependencies.py
Comment threadsagemaker-train/tests/unit/train/test_model_trainer_dependencies.py
Comment threadsagemaker-core/src/sagemaker/core/training/configs.py
Comment threadsagemaker-train/src/sagemaker/train/model_trainer.py
Comment threadsagemaker-train/tests/unit/train/test_model_trainer_dependencies.py
@aviruthen
aviruthen
commented
Apr 7, 2026

Copy link


Copy Markdown

CollaboratorAuthor
🤖 Iteration #1 — Review Comments Addressed
Add dependencies support for ModelTrainer's SourceCode config
Description
When migrating from SageMaker v2 to v3, the V2 Estimator's dependencies parameter allows users to specify a list of local directory paths containing additional libraries that are uploaded to S3, mounted in the training container, and added to PYTHONPATH. The V3 ModelTrainer's SourceCode config had no equivalent field.
This PR adds dependencies support to SourceCode with the following changes:
Changes Made
sagemaker-core/src/sagemaker/core/training/configs.py
  • Added dependencies field to SourceCode config class (list of local directory paths)
  • Fixed PEP 8 blank line formatting
sagemaker-train/src/sagemaker/train/constants.py
  • Added SM_DEPENDENCIES and SM_DEPENDENCIES_CONTAINER_PATH constants for the dependencies channel
  • Fixed missing newline at end of file
sagemaker-train/src/sagemaker/train/templates.py
  • Added INSTALL_DEPENDENCIES template that:

    • Adds dependency directories to PYTHONPATH
    • Installs .whl and .tar.gz files via pip
    • Handles other files by adding the parent directory to PYTHONPATH
sagemaker-train/src/sagemaker/train/model_trainer.py
  • Added _temp_deps_dir private attribute for deterministic cleanup of the dependencies temp directory
  • In _create_training_job_args(), creates input data channels for dependencies by copying them to a temp dir preserving basenames
  • Includes the INSTALL_DEPENDENCIES step in the generated train script when dependencies are present
  • Validates that dependency paths exist and are directories during _validate_source_code()
  • Cleans up _temp_deps_dir in destructor and after training completes
sagemaker-train/tests/unit/train/test_model_trainer_dependencies.py
  • Updated to use from __future__ import annotations per V3 conventions
  • Changed fixture scope from module to function for proper test isolation
  • Added test for rejecting file dependencies (only directories allowed)
  • Fixed weak tests to actually verify train script content and temp directory structure
Design Decisions
  • Directories only: Dependencies are restricted to directories (matching V2 Estimator behavior), not individual files
  • PYTHONPATH approach: Dependencies are added to PYTHONPATH via the bash train script, which is sufficient since Python processes inherit environment variables
  • Dependencies before requirements: Dependencies are set up before requirements.txt installation, matching the logical ordering where dependencies should be available first
Comments reviewed: 34
Files modified:sagemaker-core/src/sagemaker/core/training/configs.py, sagemaker-train/src/sagemaker/train/constants.py, sagemaker-train/src/sagemaker/train/model_trainer.py, sagemaker-train/src/sagemaker/train/templates.py, sagemaker-train/tests/unit/train/test_model_trainer_dependencies.py
  • sagemaker-core/src/sagemaker/core/training/configs.py: Add blank line before OutputDataConfig class definition per PEP 8
  • sagemaker-train/src/sagemaker/train/constants.py: Ensure file ends with newline after SM_RECIPE_CONTAINER_PATH
  • sagemaker-train/src/sagemaker/train/model_trainer.py: Fix resource leak for deps temp dir, restrict dependencies to directories only, ensure shutil import exists, and add _temp_deps_dir as instance attribute with cleanup
  • sagemaker-train/src/sagemaker/train/templates.py: Improve INSTALL_DEPENDENCIES template to handle both directories and .whl/.tar.gz files via pip install
  • sagemaker-train/tests/unit/train/test_model_trainer_dependencies.py: Fix weak tests, change fixture scope, use proper future import, and add meaningful assertions
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@sagemaker-botsagemaker-botsagemaker-bot left review comments

At least 1 approving review is required to merge this pull request.


Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

2 participants

@aviruthen@sagemaker-bot

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers");
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
fix: Add additional dependencies for ModelTrainer (5668) - #5731
Draft
aviruthen wants to merge 2 commits into
aws:masterfrom 
aviruthen:fix/add-additional-dependencies-for-modeltrainer-5668
Draft
fix: Add additional dependencies for ModelTrainer (5668)#5731
aviruthen wants to merge 2 commits into
aws:masterfrom 
aviruthen:fix/add-additional-dependencies-for-modeltrainer-5668
Conversation
@aviruthen
@aviruthenaviruthen
commented
Apr 7, 2026

Copy link


Copy Markdown

Collaborator
Description
The V2 Estimator's dependencies parameter allows users to specify a list of local directory/file paths containing additional libraries that are uploaded to S3, mounted in the training container, and added to PYTHONPATH. The V3 ModelTrainer's SourceCode config has no equivalent field. The fix requires: (1) Adding a dependencies field to the SourceCode config class, (2) Adding a new constant for the dependencies channel and container path, (3) In _create_training_job_args(), creating input data channels for each dependency and copying them into the sm_drivers temp dir, (4) Adding an INSTALL_DEPENDENCIES template that adds dependency paths to PYTHONPATH and sys.path, (5) Including the dependencies installation step in the generated train script, and (6) Validating dependency paths exist during _validate_source_code().
Related Issue
Related issue: 5668
Changes Made
  • sagemaker-core/src/sagemaker/core/training/configs.py
  • sagemaker-train/src/sagemaker/train/constants.py
  • sagemaker-train/src/sagemaker/train/templates.py
  • sagemaker-train/src/sagemaker/train/model_trainer.py
  • sagemaker-train/tests/unit/train/test_model_trainer_dependencies.py
AI-Generated PR
This PR was automatically generated by the PySDK Issue Agent.
  • Confidence score: 85%
  • Classification: bug
  • SDK version target: V3
Merge Checklist
  •  Changes are backward compatible
  •  Commit message follows prefix: description format
  •  Unit tests added/updated
  •  Integration tests added (if applicable)
  •  Documentation updated (if applicable)
sagemaker-bot
sagemaker-bot
reviewed
Apr 7, 2026
View reviewed changes
@sagemaker-botsagemaker-bot
left a comment


Copy link


Copy Markdown

Collaborator
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 AI Code Review
This PR adds a dependencies field to SourceCode for ModelTrainer, enabling users to specify additional local directories/files to be uploaded and added to PYTHONPATH in the training container. While the feature is well-structured, there are several issues: a resource leak with TemporaryDirectory, the INSTALL_DEPENDENCIES template only handles directories (not files), the PR description mentions sys.path but the implementation doesn't add it, and some tests are weak assertions that don't actually verify the behavior they claim to test.
Comment threadsagemaker-train/src/sagemaker/train/model_trainer.py
Outdated

Comment threadsagemaker-train/src/sagemaker/train/templates.py
Comment threadsagemaker-train/src/sagemaker/train/model_trainer.py
Comment threadsagemaker-train/src/sagemaker/train/model_trainer.py
Outdated

Comment threadsagemaker-train/src/sagemaker/train/constants.py
Comment threadsagemaker-train/tests/unit/train/test_model_trainer_dependencies.py
Comment threadsagemaker-train/tests/unit/train/test_model_trainer_dependencies.py
Comment threadsagemaker-core/src/sagemaker/core/training/configs.py
Comment threadsagemaker-train/src/sagemaker/train/model_trainer.py
Comment threadsagemaker-train/tests/unit/train/test_model_trainer_dependencies.py
@aviruthen
aviruthen
commented
Apr 7, 2026

Copy link


Copy Markdown

CollaboratorAuthor
🤖 Iteration #1 — Review Comments Addressed
Add dependencies support for ModelTrainer's SourceCode config
Description
When migrating from SageMaker v2 to v3, the V2 Estimator's dependencies parameter allows users to specify a list of local directory paths containing additional libraries that are uploaded to S3, mounted in the training container, and added to PYTHONPATH. The V3 ModelTrainer's SourceCode config had no equivalent field.
This PR adds dependencies support to SourceCode with the following changes:
Changes Made
sagemaker-core/src/sagemaker/core/training/configs.py
  • Added dependencies field to SourceCode config class (list of local directory paths)
  • Fixed PEP 8 blank line formatting
sagemaker-train/src/sagemaker/train/constants.py
  • Added SM_DEPENDENCIES and SM_DEPENDENCIES_CONTAINER_PATH constants for the dependencies channel
  • Fixed missing newline at end of file
sagemaker-train/src/sagemaker/train/templates.py
  • Added INSTALL_DEPENDENCIES template that:

    • Adds dependency directories to PYTHONPATH
    • Installs .whl and .tar.gz files via pip
    • Handles other files by adding the parent directory to PYTHONPATH
sagemaker-train/src/sagemaker/train/model_trainer.py
  • Added _temp_deps_dir private attribute for deterministic cleanup of the dependencies temp directory
  • In _create_training_job_args(), creates input data channels for dependencies by copying them to a temp dir preserving basenames
  • Includes the INSTALL_DEPENDENCIES step in the generated train script when dependencies are present
  • Validates that dependency paths exist and are directories during _validate_source_code()
  • Cleans up _temp_deps_dir in destructor and after training completes
sagemaker-train/tests/unit/train/test_model_trainer_dependencies.py
  • Updated to use from __future__ import annotations per V3 conventions
  • Changed fixture scope from module to function for proper test isolation
  • Added test for rejecting file dependencies (only directories allowed)
  • Fixed weak tests to actually verify train script content and temp directory structure
Design Decisions
  • Directories only: Dependencies are restricted to directories (matching V2 Estimator behavior), not individual files
  • PYTHONPATH approach: Dependencies are added to PYTHONPATH via the bash train script, which is sufficient since Python processes inherit environment variables
  • Dependencies before requirements: Dependencies are set up before requirements.txt installation, matching the logical ordering where dependencies should be available first
Comments reviewed: 34
Files modified:sagemaker-core/src/sagemaker/core/training/configs.py, sagemaker-train/src/sagemaker/train/constants.py, sagemaker-train/src/sagemaker/train/model_trainer.py, sagemaker-train/src/sagemaker/train/templates.py, sagemaker-train/tests/unit/train/test_model_trainer_dependencies.py
  • sagemaker-core/src/sagemaker/core/training/configs.py: Add blank line before OutputDataConfig class definition per PEP 8
  • sagemaker-train/src/sagemaker/train/constants.py: Ensure file ends with newline after SM_RECIPE_CONTAINER_PATH
  • sagemaker-train/src/sagemaker/train/model_trainer.py: Fix resource leak for deps temp dir, restrict dependencies to directories only, ensure shutil import exists, and add _temp_deps_dir as instance attribute with cleanup
  • sagemaker-train/src/sagemaker/train/templates.py: Improve INSTALL_DEPENDENCIES template to handle both directories and .whl/.tar.gz files via pip install
  • sagemaker-train/tests/unit/train/test_model_trainer_dependencies.py: Fix weak tests, change fixture scope, use proper future import, and add meaningful assertions
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@sagemaker-botsagemaker-botsagemaker-bot left review comments

At least 1 approving review is required to merge this pull request.


Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

2 participants

@aviruthen@sagemaker-bot

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode");
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();

Skip to content
fix: Add additional dependencies for ModelTrainer (5668) - #5731
Draft
aviruthen wants to merge 2 commits into
aws:masterfrom 
aviruthen:fix/add-additional-dependencies-for-modeltrainer-5668
Draft
fix: Add additional dependencies for ModelTrainer (5668)#5731
aviruthen wants to merge 2 commits into
aws:masterfrom 
aviruthen:fix/add-additional-dependencies-for-modeltrainer-5668
Conversation
@aviruthen
@aviruthenaviruthen
commented
Apr 7, 2026

Copy link


Copy Markdown

Collaborator
Description
The V2 Estimator's dependencies parameter allows users to specify a list of local directory/file paths containing additional libraries that are uploaded to S3, mounted in the training container, and added to PYTHONPATH. The V3 ModelTrainer's SourceCode config has no equivalent field. The fix requires: (1) Adding a dependencies field to the SourceCode config class, (2) Adding a new constant for the dependencies channel and container path, (3) In _create_training_job_args(), creating input data channels for each dependency and copying them into the sm_drivers temp dir, (4) Adding an INSTALL_DEPENDENCIES template that adds dependency paths to PYTHONPATH and sys.path, (5) Including the dependencies installation step in the generated train script, and (6) Validating dependency paths exist during _validate_source_code().
Related Issue
Related issue: 5668
Changes Made
  • sagemaker-core/src/sagemaker/core/training/configs.py
  • sagemaker-train/src/sagemaker/train/constants.py
  • sagemaker-train/src/sagemaker/train/templates.py
  • sagemaker-train/src/sagemaker/train/model_trainer.py
  • sagemaker-train/tests/unit/train/test_model_trainer_dependencies.py
AI-Generated PR
This PR was automatically generated by the PySDK Issue Agent.
  • Confidence score: 85%
  • Classification: bug
  • SDK version target: V3
Merge Checklist
  •  Changes are backward compatible
  •  Commit message follows prefix: description format
  •  Unit tests added/updated
  •  Integration tests added (if applicable)
  •  Documentation updated (if applicable)
sagemaker-bot
sagemaker-bot
reviewed
Apr 7, 2026
View reviewed changes
@sagemaker-botsagemaker-bot
left a comment


Copy link


Copy Markdown

Collaborator
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 AI Code Review
This PR adds a dependencies field to SourceCode for ModelTrainer, enabling users to specify additional local directories/files to be uploaded and added to PYTHONPATH in the training container. While the feature is well-structured, there are several issues: a resource leak with TemporaryDirectory, the INSTALL_DEPENDENCIES template only handles directories (not files), the PR description mentions sys.path but the implementation doesn't add it, and some tests are weak assertions that don't actually verify the behavior they claim to test.
Comment threadsagemaker-train/src/sagemaker/train/model_trainer.py
Outdated

Comment threadsagemaker-train/src/sagemaker/train/templates.py
Comment threadsagemaker-train/src/sagemaker/train/model_trainer.py
Comment threadsagemaker-train/src/sagemaker/train/model_trainer.py
Outdated

Comment threadsagemaker-train/src/sagemaker/train/constants.py
Comment threadsagemaker-train/tests/unit/train/test_model_trainer_dependencies.py
Comment threadsagemaker-train/tests/unit/train/test_model_trainer_dependencies.py
Comment threadsagemaker-core/src/sagemaker/core/training/configs.py
Comment threadsagemaker-train/src/sagemaker/train/model_trainer.py
Comment threadsagemaker-train/tests/unit/train/test_model_trainer_dependencies.py
@aviruthen
aviruthen
commented
Apr 7, 2026

Copy link


Copy Markdown

CollaboratorAuthor
🤖 Iteration #1 — Review Comments Addressed
Add dependencies support for ModelTrainer's SourceCode config
Description
When migrating from SageMaker v2 to v3, the V2 Estimator's dependencies parameter allows users to specify a list of local directory paths containing additional libraries that are uploaded to S3, mounted in the training container, and added to PYTHONPATH. The V3 ModelTrainer's SourceCode config had no equivalent field.
This PR adds dependencies support to SourceCode with the following changes:
Changes Made
sagemaker-core/src/sagemaker/core/training/configs.py
  • Added dependencies field to SourceCode config class (list of local directory paths)
  • Fixed PEP 8 blank line formatting
sagemaker-train/src/sagemaker/train/constants.py
  • Added SM_DEPENDENCIES and SM_DEPENDENCIES_CONTAINER_PATH constants for the dependencies channel
  • Fixed missing newline at end of file
sagemaker-train/src/sagemaker/train/templates.py
  • Added INSTALL_DEPENDENCIES template that:

    • Adds dependency directories to PYTHONPATH
    • Installs .whl and .tar.gz files via pip
    • Handles other files by adding the parent directory to PYTHONPATH
sagemaker-train/src/sagemaker/train/model_trainer.py
  • Added _temp_deps_dir private attribute for deterministic cleanup of the dependencies temp directory
  • In _create_training_job_args(), creates input data channels for dependencies by copying them to a temp dir preserving basenames
  • Includes the INSTALL_DEPENDENCIES step in the generated train script when dependencies are present
  • Validates that dependency paths exist and are directories during _validate_source_code()
  • Cleans up _temp_deps_dir in destructor and after training completes
sagemaker-train/tests/unit/train/test_model_trainer_dependencies.py
  • Updated to use from __future__ import annotations per V3 conventions
  • Changed fixture scope from module to function for proper test isolation
  • Added test for rejecting file dependencies (only directories allowed)
  • Fixed weak tests to actually verify train script content and temp directory structure
Design Decisions
  • Directories only: Dependencies are restricted to directories (matching V2 Estimator behavior), not individual files
  • PYTHONPATH approach: Dependencies are added to PYTHONPATH via the bash train script, which is sufficient since Python processes inherit environment variables
  • Dependencies before requirements: Dependencies are set up before requirements.txt installation, matching the logical ordering where dependencies should be available first
Comments reviewed: 34
Files modified:sagemaker-core/src/sagemaker/core/training/configs.py, sagemaker-train/src/sagemaker/train/constants.py, sagemaker-train/src/sagemaker/train/model_trainer.py, sagemaker-train/src/sagemaker/train/templates.py, sagemaker-train/tests/unit/train/test_model_trainer_dependencies.py
  • sagemaker-core/src/sagemaker/core/training/configs.py: Add blank line before OutputDataConfig class definition per PEP 8
  • sagemaker-train/src/sagemaker/train/constants.py: Ensure file ends with newline after SM_RECIPE_CONTAINER_PATH
  • sagemaker-train/src/sagemaker/train/model_trainer.py: Fix resource leak for deps temp dir, restrict dependencies to directories only, ensure shutil import exists, and add _temp_deps_dir as instance attribute with cleanup
  • sagemaker-train/src/sagemaker/train/templates.py: Improve INSTALL_DEPENDENCIES template to handle both directories and .whl/.tar.gz files via pip install
  • sagemaker-train/tests/unit/train/test_model_trainer_dependencies.py: Fix weak tests, change fixture scope, use proper future import, and add meaningful assertions
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@sagemaker-botsagemaker-botsagemaker-bot left review comments

At least 1 approving review is required to merge this pull request.


Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

2 participants

@aviruthen@sagemaker-bot