fix(train): Enforce Nova Restricted storage on all MTRL MPG branches - #6186

Open
SifeiLi wants to merge 1 commit into
aws:masterfrom
SifeiLi:fix-nova-rmpg-mpg-resolution
Open

fix(train): Enforce Nova Restricted storage on all MTRL MPG branches#6186
SifeiLi wants to merge 1 commit into
aws:masterfrom
SifeiLi:fix-nova-rmpg-mpg-resolution

Conversation

@SifeiLi

Copy link
Copy Markdown
Contributor

Restricted managed storage was only applied when MultiTurnRLTrainer auto-created a fresh model package group. The explicit output_model_package_group branch and the continued-customization branch (group derived from the source ModelPackage) performed no Nova check.

Centralize the requirement in _nova_managed_configuration() and apply it on every resolution path for both the output and intermediate checkpoint groups. Existing groups that do not satisfy the requirement now fail fast at trainer construction with an actionable error, since groups cannot be converted after creation. Non-Nova (OSS) models are unaffected.

Adds unit tests for the explicit, object, derived, and pre-existing default-named group branches, plus OSS pass-through.

Issue #, if available:

Description of changes:

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Tested below cases

Integration Tests — Happy Path (real SageMaker, us-east-1, wheel build)

#CaseVerification MethodResult
I1Nova + explicit Standard groupTrainer construction raises ValueError (bug repro — old SDK proceeded silently)✅ PASS
I2Nova + no group (auto-create)DescribeModelPackageGroup on both auto-created groups returns ManagedStorageType=Restricted (output + intermediate checkpoint)✅ PASS
I3Nova + explicit Restricted groupAccepted, resolves to group ARN✅ PASS

Integration Tests — Deny Cases (real SageMaker, us-east-1, wheel build)

#CaseWhy It MattersResult
D1Explicit Standard output group (name string)Original repro path✅ PASS
D2Pre-existing default-named Standard group (nova-…-mtrl-mpg)Residue left by the old buggy SDK in customer accounts is not silently reused✅ PASS
D3Explicit Standard intermediate checkpoint groupSecond resolver enforced, not just output✅ PASS
D4ModelPackageGroup object referencing Standard groupCaller-constructed objects cannot bypass validation (authoritative re-fetch)✅ PASS
D5Group derived from source ModelPackage (continued customization)The exact branch from the issue — Nova chains no longer inherit Standard groups✅ PASS

Notes: All rejections produce the actionable message "ModelPackageGroup 'X' uses 'Standard' managed storage, but model 'Y' requires 'Restricted'. Existing groups cannot be converted…". Integration runs used the built wheel
(sagemaker_train-1.20.0) in a clean venv; construction exercised the full real path (live hub metadata for nova-textgeneration-lite-v2, real MPG describe/create, S3 validation). All test fixtures deleted post-run. Non-Nova (OSS)
models: no validation, no extra API calls — behavior unchanged.

Restricted managed storage was only applied when MultiTurnRLTrainer
auto-created a fresh model package group. The explicit
output_model_package_group branch and the continued-customization
branch (group derived from the source ModelPackage) performed no Nova
check, so a Nova run could silently land its output or intermediate
checkpoints in a Standard-storage group, or split lineage across
groups depending on the branch taken.
Centralize the requirement in _nova_managed_configuration() and apply
it on every resolution path for both the output and intermediate
checkpoint groups. Existing groups that do not satisfy the requirement
now fail fast at trainer construction with an actionable error, since
groups cannot be converted after creation. Non-Nova (OSS) models are
unaffected.
Adds unit tests for the explicit, object, derived, and pre-existing
default-named group branches, plus OSS pass-through.
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@SifeiLi
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

fix(train): Enforce Nova Restricted storage on all MTRL MPG branches - #6186

Open
SifeiLi wants to merge 1 commit into
aws:masterfrom
SifeiLi:fix-nova-rmpg-mpg-resolution
Open

fix(train): Enforce Nova Restricted storage on all MTRL MPG branches#6186
SifeiLi wants to merge 1 commit into
aws:masterfrom
SifeiLi:fix-nova-rmpg-mpg-resolution

Conversation

@SifeiLi

Copy link
Copy Markdown
Contributor

Restricted managed storage was only applied when MultiTurnRLTrainer auto-created a fresh model package group. The explicit output_model_package_group branch and the continued-customization branch (group derived from the source ModelPackage) performed no Nova check.

Centralize the requirement in _nova_managed_configuration() and apply it on every resolution path for both the output and intermediate checkpoint groups. Existing groups that do not satisfy the requirement now fail fast at trainer construction with an actionable error, since groups cannot be converted after creation. Non-Nova (OSS) models are unaffected.

Adds unit tests for the explicit, object, derived, and pre-existing default-named group branches, plus OSS pass-through.

Issue #, if available:

Description of changes:

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Tested below cases

Integration Tests — Happy Path (real SageMaker, us-east-1, wheel build)

#CaseVerification MethodResult
I1Nova + explicit Standard groupTrainer construction raises ValueError (bug repro — old SDK proceeded silently)✅ PASS
I2Nova + no group (auto-create)DescribeModelPackageGroup on both auto-created groups returns ManagedStorageType=Restricted (output + intermediate checkpoint)✅ PASS
I3Nova + explicit Restricted groupAccepted, resolves to group ARN✅ PASS

Integration Tests — Deny Cases (real SageMaker, us-east-1, wheel build)

#CaseWhy It MattersResult
D1Explicit Standard output group (name string)Original repro path✅ PASS
D2Pre-existing default-named Standard group (nova-…-mtrl-mpg)Residue left by the old buggy SDK in customer accounts is not silently reused✅ PASS
D3Explicit Standard intermediate checkpoint groupSecond resolver enforced, not just output✅ PASS
D4ModelPackageGroup object referencing Standard groupCaller-constructed objects cannot bypass validation (authoritative re-fetch)✅ PASS
D5Group derived from source ModelPackage (continued customization)The exact branch from the issue — Nova chains no longer inherit Standard groups✅ PASS

Notes: All rejections produce the actionable message "ModelPackageGroup 'X' uses 'Standard' managed storage, but model 'Y' requires 'Restricted'. Existing groups cannot be converted…". Integration runs used the built wheel
(sagemaker_train-1.20.0) in a clean venv; construction exercised the full real path (live hub metadata for nova-textgeneration-lite-v2, real MPG describe/create, S3 validation). All test fixtures deleted post-run. Non-Nova (OSS)
models: no validation, no extra API calls — behavior unchanged.

Restricted managed storage was only applied when MultiTurnRLTrainer
auto-created a fresh model package group. The explicit
output_model_package_group branch and the continued-customization
branch (group derived from the source ModelPackage) performed no Nova
check, so a Nova run could silently land its output or intermediate
checkpoints in a Standard-storage group, or split lineage across
groups depending on the branch taken.
Centralize the requirement in _nova_managed_configuration() and apply
it on every resolution path for both the output and intermediate
checkpoint groups. Existing groups that do not satisfy the requirement
now fail fast at trainer construction with an actionable error, since
groups cannot be converted after creation. Non-Nova (OSS) models are
unaffected.
Adds unit tests for the explicit, object, derived, and pre-existing
default-named group branches, plus OSS pass-through.
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@SifeiLi
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

fix(train): Enforce Nova Restricted storage on all MTRL MPG branches - #6186

Open
SifeiLi wants to merge 1 commit into
aws:masterfrom
SifeiLi:fix-nova-rmpg-mpg-resolution
Open

fix(train): Enforce Nova Restricted storage on all MTRL MPG branches#6186
SifeiLi wants to merge 1 commit into
aws:masterfrom
SifeiLi:fix-nova-rmpg-mpg-resolution

Conversation

@SifeiLi

Copy link
Copy Markdown
Contributor

Restricted managed storage was only applied when MultiTurnRLTrainer auto-created a fresh model package group. The explicit output_model_package_group branch and the continued-customization branch (group derived from the source ModelPackage) performed no Nova check.

Centralize the requirement in _nova_managed_configuration() and apply it on every resolution path for both the output and intermediate checkpoint groups. Existing groups that do not satisfy the requirement now fail fast at trainer construction with an actionable error, since groups cannot be converted after creation. Non-Nova (OSS) models are unaffected.

Adds unit tests for the explicit, object, derived, and pre-existing default-named group branches, plus OSS pass-through.

Issue #, if available:

Description of changes:

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Tested below cases

Integration Tests — Happy Path (real SageMaker, us-east-1, wheel build)

#CaseVerification MethodResult
I1Nova + explicit Standard groupTrainer construction raises ValueError (bug repro — old SDK proceeded silently)✅ PASS
I2Nova + no group (auto-create)DescribeModelPackageGroup on both auto-created groups returns ManagedStorageType=Restricted (output + intermediate checkpoint)✅ PASS
I3Nova + explicit Restricted groupAccepted, resolves to group ARN✅ PASS

Integration Tests — Deny Cases (real SageMaker, us-east-1, wheel build)

#CaseWhy It MattersResult
D1Explicit Standard output group (name string)Original repro path✅ PASS
D2Pre-existing default-named Standard group (nova-…-mtrl-mpg)Residue left by the old buggy SDK in customer accounts is not silently reused✅ PASS
D3Explicit Standard intermediate checkpoint groupSecond resolver enforced, not just output✅ PASS
D4ModelPackageGroup object referencing Standard groupCaller-constructed objects cannot bypass validation (authoritative re-fetch)✅ PASS
D5Group derived from source ModelPackage (continued customization)The exact branch from the issue — Nova chains no longer inherit Standard groups✅ PASS

Notes: All rejections produce the actionable message "ModelPackageGroup 'X' uses 'Standard' managed storage, but model 'Y' requires 'Restricted'. Existing groups cannot be converted…". Integration runs used the built wheel
(sagemaker_train-1.20.0) in a clean venv; construction exercised the full real path (live hub metadata for nova-textgeneration-lite-v2, real MPG describe/create, S3 validation). All test fixtures deleted post-run. Non-Nova (OSS)
models: no validation, no extra API calls — behavior unchanged.

Restricted managed storage was only applied when MultiTurnRLTrainer
auto-created a fresh model package group. The explicit
output_model_package_group branch and the continued-customization
branch (group derived from the source ModelPackage) performed no Nova
check, so a Nova run could silently land its output or intermediate
checkpoints in a Standard-storage group, or split lineage across
groups depending on the branch taken.
Centralize the requirement in _nova_managed_configuration() and apply
it on every resolution path for both the output and intermediate
checkpoint groups. Existing groups that do not satisfy the requirement
now fail fast at trainer construction with an actionable error, since
groups cannot be converted after creation. Non-Nova (OSS) models are
unaffected.
Adds unit tests for the explicit, object, derived, and pre-existing
default-named group branches, plus OSS pass-through.
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@SifeiLi
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

fix(train): Enforce Nova Restricted storage on all MTRL MPG branches - #6186

Open
SifeiLi wants to merge 1 commit into
aws:masterfrom
SifeiLi:fix-nova-rmpg-mpg-resolution
Open

fix(train): Enforce Nova Restricted storage on all MTRL MPG branches#6186
SifeiLi wants to merge 1 commit into
aws:masterfrom
SifeiLi:fix-nova-rmpg-mpg-resolution

Conversation

@SifeiLi

Copy link
Copy Markdown
Contributor

Restricted managed storage was only applied when MultiTurnRLTrainer auto-created a fresh model package group. The explicit output_model_package_group branch and the continued-customization branch (group derived from the source ModelPackage) performed no Nova check.

Centralize the requirement in _nova_managed_configuration() and apply it on every resolution path for both the output and intermediate checkpoint groups. Existing groups that do not satisfy the requirement now fail fast at trainer construction with an actionable error, since groups cannot be converted after creation. Non-Nova (OSS) models are unaffected.

Adds unit tests for the explicit, object, derived, and pre-existing default-named group branches, plus OSS pass-through.

Issue #, if available:

Description of changes:

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Tested below cases

Integration Tests — Happy Path (real SageMaker, us-east-1, wheel build)

#CaseVerification MethodResult
I1Nova + explicit Standard groupTrainer construction raises ValueError (bug repro — old SDK proceeded silently)✅ PASS
I2Nova + no group (auto-create)DescribeModelPackageGroup on both auto-created groups returns ManagedStorageType=Restricted (output + intermediate checkpoint)✅ PASS
I3Nova + explicit Restricted groupAccepted, resolves to group ARN✅ PASS

Integration Tests — Deny Cases (real SageMaker, us-east-1, wheel build)

#CaseWhy It MattersResult
D1Explicit Standard output group (name string)Original repro path✅ PASS
D2Pre-existing default-named Standard group (nova-…-mtrl-mpg)Residue left by the old buggy SDK in customer accounts is not silently reused✅ PASS
D3Explicit Standard intermediate checkpoint groupSecond resolver enforced, not just output✅ PASS
D4ModelPackageGroup object referencing Standard groupCaller-constructed objects cannot bypass validation (authoritative re-fetch)✅ PASS
D5Group derived from source ModelPackage (continued customization)The exact branch from the issue — Nova chains no longer inherit Standard groups✅ PASS

Notes: All rejections produce the actionable message "ModelPackageGroup 'X' uses 'Standard' managed storage, but model 'Y' requires 'Restricted'. Existing groups cannot be converted…". Integration runs used the built wheel
(sagemaker_train-1.20.0) in a clean venv; construction exercised the full real path (live hub metadata for nova-textgeneration-lite-v2, real MPG describe/create, S3 validation). All test fixtures deleted post-run. Non-Nova (OSS)
models: no validation, no extra API calls — behavior unchanged.

Restricted managed storage was only applied when MultiTurnRLTrainer
auto-created a fresh model package group. The explicit
output_model_package_group branch and the continued-customization
branch (group derived from the source ModelPackage) performed no Nova
check, so a Nova run could silently land its output or intermediate
checkpoints in a Standard-storage group, or split lineage across
groups depending on the branch taken.
Centralize the requirement in _nova_managed_configuration() and apply
it on every resolution path for both the output and intermediate
checkpoint groups. Existing groups that do not satisfy the requirement
now fail fast at trainer construction with an actionable error, since
groups cannot be converted after creation. Non-Nova (OSS) models are
unaffected.
Adds unit tests for the explicit, object, derived, and pre-existing
default-named group branches, plus OSS pass-through.
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@SifeiLi
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

fix(train): Enforce Nova Restricted storage on all MTRL MPG branches - #6186

Open
SifeiLi wants to merge 1 commit into
aws:masterfrom
SifeiLi:fix-nova-rmpg-mpg-resolution
Open

fix(train): Enforce Nova Restricted storage on all MTRL MPG branches#6186
SifeiLi wants to merge 1 commit into
aws:masterfrom
SifeiLi:fix-nova-rmpg-mpg-resolution

Conversation

@SifeiLi

Copy link
Copy Markdown
Contributor

Restricted managed storage was only applied when MultiTurnRLTrainer auto-created a fresh model package group. The explicit output_model_package_group branch and the continued-customization branch (group derived from the source ModelPackage) performed no Nova check.

Centralize the requirement in _nova_managed_configuration() and apply it on every resolution path for both the output and intermediate checkpoint groups. Existing groups that do not satisfy the requirement now fail fast at trainer construction with an actionable error, since groups cannot be converted after creation. Non-Nova (OSS) models are unaffected.

Adds unit tests for the explicit, object, derived, and pre-existing default-named group branches, plus OSS pass-through.

Issue #, if available:

Description of changes:

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Tested below cases

Integration Tests — Happy Path (real SageMaker, us-east-1, wheel build)

#CaseVerification MethodResult
I1Nova + explicit Standard groupTrainer construction raises ValueError (bug repro — old SDK proceeded silently)✅ PASS
I2Nova + no group (auto-create)DescribeModelPackageGroup on both auto-created groups returns ManagedStorageType=Restricted (output + intermediate checkpoint)✅ PASS
I3Nova + explicit Restricted groupAccepted, resolves to group ARN✅ PASS

Integration Tests — Deny Cases (real SageMaker, us-east-1, wheel build)

#CaseWhy It MattersResult
D1Explicit Standard output group (name string)Original repro path✅ PASS
D2Pre-existing default-named Standard group (nova-…-mtrl-mpg)Residue left by the old buggy SDK in customer accounts is not silently reused✅ PASS
D3Explicit Standard intermediate checkpoint groupSecond resolver enforced, not just output✅ PASS
D4ModelPackageGroup object referencing Standard groupCaller-constructed objects cannot bypass validation (authoritative re-fetch)✅ PASS
D5Group derived from source ModelPackage (continued customization)The exact branch from the issue — Nova chains no longer inherit Standard groups✅ PASS

Notes: All rejections produce the actionable message "ModelPackageGroup 'X' uses 'Standard' managed storage, but model 'Y' requires 'Restricted'. Existing groups cannot be converted…". Integration runs used the built wheel
(sagemaker_train-1.20.0) in a clean venv; construction exercised the full real path (live hub metadata for nova-textgeneration-lite-v2, real MPG describe/create, S3 validation). All test fixtures deleted post-run. Non-Nova (OSS)
models: no validation, no extra API calls — behavior unchanged.

Restricted managed storage was only applied when MultiTurnRLTrainer
auto-created a fresh model package group. The explicit
output_model_package_group branch and the continued-customization
branch (group derived from the source ModelPackage) performed no Nova
check, so a Nova run could silently land its output or intermediate
checkpoints in a Standard-storage group, or split lineage across
groups depending on the branch taken.
Centralize the requirement in _nova_managed_configuration() and apply
it on every resolution path for both the output and intermediate
checkpoint groups. Existing groups that do not satisfy the requirement
now fail fast at trainer construction with an actionable error, since
groups cannot be converted after creation. Non-Nova (OSS) models are
unaffected.
Adds unit tests for the explicit, object, derived, and pre-existing
default-named group branches, plus OSS pass-through.
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@SifeiLi
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

fix(train): Enforce Nova Restricted storage on all MTRL MPG branches - #6186

Open
SifeiLi wants to merge 1 commit into
aws:masterfrom
SifeiLi:fix-nova-rmpg-mpg-resolution
Open

fix(train): Enforce Nova Restricted storage on all MTRL MPG branches#6186
SifeiLi wants to merge 1 commit into
aws:masterfrom
SifeiLi:fix-nova-rmpg-mpg-resolution

Conversation

@SifeiLi

Copy link
Copy Markdown
Contributor

Restricted managed storage was only applied when MultiTurnRLTrainer auto-created a fresh model package group. The explicit output_model_package_group branch and the continued-customization branch (group derived from the source ModelPackage) performed no Nova check.

Centralize the requirement in _nova_managed_configuration() and apply it on every resolution path for both the output and intermediate checkpoint groups. Existing groups that do not satisfy the requirement now fail fast at trainer construction with an actionable error, since groups cannot be converted after creation. Non-Nova (OSS) models are unaffected.

Adds unit tests for the explicit, object, derived, and pre-existing default-named group branches, plus OSS pass-through.

Issue #, if available:

Description of changes:

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Tested below cases

Integration Tests — Happy Path (real SageMaker, us-east-1, wheel build)

#CaseVerification MethodResult
I1Nova + explicit Standard groupTrainer construction raises ValueError (bug repro — old SDK proceeded silently)✅ PASS
I2Nova + no group (auto-create)DescribeModelPackageGroup on both auto-created groups returns ManagedStorageType=Restricted (output + intermediate checkpoint)✅ PASS
I3Nova + explicit Restricted groupAccepted, resolves to group ARN✅ PASS

Integration Tests — Deny Cases (real SageMaker, us-east-1, wheel build)

#CaseWhy It MattersResult
D1Explicit Standard output group (name string)Original repro path✅ PASS
D2Pre-existing default-named Standard group (nova-…-mtrl-mpg)Residue left by the old buggy SDK in customer accounts is not silently reused✅ PASS
D3Explicit Standard intermediate checkpoint groupSecond resolver enforced, not just output✅ PASS
D4ModelPackageGroup object referencing Standard groupCaller-constructed objects cannot bypass validation (authoritative re-fetch)✅ PASS
D5Group derived from source ModelPackage (continued customization)The exact branch from the issue — Nova chains no longer inherit Standard groups✅ PASS

Notes: All rejections produce the actionable message "ModelPackageGroup 'X' uses 'Standard' managed storage, but model 'Y' requires 'Restricted'. Existing groups cannot be converted…". Integration runs used the built wheel
(sagemaker_train-1.20.0) in a clean venv; construction exercised the full real path (live hub metadata for nova-textgeneration-lite-v2, real MPG describe/create, S3 validation). All test fixtures deleted post-run. Non-Nova (OSS)
models: no validation, no extra API calls — behavior unchanged.

Restricted managed storage was only applied when MultiTurnRLTrainer
auto-created a fresh model package group. The explicit
output_model_package_group branch and the continued-customization
branch (group derived from the source ModelPackage) performed no Nova
check, so a Nova run could silently land its output or intermediate
checkpoints in a Standard-storage group, or split lineage across
groups depending on the branch taken.
Centralize the requirement in _nova_managed_configuration() and apply
it on every resolution path for both the output and intermediate
checkpoint groups. Existing groups that do not satisfy the requirement
now fail fast at trainer construction with an actionable error, since
groups cannot be converted after creation. Non-Nova (OSS) models are
unaffected.
Adds unit tests for the explicit, object, derived, and pre-existing
default-named group branches, plus OSS pass-through.
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@SifeiLi
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

fix(train): Enforce Nova Restricted storage on all MTRL MPG branches - #6186

Open
SifeiLi wants to merge 1 commit into
aws:masterfrom
SifeiLi:fix-nova-rmpg-mpg-resolution
Open

fix(train): Enforce Nova Restricted storage on all MTRL MPG branches#6186
SifeiLi wants to merge 1 commit into
aws:masterfrom
SifeiLi:fix-nova-rmpg-mpg-resolution

Conversation

@SifeiLi

Copy link
Copy Markdown
Contributor

Restricted managed storage was only applied when MultiTurnRLTrainer auto-created a fresh model package group. The explicit output_model_package_group branch and the continued-customization branch (group derived from the source ModelPackage) performed no Nova check.

Centralize the requirement in _nova_managed_configuration() and apply it on every resolution path for both the output and intermediate checkpoint groups. Existing groups that do not satisfy the requirement now fail fast at trainer construction with an actionable error, since groups cannot be converted after creation. Non-Nova (OSS) models are unaffected.

Adds unit tests for the explicit, object, derived, and pre-existing default-named group branches, plus OSS pass-through.

Issue #, if available:

Description of changes:

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Tested below cases

Integration Tests — Happy Path (real SageMaker, us-east-1, wheel build)

#CaseVerification MethodResult
I1Nova + explicit Standard groupTrainer construction raises ValueError (bug repro — old SDK proceeded silently)✅ PASS
I2Nova + no group (auto-create)DescribeModelPackageGroup on both auto-created groups returns ManagedStorageType=Restricted (output + intermediate checkpoint)✅ PASS
I3Nova + explicit Restricted groupAccepted, resolves to group ARN✅ PASS

Integration Tests — Deny Cases (real SageMaker, us-east-1, wheel build)

#CaseWhy It MattersResult
D1Explicit Standard output group (name string)Original repro path✅ PASS
D2Pre-existing default-named Standard group (nova-…-mtrl-mpg)Residue left by the old buggy SDK in customer accounts is not silently reused✅ PASS
D3Explicit Standard intermediate checkpoint groupSecond resolver enforced, not just output✅ PASS
D4ModelPackageGroup object referencing Standard groupCaller-constructed objects cannot bypass validation (authoritative re-fetch)✅ PASS
D5Group derived from source ModelPackage (continued customization)The exact branch from the issue — Nova chains no longer inherit Standard groups✅ PASS

Notes: All rejections produce the actionable message "ModelPackageGroup 'X' uses 'Standard' managed storage, but model 'Y' requires 'Restricted'. Existing groups cannot be converted…". Integration runs used the built wheel
(sagemaker_train-1.20.0) in a clean venv; construction exercised the full real path (live hub metadata for nova-textgeneration-lite-v2, real MPG describe/create, S3 validation). All test fixtures deleted post-run. Non-Nova (OSS)
models: no validation, no extra API calls — behavior unchanged.

Restricted managed storage was only applied when MultiTurnRLTrainer
auto-created a fresh model package group. The explicit
output_model_package_group branch and the continued-customization
branch (group derived from the source ModelPackage) performed no Nova
check, so a Nova run could silently land its output or intermediate
checkpoints in a Standard-storage group, or split lineage across
groups depending on the branch taken.
Centralize the requirement in _nova_managed_configuration() and apply
it on every resolution path for both the output and intermediate
checkpoint groups. Existing groups that do not satisfy the requirement
now fail fast at trainer construction with an actionable error, since
groups cannot be converted after creation. Non-Nova (OSS) models are
unaffected.
Adds unit tests for the explicit, object, derived, and pre-existing
default-named group branches, plus OSS pass-through.
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@SifeiLi
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

fix(train): Enforce Nova Restricted storage on all MTRL MPG branches - #6186

Open
SifeiLi wants to merge 1 commit into
aws:masterfrom
SifeiLi:fix-nova-rmpg-mpg-resolution
Open

fix(train): Enforce Nova Restricted storage on all MTRL MPG branches#6186
SifeiLi wants to merge 1 commit into
aws:masterfrom
SifeiLi:fix-nova-rmpg-mpg-resolution

Conversation

@SifeiLi

Copy link
Copy Markdown
Contributor

Restricted managed storage was only applied when MultiTurnRLTrainer auto-created a fresh model package group. The explicit output_model_package_group branch and the continued-customization branch (group derived from the source ModelPackage) performed no Nova check.

Centralize the requirement in _nova_managed_configuration() and apply it on every resolution path for both the output and intermediate checkpoint groups. Existing groups that do not satisfy the requirement now fail fast at trainer construction with an actionable error, since groups cannot be converted after creation. Non-Nova (OSS) models are unaffected.

Adds unit tests for the explicit, object, derived, and pre-existing default-named group branches, plus OSS pass-through.

Issue #, if available:

Description of changes:

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Tested below cases

Integration Tests — Happy Path (real SageMaker, us-east-1, wheel build)

#CaseVerification MethodResult
I1Nova + explicit Standard groupTrainer construction raises ValueError (bug repro — old SDK proceeded silently)✅ PASS
I2Nova + no group (auto-create)DescribeModelPackageGroup on both auto-created groups returns ManagedStorageType=Restricted (output + intermediate checkpoint)✅ PASS
I3Nova + explicit Restricted groupAccepted, resolves to group ARN✅ PASS

Integration Tests — Deny Cases (real SageMaker, us-east-1, wheel build)

#CaseWhy It MattersResult
D1Explicit Standard output group (name string)Original repro path✅ PASS
D2Pre-existing default-named Standard group (nova-…-mtrl-mpg)Residue left by the old buggy SDK in customer accounts is not silently reused✅ PASS
D3Explicit Standard intermediate checkpoint groupSecond resolver enforced, not just output✅ PASS
D4ModelPackageGroup object referencing Standard groupCaller-constructed objects cannot bypass validation (authoritative re-fetch)✅ PASS
D5Group derived from source ModelPackage (continued customization)The exact branch from the issue — Nova chains no longer inherit Standard groups✅ PASS

Notes: All rejections produce the actionable message "ModelPackageGroup 'X' uses 'Standard' managed storage, but model 'Y' requires 'Restricted'. Existing groups cannot be converted…". Integration runs used the built wheel
(sagemaker_train-1.20.0) in a clean venv; construction exercised the full real path (live hub metadata for nova-textgeneration-lite-v2, real MPG describe/create, S3 validation). All test fixtures deleted post-run. Non-Nova (OSS)
models: no validation, no extra API calls — behavior unchanged.

Restricted managed storage was only applied when MultiTurnRLTrainer
auto-created a fresh model package group. The explicit
output_model_package_group branch and the continued-customization
branch (group derived from the source ModelPackage) performed no Nova
check, so a Nova run could silently land its output or intermediate
checkpoints in a Standard-storage group, or split lineage across
groups depending on the branch taken.
Centralize the requirement in _nova_managed_configuration() and apply
it on every resolution path for both the output and intermediate
checkpoint groups. Existing groups that do not satisfy the requirement
now fail fast at trainer construction with an actionable error, since
groups cannot be converted after creation. Non-Nova (OSS) models are
unaffected.
Adds unit tests for the explicit, object, derived, and pre-existing
default-named group branches, plus OSS pass-through.
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@SifeiLi