docs(kubeflow): add Trainer v2 training runtime catalog - #255
Conversation
WalkthroughAdds six TrainingRuntime manifests (CUDA and Ascend), a TrainJob smoke-test example, and a documentation page describing runtime selection, device resource models, smoke-test steps, and image-specific caveats. ChangesCUDA GPU Training Runtimes
Ascend NPU Training Runtimes
Training Runtime Documentation and Guide
🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly Related PRs
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Deploying alauda-ai with
|
| Latest commit: |
32f075a
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://8f69221d.alauda-ai.pages.dev |
| Branch Preview URL: | https://codex-training-runtimes.alauda-ai.pages.dev |
There was a problem hiding this comment.
Actionable comments posted: 7
🧹 Nitpick comments (1)
docs/en/kubeflow/how_to/assets/training-runtimes/llamafactory0.9-cann8.5-arm64-trainingruntime.yaml (1)
42-42: ⚡ Quick winConsider removing
|| trueto catch CLI failures.The
|| truesuppresses failures fromllamafactory-cli version, which means the smoke test will succeed even if the CLI tool is broken or misconfigured. For a smoke test runtime, failing fast when core tooling doesn't work provides clearer diagnostics.If the CLI check is intentionally optional (e.g., known flakiness), consider adding a comment explaining why failures are suppressed.
🔧 Proposed change to fail fast on CLI issues
set -ex python -c "import torch, torch_npu, llamafactory; print('torch', torch.__version__, 'torch_npu', torch_npu.__version__, 'lf', llamafactory.__version__, 'npu_available', torch_npu.npu.is_available())" - llamafactory-cli version || true + llamafactory-cli version🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/en/kubeflow/how_to/assets/training-runtimes/llamafactory0.9-cann8.5-arm64-trainingruntime.yaml` at line 42, The smoke-test line currently runs "llamafactory-cli version || true", which silences failures; remove the "|| true" so the step fails fast on CLI errors and surfaces diagnostics, or if suppression is deliberate add an inline comment explaining the known flakiness and why the failure is acceptable; update the line that contains the exact token "llamafactory-cli version || true" accordingly and ensure the job/step will fail when "llamafactory-cli version" exits non‑zero.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In
`@docs/en/kubeflow/how_to/assets/training-runtimes/llamafactory0.9-cu126-amd64-trainingruntime.yaml`:
- Line 55: The smoke-check currently masks failures by appending "|| true" to
the "llamafactory-cli version" invocation, allowing the runtime to report
success even when the CLI is absent or failing; remove the "|| true" from the
"llamafactory-cli version" call (or replace it with an explicit
existence/version check that fails the step and surfaces/logs the error) so that
the command's non-zero exit propagates and the validation fails when the CLI is
missing or broken.
- Around line 39-41: The comment block starting with "Synthetic identity SFT
smoke" (the lines that say "Writes a 4-row alpaca dataset, fine-tunes qwen-style
chat template...") is misleading because the YAML step only prepares data and
performs version checks; update that comment to accurately describe the executed
behavior (e.g., "prepares a 4-row alpaca dataset and validates LLaMA-Factory
CLI/model versions; does not run a fine-tune"), or alternatively modify the
associated step that invokes the LLaMA-Factory CLI to actually kick off a
fine-tune if the original intention was to run one; ensure the comment and the
step invoking the LLaMA-Factory CLI / data preparation are consistent.
In
`@docs/en/kubeflow/how_to/assets/training-runtimes/torch2.6-cu126-amd64-trainingruntime.yaml`:
- Around line 41-44: The GPU smoke-check currently skips failure when CUDA is
missing because it only runs the matmul under if torch.cuda.is_available();
change this to fail fast: detect CUDA unavailability with
torch.cuda.is_available() and immediately print a clear error and exit non‑zero
(or raise an exception) so the container fails the smoke TrainJob; keep the
existing CUDA matmul path (creating x on device='cuda', computing y = x @ x.T
and printing the success message) but ensure the missing-CUDA branch aborts the
process.
In
`@docs/en/kubeflow/how_to/assets/training-runtimes/traininghub0.1-cu126-amd64-trainingruntime.yaml`:
- Around line 41-44: The current smoke check wraps CUDA-dependent logic in an if
torch.cuda.is_available(): block so a no-CUDA environment still passes; change
this to require CUDA by checking torch.cuda.is_available() and failing fast
(raise a RuntimeError or call sys.exit(1)) if False before importing peft and
instantiating LoraConfig (symbols: torch.cuda.is_available, LoraConfig, cfg,
print), so the GPU runtime image fails validation when CUDA is not present.
In `@docs/en/kubeflow/how_to/assets/training-runtimes/trainjob-smoke.yaml`:
- Line 13: The manifest hardcodes namespace: kubeflow-admin-cpaas-io which
breaks runtime resolution because spec.runtimeRef.name must exist in the same
namespace; remove the fixed "namespace" line (or replace it with a
template/placeholder that uses the target Profile namespace) so users can apply
the smoke manifest into their own Profile namespace, and add a brief note near
spec.runtimeRef.name to remind users that the referenced runtime must be created
in the same namespace as the manifest.
In `@docs/en/kubeflow/how_to/training-runtimes.mdx`:
- Around line 39-40: The doc line describing the image tag scheme
(v<semver>-build.<YYYYMMDDHHMMSS>) conflicts with the TrainingRuntime manifests
that use simple semantic tags like v0.1.0; update the sentence in the docs to
either (a) match the manifests by stating that manifests currently use plain
semantic tags (e.g., v0.1.0) or (b) clarify both formats and when each is used
by adding a short note that timestamped tags (v<semver>-build.<YYYYMMDDHHMMSS>)
are produced for rebuilt images while released manifests are pinned to simple
semantic tags (e.g., v0.1.0) — edit the paragraph that contains the tag scheme
text to implement one of these options and ensure the example tags in the doc
align with the TrainingRuntime YAMLs.
- Around line 71-73: The current sentence incorrectly tells users to "Replace
the placeholder image tag..." even though the YAMLs already contain concrete
tags; update the prose around TrainingRuntime/metadata.namespace and TrainJob to
note that manifests are pinned to a specific version and instruct users to
update the image: field if they need a different release (suggested phrasing:
"The manifests are pinned to a specific version; update image: if you need a
different release.").
---
Nitpick comments:
In
`@docs/en/kubeflow/how_to/assets/training-runtimes/llamafactory0.9-cann8.5-arm64-trainingruntime.yaml`:
- Line 42: The smoke-test line currently runs "llamafactory-cli version ||
true", which silences failures; remove the "|| true" so the step fails fast on
CLI errors and surfaces diagnostics, or if suppression is deliberate add an
inline comment explaining the known flakiness and why the failure is acceptable;
update the line that contains the exact token "llamafactory-cli version || true"
accordingly and ensure the job/step will fail when "llamafactory-cli version"
exits non‑zero.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 1db28062-bf1e-4cd2-a2bc-8c200768bc07
📒 Files selected for processing (8)
docs/en/kubeflow/how_to/assets/training-runtimes/llamafactory0.9-cann8.5-arm64-trainingruntime.yamldocs/en/kubeflow/how_to/assets/training-runtimes/llamafactory0.9-cu126-amd64-trainingruntime.yamldocs/en/kubeflow/how_to/assets/training-runtimes/mindspeed-llm-cann8.5-arm64-trainingruntime.yamldocs/en/kubeflow/how_to/assets/training-runtimes/torch2.6-cann8.5-arm64-trainingruntime.yamldocs/en/kubeflow/how_to/assets/training-runtimes/torch2.6-cu126-amd64-trainingruntime.yamldocs/en/kubeflow/how_to/assets/training-runtimes/traininghub0.1-cu126-amd64-trainingruntime.yamldocs/en/kubeflow/how_to/assets/training-runtimes/trainjob-smoke.yamldocs/en/kubeflow/how_to/training-runtimes.mdx
| # Synthetic identity SFT smoke. Writes a 4-row alpaca dataset, fine-tunes | ||
| # qwen-style chat template against an empty config to exercise the | ||
| # LLaMA-Factory CLI without downloading a real model. |
There was a problem hiding this comment.
Align the smoke comment with executed behavior.
Lines 39-41 say this block fine-tunes, but the script only prepares data and checks versions. This mismatch can mislead debugging and runbook expectations.
Suggested fix
- # Synthetic identity SFT smoke. Writes a 4-row alpaca dataset, fine-tunes
- # qwen-style chat template against an empty config to exercise the
- # LLaMA-Factory CLI without downloading a real model.
+ # Synthetic smoke: writes a 4-row alpaca dataset and validates
+ # LLaMA-Factory/PyTorch imports + CLI availability without downloading
+ # a real model or running full fine-tuning.📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| # Synthetic identity SFT smoke. Writes a 4-row alpaca dataset, fine-tunes | |
| # qwen-style chat template against an empty config to exercise the | |
| # LLaMA-Factory CLI without downloading a real model. | |
| # Synthetic smoke: writes a 4-row alpaca dataset and validates | |
| # LLaMA-Factory/PyTorch imports + CLI availability without downloading | |
| # a real model or running full fine-tuning. |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In
`@docs/en/kubeflow/how_to/assets/training-runtimes/llamafactory0.9-cu126-amd64-trainingruntime.yaml`
around lines 39 - 41, The comment block starting with "Synthetic identity SFT
smoke" (the lines that say "Writes a 4-row alpaca dataset, fine-tunes qwen-style
chat template...") is misleading because the YAML step only prepares data and
performs version checks; update that comment to accurately describe the executed
behavior (e.g., "prepares a 4-row alpaca dataset and validates LLaMA-Factory
CLI/model versions; does not run a fine-tune"), or alternatively modify the
associated step that invokes the LLaMA-Factory CLI to actually kick off a
fine-tune if the original intention was to run one; ensure the comment and the
step invoking the LLaMA-Factory CLI / data preparation are consistent.
| ] | ||
| JSON | ||
| python -c "import llamafactory, torch; print('lf', llamafactory.__version__, 'torch', torch.__version__, 'cuda', torch.cuda.is_available())" | ||
| llamafactory-cli version || true |
There was a problem hiding this comment.
Do not swallow llamafactory-cli failures in smoke validation.
Line 55 masks command failures with || true, so this runtime can report success even when the CLI is missing/broken.
Suggested fix
- llamafactory-cli version || true
+ llamafactory-cli version📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| llamafactory-cli version || true | |
| llamafactory-cli version |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In
`@docs/en/kubeflow/how_to/assets/training-runtimes/llamafactory0.9-cu126-amd64-trainingruntime.yaml`
at line 55, The smoke-check currently masks failures by appending "|| true" to
the "llamafactory-cli version" invocation, allowing the runtime to report
success even when the CLI is absent or failing; remove the "|| true" from the
"llamafactory-cli version" call (or replace it with an explicit
existence/version check that fails the step and surfaces/logs the error) so that
the command's non-zero exit propagates and the validation fails when the CLI is
missing or broken.
| if torch.cuda.is_available(): | ||
| x = torch.randn(1024, 1024, device='cuda') | ||
| y = x @ x.T | ||
| print(f"matmul ok, shape={tuple(y.shape)}, mean={y.mean().item():.4f}") |
There was a problem hiding this comment.
Fail fast when CUDA is unavailable in the smoke runtime.
Line 41 currently makes GPU validation optional; if CUDA is unavailable, the container still exits successfully, so the smoke TrainJob can pass with a broken GPU runtime.
Suggested fix
- if torch.cuda.is_available():
- x = torch.randn(1024, 1024, device='cuda')
- y = x @ x.T
- print(f"matmul ok, shape={tuple(y.shape)}, mean={y.mean().item():.4f}")
+ if not torch.cuda.is_available() or torch.cuda.device_count() < 1:
+ raise SystemExit("CUDA is not available; smoke check failed")
+ x = torch.randn(1024, 1024, device='cuda')
+ y = x @ x.T
+ print(f"matmul ok, shape={tuple(y.shape)}, mean={y.mean().item():.4f}")📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| if torch.cuda.is_available(): | |
| x = torch.randn(1024, 1024, device='cuda') | |
| y = x @ x.T | |
| print(f"matmul ok, shape={tuple(y.shape)}, mean={y.mean().item():.4f}") | |
| if not torch.cuda.is_available() or torch.cuda.device_count() < 1: | |
| raise SystemExit("CUDA is not available; smoke check failed") | |
| x = torch.randn(1024, 1024, device='cuda') | |
| y = x @ x.T | |
| print(f"matmul ok, shape={tuple(y.shape)}, mean={y.mean().item():.4f}") |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In
`@docs/en/kubeflow/how_to/assets/training-runtimes/torch2.6-cu126-amd64-trainingruntime.yaml`
around lines 41 - 44, The GPU smoke-check currently skips failure when CUDA is
missing because it only runs the matmul under if torch.cuda.is_available();
change this to fail fast: detect CUDA unavailability with
torch.cuda.is_available() and immediately print a clear error and exit non‑zero
(or raise an exception) so the container fails the smoke TrainJob; keep the
existing CUDA matmul path (creating x on device='cuda', computing y = x @ x.T
and printing the success message) but ensure the missing-CUDA branch aborts the
process.
| if torch.cuda.is_available(): | ||
| from peft import LoraConfig | ||
| cfg = LoraConfig(r=4, lora_alpha=8, lora_dropout=0.05, target_modules=["q_proj","v_proj"]) | ||
| print("peft LoraConfig ok", cfg) |
There was a problem hiding this comment.
Require CUDA for this GPU smoke runtime instead of treating it as optional.
At Line 41, CUDA-dependent checks are conditional, so a no-CUDA environment still succeeds. That weakens smoke validation for a CUDA runtime image.
Suggested fix
- if torch.cuda.is_available():
- from peft import LoraConfig
- cfg = LoraConfig(r=4, lora_alpha=8, lora_dropout=0.05, target_modules=["q_proj","v_proj"])
- print("peft LoraConfig ok", cfg)
+ if not torch.cuda.is_available() or torch.cuda.device_count() < 1:
+ raise SystemExit("CUDA is not available; smoke check failed")
+ from peft import LoraConfig
+ cfg = LoraConfig(r=4, lora_alpha=8, lora_dropout=0.05, target_modules=["q_proj","v_proj"])
+ print("peft LoraConfig ok", cfg)📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| if torch.cuda.is_available(): | |
| from peft import LoraConfig | |
| cfg = LoraConfig(r=4, lora_alpha=8, lora_dropout=0.05, target_modules=["q_proj","v_proj"]) | |
| print("peft LoraConfig ok", cfg) | |
| if not torch.cuda.is_available() or torch.cuda.device_count() < 1: | |
| raise SystemExit("CUDA is not available; smoke check failed") | |
| from peft import LoraConfig | |
| cfg = LoraConfig(r=4, lora_alpha=8, lora_dropout=0.05, target_modules=["q_proj","v_proj"]) | |
| print("peft LoraConfig ok", cfg) |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In
`@docs/en/kubeflow/how_to/assets/training-runtimes/traininghub0.1-cu126-amd64-trainingruntime.yaml`
around lines 41 - 44, The current smoke check wraps CUDA-dependent logic in an
if torch.cuda.is_available(): block so a no-CUDA environment still passes;
change this to require CUDA by checking torch.cuda.is_available() and failing
fast (raise a RuntimeError or call sys.exit(1)) if False before importing peft
and instantiating LoraConfig (symbols: torch.cuda.is_available, LoraConfig, cfg,
print), so the GPU runtime image fails validation when CUDA is not present.
| kind: TrainJob | ||
| metadata: | ||
| generateName: smoke- | ||
| namespace: kubeflow-admin-cpaas-io |
There was a problem hiding this comment.
Avoid hardcoded namespace in the shared smoke manifest.
Line 13 fixes the namespace to kubeflow-admin-cpaas-io; users often apply examples in their own Profile namespace, which then breaks runtime resolution (spec.runtimeRef.name must exist in the same namespace).
Suggested fix
metadata:
generateName: smoke-
- namespace: kubeflow-admin-cpaas-io
+ # Replace with your Profile namespace.
+ namespace: <your-namespace>🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/en/kubeflow/how_to/assets/training-runtimes/trainjob-smoke.yaml` at line
13, The manifest hardcodes namespace: kubeflow-admin-cpaas-io which breaks
runtime resolution because spec.runtimeRef.name must exist in the same
namespace; remove the fixed "namespace" line (or replace it with a
template/placeholder that uses the target Profile namespace) so users can apply
the smoke manifest into their own Profile namespace, and add a brief note near
spec.runtimeRef.name to remind users that the referenced runtime must be created
in the same namespace as the manifest.
| Tag scheme is `v<semver>-build.<YYYYMMDDHHMMSS>`; each rebuild gets a unique | ||
| tag so scan evidence stays correlated. |
There was a problem hiding this comment.
Clarify the tag scheme or update example tags.
The documentation describes a tag scheme with build timestamps (v<semver>-build.<YYYYMMDDHHMMSS>), but the actual TrainingRuntime YAML files use simple semantic versions like v0.1.0. This inconsistency may confuse users about which tag format to expect when pinning versions.
Consider either:
- Updating the description to match the actual tag format used in the manifests
- Or adding a note explaining when timestamped tags vs simple semantic versions are used
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/en/kubeflow/how_to/training-runtimes.mdx` around lines 39 - 40, The doc
line describing the image tag scheme (v<semver>-build.<YYYYMMDDHHMMSS>)
conflicts with the TrainingRuntime manifests that use simple semantic tags like
v0.1.0; update the sentence in the docs to either (a) match the manifests by
stating that manifests currently use plain semantic tags (e.g., v0.1.0) or (b)
clarify both formats and when each is used by adding a short note that
timestamped tags (v<semver>-build.<YYYYMMDDHHMMSS>) are produced for rebuilt
images while released manifests are pinned to simple semantic tags (e.g.,
v0.1.0) — edit the paragraph that contains the tag scheme text to implement one
of these options and ensure the example tags in the doc align with the
TrainingRuntime YAMLs.
Six curated TrainingRuntime images (PyTorch + LLaMA-Factory + TrainingHub on NVIDIA CUDA, plus torch_npu / LLaMA-Factory / MindSpeed-LLM on Huawei Ascend NPU via CANN) so users can submit TrainJobs without building a custom image. - training-runtimes.mdx: device + framework catalog, HAMI vGPU / vNPU resource model, NPU host-driver caveat, smoke TrainJob walkthrough. - assets/training-runtimes/*.yaml: ready-to-apply TrainingRuntime for each image plus a shared smoke TrainJob template. Source Containerfiles, multi-arch build helper, e2e harness, and post-fix security scan evidence live in alauda/kubeflow-plugin under training-runtimes/.
9751aa0 to
32f075a
Compare
There was a problem hiding this comment.
🧹 Nitpick comments (1)
docs/en/kubeflow/how_to/training-runtimes.mdx (1)
156-164: 💤 Low valueTerminology inconsistency: "adaptor" vs "adapter".
Line 160 uses "megatron_adaptor_v2" (likely the actual module name) while line 164 uses "adapter shim." While "adaptor" may be the correct module name, using both "adaptor" and "adapter" in the same paragraph could confuse readers about whether these refer to the same concept.
Consider adding a brief clarification, e.g., "...without going through the MindSpeed-LLM adaptor shim" to maintain consistent terminology.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/en/kubeflow/how_to/training-runtimes.mdx` around lines 156 - 164, The paragraph mixes "adaptor" and "adapter" which is inconsistent; update the text to use the module-correct spelling "adaptor" everywhere (e.g., change "adapter shim" to "adaptor shim") so readers know "megatron_adaptor_v2" and the MindSpeed-LLM shim refer to the same thing, and ensure mentions of mindspeed_llm and MindSpeed-LLM use consistent casing/terminology.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@docs/en/kubeflow/how_to/training-runtimes.mdx`:
- Around line 156-164: The paragraph mixes "adaptor" and "adapter" which is
inconsistent; update the text to use the module-correct spelling "adaptor"
everywhere (e.g., change "adapter shim" to "adaptor shim") so readers know
"megatron_adaptor_v2" and the MindSpeed-LLM shim refer to the same thing, and
ensure mentions of mindspeed_llm and MindSpeed-LLM use consistent
casing/terminology.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 06b32e25-623d-4ee2-a869-8d8e2c135d04
📒 Files selected for processing (8)
docs/en/kubeflow/how_to/assets/training-runtimes/llamafactory0.9-cann8.5-arm64-trainingruntime.yamldocs/en/kubeflow/how_to/assets/training-runtimes/llamafactory0.9-cu126-amd64-trainingruntime.yamldocs/en/kubeflow/how_to/assets/training-runtimes/mindspeed-llm-cann8.5-arm64-trainingruntime.yamldocs/en/kubeflow/how_to/assets/training-runtimes/torch2.6-cann8.5-arm64-trainingruntime.yamldocs/en/kubeflow/how_to/assets/training-runtimes/torch2.6-cu126-amd64-trainingruntime.yamldocs/en/kubeflow/how_to/assets/training-runtimes/traininghub0.1-cu126-amd64-trainingruntime.yamldocs/en/kubeflow/how_to/assets/training-runtimes/trainjob-smoke.yamldocs/en/kubeflow/how_to/training-runtimes.mdx
✅ Files skipped from review due to trivial changes (1)
- docs/en/kubeflow/how_to/assets/training-runtimes/trainjob-smoke.yaml
🚧 Files skipped from review as they are similar to previous changes (5)
- docs/en/kubeflow/how_to/assets/training-runtimes/torch2.6-cann8.5-arm64-trainingruntime.yaml
- docs/en/kubeflow/how_to/assets/training-runtimes/traininghub0.1-cu126-amd64-trainingruntime.yaml
- docs/en/kubeflow/how_to/assets/training-runtimes/mindspeed-llm-cann8.5-arm64-trainingruntime.yaml
- docs/en/kubeflow/how_to/assets/training-runtimes/llamafactory0.9-cu126-amd64-trainingruntime.yaml
- docs/en/kubeflow/how_to/assets/training-runtimes/torch2.6-cu126-amd64-trainingruntime.yaml
Six curated TrainingRuntime images (PyTorch + LLaMA-Factory + TrainingHub on NVIDIA CUDA, plus torch_npu / LLaMA-Factory / MindSpeed-LLM on Huawei Ascend NPU via CANN) so users can submit TrainJobs without building a custom image.
Source Containerfiles, multi-arch build helper, e2e harness, and post-fix security scan evidence live in alauda/kubeflow-plugin under training-runtimes/.
Summary by CodeRabbit