[Docs] Add documentation for show_metrics(), stream_logs(), and job notification setup

docs/model_customization/model_customization.rst

@@ -53,6 +53,116 @@ Key Features
  with clear precedence. Use ``get_resolved_recipe()`` to inspect the merged configuration
  before job submission. See :doc:`finetuning_serverful` and :doc:`finetuning_hyperpod` for examples.
 
+**Job Notifications**
+ Receive SNS notifications when training jobs complete, fail, or stop. Uses EventBridge
+ rules to route SageMaker Training Job status changes to your SNS topic. 
+
+ The SDK creates one rule per unique config (topic + events + prefix). Re-running with 
+ the same config reuses the existing rule. Different configs create separate rules.
+
+ .. note::
+ Supported for SMTJ (serverful and serverless) compute only. HyperPod is not currently supported.
+
+ .. code-block:: python
+
+ trainer = SFTTrainer(
+ model="nova-textgeneration-micro",
+ training_dataset="s3://my-bucket/train.jsonl",
+ accept_eula=True,
+ notifications={
+ "sns_topic_arn": "arn:aws:sns:us-east-1:123456789012:my-topic", # Required
+ "events": ["Completed", "Failed"], # Optional (default: Completed, Failed, Stopped)
+ "job_name_prefix": "my-team-sft-", # Optional: filter by job name
+ "event_bus_arn": "arn:aws:events:us-east-1:123456789012:event-bus/custom-bus" # Optional
+ },
+ )
+ job = trainer.train(wait=False) # Notification sent on completion
+
+ # Access the rule ARN
+ print(trainer.notification_rule_arn)
+
+ # List and manage rules
+ rules = trainer.list_notification_rules()
+ trainer.delete_notification_rule(rule_arn=trainer.notification_rule_arn)
+
+ **Prerequisites:**
+
+ - An SNS topic (and subscription) with a resource policy allowing ``events.amazonaws.com``. 
+ To set up a topic and subscription, see `Creating an SNS topic and subscription <https://docs.aws.amazon.com/sns/latest/dg/sns-create-subscribe-endpoint-to-topic.html>`_. 
+ - IAM permissions: ``events:PutRule``, ``events:PutTargets``, ``events:ListRules``,
+ ``events:RemoveTargets``, ``events:DeleteRule``
+
+**Monitoring: show_metrics()**
+ Plot training metrics after a job completes. Works across all compute types.
+
+ - **Nova models**: Metrics parsed from CloudWatch logs.
+ - **OSS models**: Metrics pulled from MLflow.
+
+ .. code-block:: python
+
+ # Plot all available metrics
+ df = trainer.show_metrics()
+
+ # Plot specific metrics
+ df = trainer.show_metrics(metrics=["training_loss", "lr"])
+
+ # Filter by step range
+ df = trainer.show_metrics(starting_step=10, ending_step=100)
+
+ # Filter by time window
+ from datetime import datetime
+ df = trainer.show_metrics(
+ start_time=datetime(2026, 1, 1, 10, 0, 0),
+ end_time=datetime(2026, 1, 1, 12, 0, 0),
+ )
+
+ **After a kernel restart:**
+
+ .. code-block:: python
+
+ # Standalone (SMTJ)
+ from sagemaker.train import plot_training_metrics
+ plot_training_metrics("my-sft-job")
+
+ # Re-attach (HyperPod — needs cluster name for log group resolution)
+ from sagemaker.train import SFTTrainer
+ from sagemaker.core.training.configs import HyperPodCompute
+
+ trainer = SFTTrainer(
+ model="nova-textgeneration-micro",
+ training_dataset="s3://unused",
+ compute=HyperPodCompute(cluster_name="my-cluster", instance_type="ml.p5.48xlarge")
+ )
+ trainer._latest_training_job = "my-hp-job"
+ df = trainer.show_metrics()
+
+**Monitoring: stream_logs()**
+ Stream CloudWatch logs in real-time while a job is running.
+
+ .. code-block:: python
+
+ # Start training non-blocking
+ job = trainer.train(wait=False)
+
+ # Stream logs (blocks until job completes or Ctrl+C)
+ trainer.stream_logs()
+
+ # Custom polling interval (seconds)
+ trainer.stream_logs(poll=10)
+
+ # Stream from a specific start time - providing this will speed up execution.
+ from datetime import datetime
+ trainer.stream_logs(start_time=datetime(2026, 1, 1, 15, 0, 0))
+
+ .. note::
+
+ - **SMTJ**: Streaming auto-stops when the job reaches a terminal state.
+ - **HyperPod**: Streaming runs until you press Ctrl+C. Logs may take a few minutes
+ to propagate to CloudWatch on first run.
+
+
+----
+
 
 .. toctree::
  :maxdepth: 1

docs/model_customization/model_customization.rst

@@ -53,6 +53,116 @@ Key Features
  with clear precedence. Use ``get_resolved_recipe()`` to inspect the merged configuration
  before job submission. See :doc:`finetuning_serverful` and :doc:`finetuning_hyperpod` for examples.
 
+**Job Notifications**
+ Receive SNS notifications when training jobs complete, fail, or stop. Uses EventBridge
+ rules to route SageMaker Training Job status changes to your SNS topic. 
+
+ The SDK creates one rule per unique config (topic + events + prefix). Re-running with 
+ the same config reuses the existing rule. Different configs create separate rules.
+
+ .. note::
+ Supported for SMTJ (serverful and serverless) compute only. HyperPod is not currently supported.
+
+ .. code-block:: python
+
+ trainer = SFTTrainer(
+ model="nova-textgeneration-micro",
+ training_dataset="s3://my-bucket/train.jsonl",
+ accept_eula=True,
+ notifications={
+ "sns_topic_arn": "arn:aws:sns:us-east-1:123456789012:my-topic", # Required
+ "events": ["Completed", "Failed"], # Optional (default: Completed, Failed, Stopped)
+ "job_name_prefix": "my-team-sft-", # Optional: filter by job name
+ "event_bus_arn": "arn:aws:events:us-east-1:123456789012:event-bus/custom-bus" # Optional
+ },
+ )
+ job = trainer.train(wait=False) # Notification sent on completion
+
+ # Access the rule ARN
+ print(trainer.notification_rule_arn)
+
+ # List and manage rules
+ rules = trainer.list_notification_rules()
+ trainer.delete_notification_rule(rule_arn=trainer.notification_rule_arn)
+
+ **Prerequisites:**
+
+ - An SNS topic (and subscription) with a resource policy allowing ``events.amazonaws.com``. 
+ To set up a topic and subscription, see `Creating an SNS topic and subscription <https://docs.aws.amazon.com/sns/latest/dg/sns-create-subscribe-endpoint-to-topic.html>`_. 
+ - IAM permissions: ``events:PutRule``, ``events:PutTargets``, ``events:ListRules``,
+ ``events:RemoveTargets``, ``events:DeleteRule``
+
+**Monitoring: show_metrics()**
+ Plot training metrics after a job completes. Works across all compute types.
+
+ - **Nova models**: Metrics parsed from CloudWatch logs.
+ - **OSS models**: Metrics pulled from MLflow.
+
+ .. code-block:: python
+
+ # Plot all available metrics
+ df = trainer.show_metrics()
+
+ # Plot specific metrics
+ df = trainer.show_metrics(metrics=["training_loss", "lr"])
+
+ # Filter by step range
+ df = trainer.show_metrics(starting_step=10, ending_step=100)
+
+ # Filter by time window
+ from datetime import datetime
+ df = trainer.show_metrics(
+ start_time=datetime(2026, 1, 1, 10, 0, 0),
+ end_time=datetime(2026, 1, 1, 12, 0, 0),
+ )
+
+ **After a kernel restart:**
+
+ .. code-block:: python
+
+ # Standalone (SMTJ)
+ from sagemaker.train import plot_training_metrics
+ plot_training_metrics("my-sft-job")
+
+ # Re-attach (HyperPod — needs cluster name for log group resolution)
+ from sagemaker.train import SFTTrainer
+ from sagemaker.core.training.configs import HyperPodCompute
+
+ trainer = SFTTrainer(
+ model="nova-textgeneration-micro",
+ training_dataset="s3://unused",
+ compute=HyperPodCompute(cluster_name="my-cluster", instance_type="ml.p5.48xlarge")
+ )
+ trainer._latest_training_job = "my-hp-job"
+ df = trainer.show_metrics()
+
+**Monitoring: stream_logs()**
+ Stream CloudWatch logs in real-time while a job is running.
+
+ .. code-block:: python
+
+ # Start training non-blocking
+ job = trainer.train(wait=False)
+
+ # Stream logs (blocks until job completes or Ctrl+C)
+ trainer.stream_logs()
+
+ # Custom polling interval (seconds)
+ trainer.stream_logs(poll=10)
+
+ # Stream from a specific start time - providing this will speed up execution.
+ from datetime import datetime
+ trainer.stream_logs(start_time=datetime(2026, 1, 1, 15, 0, 0))
+
+ .. note::
+
+ - **SMTJ**: Streaming auto-stops when the job reaches a terminal state.
+ - **HyperPod**: Streaming runs until you press Ctrl+C. Logs may take a few minutes
+ to propagate to CloudWatch on first run.
+
+
+----
+
 
 .. toctree::
  :maxdepth: 1

docs/model_customization/model_customization.rst

@@ -53,6 +53,116 @@ Key Features
  with clear precedence. Use ``get_resolved_recipe()`` to inspect the merged configuration
  before job submission. See :doc:`finetuning_serverful` and :doc:`finetuning_hyperpod` for examples.
 
+**Job Notifications**
+ Receive SNS notifications when training jobs complete, fail, or stop. Uses EventBridge
+ rules to route SageMaker Training Job status changes to your SNS topic. 
+
+ The SDK creates one rule per unique config (topic + events + prefix). Re-running with 
+ the same config reuses the existing rule. Different configs create separate rules.
+
+ .. note::
+ Supported for SMTJ (serverful and serverless) compute only. HyperPod is not currently supported.
+
+ .. code-block:: python
+
+ trainer = SFTTrainer(
+ model="nova-textgeneration-micro",
+ training_dataset="s3://my-bucket/train.jsonl",
+ accept_eula=True,
+ notifications={
+ "sns_topic_arn": "arn:aws:sns:us-east-1:123456789012:my-topic", # Required
+ "events": ["Completed", "Failed"], # Optional (default: Completed, Failed, Stopped)
+ "job_name_prefix": "my-team-sft-", # Optional: filter by job name
+ "event_bus_arn": "arn:aws:events:us-east-1:123456789012:event-bus/custom-bus" # Optional
+ },
+ )
+ job = trainer.train(wait=False) # Notification sent on completion
+
+ # Access the rule ARN
+ print(trainer.notification_rule_arn)
+
+ # List and manage rules
+ rules = trainer.list_notification_rules()
+ trainer.delete_notification_rule(rule_arn=trainer.notification_rule_arn)
+
+ **Prerequisites:**
+
+ - An SNS topic (and subscription) with a resource policy allowing ``events.amazonaws.com``. 
+ To set up a topic and subscription, see `Creating an SNS topic and subscription <https://docs.aws.amazon.com/sns/latest/dg/sns-create-subscribe-endpoint-to-topic.html>`_. 
+ - IAM permissions: ``events:PutRule``, ``events:PutTargets``, ``events:ListRules``,
+ ``events:RemoveTargets``, ``events:DeleteRule``
+
+**Monitoring: show_metrics()**
+ Plot training metrics after a job completes. Works across all compute types.
+
+ - **Nova models**: Metrics parsed from CloudWatch logs.
+ - **OSS models**: Metrics pulled from MLflow.
+
+ .. code-block:: python
+
+ # Plot all available metrics
+ df = trainer.show_metrics()
+
+ # Plot specific metrics
+ df = trainer.show_metrics(metrics=["training_loss", "lr"])
+
+ # Filter by step range
+ df = trainer.show_metrics(starting_step=10, ending_step=100)
+
+ # Filter by time window
+ from datetime import datetime
+ df = trainer.show_metrics(
+ start_time=datetime(2026, 1, 1, 10, 0, 0),
+ end_time=datetime(2026, 1, 1, 12, 0, 0),
+ )
+
+ **After a kernel restart:**
+
+ .. code-block:: python
+
+ # Standalone (SMTJ)
+ from sagemaker.train import plot_training_metrics
+ plot_training_metrics("my-sft-job")
+
+ # Re-attach (HyperPod — needs cluster name for log group resolution)
+ from sagemaker.train import SFTTrainer
+ from sagemaker.core.training.configs import HyperPodCompute
+
+ trainer = SFTTrainer(
+ model="nova-textgeneration-micro",
+ training_dataset="s3://unused",
+ compute=HyperPodCompute(cluster_name="my-cluster", instance_type="ml.p5.48xlarge")
+ )
+ trainer._latest_training_job = "my-hp-job"
+ df = trainer.show_metrics()
+
+**Monitoring: stream_logs()**
+ Stream CloudWatch logs in real-time while a job is running.
+
+ .. code-block:: python
+
+ # Start training non-blocking
+ job = trainer.train(wait=False)
+
+ # Stream logs (blocks until job completes or Ctrl+C)
+ trainer.stream_logs()
+
+ # Custom polling interval (seconds)
+ trainer.stream_logs(poll=10)
+
+ # Stream from a specific start time - providing this will speed up execution.
+ from datetime import datetime
+ trainer.stream_logs(start_time=datetime(2026, 1, 1, 15, 0, 0))
+
+ .. note::
+
+ - **SMTJ**: Streaming auto-stops when the job reaches a terminal state.
+ - **HyperPod**: Streaming runs until you press Ctrl+C. Logs may take a few minutes
+ to propagate to CloudWatch on first run.
+
+
+----
+
 
 .. toctree::
  :maxdepth: 1

docs/model_customization/model_customization.rst

@@ -53,6 +53,116 @@ Key Features
  with clear precedence. Use ``get_resolved_recipe()`` to inspect the merged configuration
  before job submission. See :doc:`finetuning_serverful` and :doc:`finetuning_hyperpod` for examples.
 
+**Job Notifications**
+ Receive SNS notifications when training jobs complete, fail, or stop. Uses EventBridge
+ rules to route SageMaker Training Job status changes to your SNS topic. 
+
+ The SDK creates one rule per unique config (topic + events + prefix). Re-running with 
+ the same config reuses the existing rule. Different configs create separate rules.
+
+ .. note::
+ Supported for SMTJ (serverful and serverless) compute only. HyperPod is not currently supported.
+
+ .. code-block:: python
+
+ trainer = SFTTrainer(
+ model="nova-textgeneration-micro",
+ training_dataset="s3://my-bucket/train.jsonl",
+ accept_eula=True,
+ notifications={
+ "sns_topic_arn": "arn:aws:sns:us-east-1:123456789012:my-topic", # Required
+ "events": ["Completed", "Failed"], # Optional (default: Completed, Failed, Stopped)
+ "job_name_prefix": "my-team-sft-", # Optional: filter by job name
+ "event_bus_arn": "arn:aws:events:us-east-1:123456789012:event-bus/custom-bus" # Optional
+ },
+ )
+ job = trainer.train(wait=False) # Notification sent on completion
+
+ # Access the rule ARN
+ print(trainer.notification_rule_arn)
+
+ # List and manage rules
+ rules = trainer.list_notification_rules()
+ trainer.delete_notification_rule(rule_arn=trainer.notification_rule_arn)
+
+ **Prerequisites:**
+
+ - An SNS topic (and subscription) with a resource policy allowing ``events.amazonaws.com``. 
+ To set up a topic and subscription, see `Creating an SNS topic and subscription <https://docs.aws.amazon.com/sns/latest/dg/sns-create-subscribe-endpoint-to-topic.html>`_. 
+ - IAM permissions: ``events:PutRule``, ``events:PutTargets``, ``events:ListRules``,
+ ``events:RemoveTargets``, ``events:DeleteRule``
+
+**Monitoring: show_metrics()**
+ Plot training metrics after a job completes. Works across all compute types.
+
+ - **Nova models**: Metrics parsed from CloudWatch logs.
+ - **OSS models**: Metrics pulled from MLflow.
+
+ .. code-block:: python
+
+ # Plot all available metrics
+ df = trainer.show_metrics()
+
+ # Plot specific metrics
+ df = trainer.show_metrics(metrics=["training_loss", "lr"])
+
+ # Filter by step range
+ df = trainer.show_metrics(starting_step=10, ending_step=100)
+
+ # Filter by time window
+ from datetime import datetime
+ df = trainer.show_metrics(
+ start_time=datetime(2026, 1, 1, 10, 0, 0),
+ end_time=datetime(2026, 1, 1, 12, 0, 0),
+ )
+
+ **After a kernel restart:**
+
+ .. code-block:: python
+
+ # Standalone (SMTJ)
+ from sagemaker.train import plot_training_metrics
+ plot_training_metrics("my-sft-job")
+
+ # Re-attach (HyperPod — needs cluster name for log group resolution)
+ from sagemaker.train import SFTTrainer
+ from sagemaker.core.training.configs import HyperPodCompute
+
+ trainer = SFTTrainer(
+ model="nova-textgeneration-micro",
+ training_dataset="s3://unused",
+ compute=HyperPodCompute(cluster_name="my-cluster", instance_type="ml.p5.48xlarge")
+ )
+ trainer._latest_training_job = "my-hp-job"
+ df = trainer.show_metrics()
+
+**Monitoring: stream_logs()**
+ Stream CloudWatch logs in real-time while a job is running.
+
+ .. code-block:: python
+
+ # Start training non-blocking
+ job = trainer.train(wait=False)
+
+ # Stream logs (blocks until job completes or Ctrl+C)
+ trainer.stream_logs()
+
+ # Custom polling interval (seconds)
+ trainer.stream_logs(poll=10)
+
+ # Stream from a specific start time - providing this will speed up execution.
+ from datetime import datetime
+ trainer.stream_logs(start_time=datetime(2026, 1, 1, 15, 0, 0))
+
+ .. note::
+
+ - **SMTJ**: Streaming auto-stops when the job reaches a terminal state.
+ - **HyperPod**: Streaming runs until you press Ctrl+C. Logs may take a few minutes
+ to propagate to CloudWatch on first run.
+
+
+----
+
 
 .. toctree::
  :maxdepth: 1

docs/model_customization/model_customization.rst

@@ -53,6 +53,116 @@ Key Features
  with clear precedence. Use ``get_resolved_recipe()`` to inspect the merged configuration
  before job submission. See :doc:`finetuning_serverful` and :doc:`finetuning_hyperpod` for examples.
 
+**Job Notifications**
+ Receive SNS notifications when training jobs complete, fail, or stop. Uses EventBridge
+ rules to route SageMaker Training Job status changes to your SNS topic. 
+
+ The SDK creates one rule per unique config (topic + events + prefix). Re-running with 
+ the same config reuses the existing rule. Different configs create separate rules.
+
+ .. note::
+ Supported for SMTJ (serverful and serverless) compute only. HyperPod is not currently supported.
+
+ .. code-block:: python
+
+ trainer = SFTTrainer(
+ model="nova-textgeneration-micro",
+ training_dataset="s3://my-bucket/train.jsonl",
+ accept_eula=True,
+ notifications={
+ "sns_topic_arn": "arn:aws:sns:us-east-1:123456789012:my-topic", # Required
+ "events": ["Completed", "Failed"], # Optional (default: Completed, Failed, Stopped)
+ "job_name_prefix": "my-team-sft-", # Optional: filter by job name
+ "event_bus_arn": "arn:aws:events:us-east-1:123456789012:event-bus/custom-bus" # Optional
+ },
+ )
+ job = trainer.train(wait=False) # Notification sent on completion
+
+ # Access the rule ARN
+ print(trainer.notification_rule_arn)
+
+ # List and manage rules
+ rules = trainer.list_notification_rules()
+ trainer.delete_notification_rule(rule_arn=trainer.notification_rule_arn)
+
+ **Prerequisites:**
+
+ - An SNS topic (and subscription) with a resource policy allowing ``events.amazonaws.com``. 
+ To set up a topic and subscription, see `Creating an SNS topic and subscription <https://docs.aws.amazon.com/sns/latest/dg/sns-create-subscribe-endpoint-to-topic.html>`_. 
+ - IAM permissions: ``events:PutRule``, ``events:PutTargets``, ``events:ListRules``,
+ ``events:RemoveTargets``, ``events:DeleteRule``
+
+**Monitoring: show_metrics()**
+ Plot training metrics after a job completes. Works across all compute types.
+
+ - **Nova models**: Metrics parsed from CloudWatch logs.
+ - **OSS models**: Metrics pulled from MLflow.
+
+ .. code-block:: python
+
+ # Plot all available metrics
+ df = trainer.show_metrics()
+
+ # Plot specific metrics
+ df = trainer.show_metrics(metrics=["training_loss", "lr"])
+
+ # Filter by step range
+ df = trainer.show_metrics(starting_step=10, ending_step=100)
+
+ # Filter by time window
+ from datetime import datetime
+ df = trainer.show_metrics(
+ start_time=datetime(2026, 1, 1, 10, 0, 0),
+ end_time=datetime(2026, 1, 1, 12, 0, 0),
+ )
+
+ **After a kernel restart:**
+
+ .. code-block:: python
+
+ # Standalone (SMTJ)
+ from sagemaker.train import plot_training_metrics
+ plot_training_metrics("my-sft-job")
+
+ # Re-attach (HyperPod — needs cluster name for log group resolution)
+ from sagemaker.train import SFTTrainer
+ from sagemaker.core.training.configs import HyperPodCompute
+
+ trainer = SFTTrainer(
+ model="nova-textgeneration-micro",
+ training_dataset="s3://unused",
+ compute=HyperPodCompute(cluster_name="my-cluster", instance_type="ml.p5.48xlarge")
+ )
+ trainer._latest_training_job = "my-hp-job"
+ df = trainer.show_metrics()
+
+**Monitoring: stream_logs()**
+ Stream CloudWatch logs in real-time while a job is running.
+
+ .. code-block:: python
+
+ # Start training non-blocking
+ job = trainer.train(wait=False)
+
+ # Stream logs (blocks until job completes or Ctrl+C)
+ trainer.stream_logs()
+
+ # Custom polling interval (seconds)
+ trainer.stream_logs(poll=10)
+
+ # Stream from a specific start time - providing this will speed up execution.
+ from datetime import datetime
+ trainer.stream_logs(start_time=datetime(2026, 1, 1, 15, 0, 0))
+
+ .. note::
+
+ - **SMTJ**: Streaming auto-stops when the job reaches a terminal state.
+ - **HyperPod**: Streaming runs until you press Ctrl+C. Logs may take a few minutes
+ to propagate to CloudWatch on first run.
+
+
+----
+
 
 .. toctree::
  :maxdepth: 1

docs/model_customization/model_customization.rst

@@ -53,6 +53,116 @@ Key Features
  with clear precedence. Use ``get_resolved_recipe()`` to inspect the merged configuration
  before job submission. See :doc:`finetuning_serverful` and :doc:`finetuning_hyperpod` for examples.
 
+**Job Notifications**
+ Receive SNS notifications when training jobs complete, fail, or stop. Uses EventBridge
+ rules to route SageMaker Training Job status changes to your SNS topic. 
+
+ The SDK creates one rule per unique config (topic + events + prefix). Re-running with 
+ the same config reuses the existing rule. Different configs create separate rules.
+
+ .. note::
+ Supported for SMTJ (serverful and serverless) compute only. HyperPod is not currently supported.
+
+ .. code-block:: python
+
+ trainer = SFTTrainer(
+ model="nova-textgeneration-micro",
+ training_dataset="s3://my-bucket/train.jsonl",
+ accept_eula=True,
+ notifications={
+ "sns_topic_arn": "arn:aws:sns:us-east-1:123456789012:my-topic", # Required
+ "events": ["Completed", "Failed"], # Optional (default: Completed, Failed, Stopped)
+ "job_name_prefix": "my-team-sft-", # Optional: filter by job name
+ "event_bus_arn": "arn:aws:events:us-east-1:123456789012:event-bus/custom-bus" # Optional
+ },
+ )
+ job = trainer.train(wait=False) # Notification sent on completion
+
+ # Access the rule ARN
+ print(trainer.notification_rule_arn)
+
+ # List and manage rules
+ rules = trainer.list_notification_rules()
+ trainer.delete_notification_rule(rule_arn=trainer.notification_rule_arn)
+
+ **Prerequisites:**
+
+ - An SNS topic (and subscription) with a resource policy allowing ``events.amazonaws.com``. 
+ To set up a topic and subscription, see `Creating an SNS topic and subscription <https://docs.aws.amazon.com/sns/latest/dg/sns-create-subscribe-endpoint-to-topic.html>`_. 
+ - IAM permissions: ``events:PutRule``, ``events:PutTargets``, ``events:ListRules``,
+ ``events:RemoveTargets``, ``events:DeleteRule``
+
+**Monitoring: show_metrics()**
+ Plot training metrics after a job completes. Works across all compute types.
+
+ - **Nova models**: Metrics parsed from CloudWatch logs.
+ - **OSS models**: Metrics pulled from MLflow.
+
+ .. code-block:: python
+
+ # Plot all available metrics
+ df = trainer.show_metrics()
+
+ # Plot specific metrics
+ df = trainer.show_metrics(metrics=["training_loss", "lr"])
+
+ # Filter by step range
+ df = trainer.show_metrics(starting_step=10, ending_step=100)
+
+ # Filter by time window
+ from datetime import datetime
+ df = trainer.show_metrics(
+ start_time=datetime(2026, 1, 1, 10, 0, 0),
+ end_time=datetime(2026, 1, 1, 12, 0, 0),
+ )
+
+ **After a kernel restart:**
+
+ .. code-block:: python
+
+ # Standalone (SMTJ)
+ from sagemaker.train import plot_training_metrics
+ plot_training_metrics("my-sft-job")
+
+ # Re-attach (HyperPod — needs cluster name for log group resolution)
+ from sagemaker.train import SFTTrainer
+ from sagemaker.core.training.configs import HyperPodCompute
+
+ trainer = SFTTrainer(
+ model="nova-textgeneration-micro",
+ training_dataset="s3://unused",
+ compute=HyperPodCompute(cluster_name="my-cluster", instance_type="ml.p5.48xlarge")
+ )
+ trainer._latest_training_job = "my-hp-job"
+ df = trainer.show_metrics()
+
+**Monitoring: stream_logs()**
+ Stream CloudWatch logs in real-time while a job is running.
+
+ .. code-block:: python
+
+ # Start training non-blocking
+ job = trainer.train(wait=False)
+
+ # Stream logs (blocks until job completes or Ctrl+C)
+ trainer.stream_logs()
+
+ # Custom polling interval (seconds)
+ trainer.stream_logs(poll=10)
+
+ # Stream from a specific start time - providing this will speed up execution.
+ from datetime import datetime
+ trainer.stream_logs(start_time=datetime(2026, 1, 1, 15, 0, 0))
+
+ .. note::
+
+ - **SMTJ**: Streaming auto-stops when the job reaches a terminal state.
+ - **HyperPod**: Streaming runs until you press Ctrl+C. Logs may take a few minutes
+ to propagate to CloudWatch on first run.
+
+
+----
+
 
 .. toctree::
  :maxdepth: 1

docs/model_customization/model_customization.rst

@@ -53,6 +53,116 @@ Key Features
  with clear precedence. Use ``get_resolved_recipe()`` to inspect the merged configuration
  before job submission. See :doc:`finetuning_serverful` and :doc:`finetuning_hyperpod` for examples.
 
+**Job Notifications**
+ Receive SNS notifications when training jobs complete, fail, or stop. Uses EventBridge
+ rules to route SageMaker Training Job status changes to your SNS topic. 
+
+ The SDK creates one rule per unique config (topic + events + prefix). Re-running with 
+ the same config reuses the existing rule. Different configs create separate rules.
+
+ .. note::
+ Supported for SMTJ (serverful and serverless) compute only. HyperPod is not currently supported.
+
+ .. code-block:: python
+
+ trainer = SFTTrainer(
+ model="nova-textgeneration-micro",
+ training_dataset="s3://my-bucket/train.jsonl",
+ accept_eula=True,
+ notifications={
+ "sns_topic_arn": "arn:aws:sns:us-east-1:123456789012:my-topic", # Required
+ "events": ["Completed", "Failed"], # Optional (default: Completed, Failed, Stopped)
+ "job_name_prefix": "my-team-sft-", # Optional: filter by job name
+ "event_bus_arn": "arn:aws:events:us-east-1:123456789012:event-bus/custom-bus" # Optional
+ },
+ )
+ job = trainer.train(wait=False) # Notification sent on completion
+
+ # Access the rule ARN
+ print(trainer.notification_rule_arn)
+
+ # List and manage rules
+ rules = trainer.list_notification_rules()
+ trainer.delete_notification_rule(rule_arn=trainer.notification_rule_arn)
+
+ **Prerequisites:**
+
+ - An SNS topic (and subscription) with a resource policy allowing ``events.amazonaws.com``. 
+ To set up a topic and subscription, see `Creating an SNS topic and subscription <https://docs.aws.amazon.com/sns/latest/dg/sns-create-subscribe-endpoint-to-topic.html>`_. 
+ - IAM permissions: ``events:PutRule``, ``events:PutTargets``, ``events:ListRules``,
+ ``events:RemoveTargets``, ``events:DeleteRule``
+
+**Monitoring: show_metrics()**
+ Plot training metrics after a job completes. Works across all compute types.
+
+ - **Nova models**: Metrics parsed from CloudWatch logs.
+ - **OSS models**: Metrics pulled from MLflow.
+
+ .. code-block:: python
+
+ # Plot all available metrics
+ df = trainer.show_metrics()
+
+ # Plot specific metrics
+ df = trainer.show_metrics(metrics=["training_loss", "lr"])
+
+ # Filter by step range
+ df = trainer.show_metrics(starting_step=10, ending_step=100)
+
+ # Filter by time window
+ from datetime import datetime
+ df = trainer.show_metrics(
+ start_time=datetime(2026, 1, 1, 10, 0, 0),
+ end_time=datetime(2026, 1, 1, 12, 0, 0),
+ )
+
+ **After a kernel restart:**
+
+ .. code-block:: python
+
+ # Standalone (SMTJ)
+ from sagemaker.train import plot_training_metrics
+ plot_training_metrics("my-sft-job")
+
+ # Re-attach (HyperPod — needs cluster name for log group resolution)
+ from sagemaker.train import SFTTrainer
+ from sagemaker.core.training.configs import HyperPodCompute
+
+ trainer = SFTTrainer(
+ model="nova-textgeneration-micro",
+ training_dataset="s3://unused",
+ compute=HyperPodCompute(cluster_name="my-cluster", instance_type="ml.p5.48xlarge")
+ )
+ trainer._latest_training_job = "my-hp-job"
+ df = trainer.show_metrics()
+
+**Monitoring: stream_logs()**
+ Stream CloudWatch logs in real-time while a job is running.
+
+ .. code-block:: python
+
+ # Start training non-blocking
+ job = trainer.train(wait=False)
+
+ # Stream logs (blocks until job completes or Ctrl+C)
+ trainer.stream_logs()
+
+ # Custom polling interval (seconds)
+ trainer.stream_logs(poll=10)
+
+ # Stream from a specific start time - providing this will speed up execution.
+ from datetime import datetime
+ trainer.stream_logs(start_time=datetime(2026, 1, 1, 15, 0, 0))
+
+ .. note::
+
+ - **SMTJ**: Streaming auto-stops when the job reaches a terminal state.
+ - **HyperPod**: Streaming runs until you press Ctrl+C. Logs may take a few minutes
+ to propagate to CloudWatch on first run.
+
+
+----
+
 
 .. toctree::
  :maxdepth: 1

docs/model_customization/model_customization.rst

@@ -53,6 +53,116 @@ Key Features
  with clear precedence. Use ``get_resolved_recipe()`` to inspect the merged configuration
  before job submission. See :doc:`finetuning_serverful` and :doc:`finetuning_hyperpod` for examples.
 
+**Job Notifications**
+ Receive SNS notifications when training jobs complete, fail, or stop. Uses EventBridge
+ rules to route SageMaker Training Job status changes to your SNS topic. 
+
+ The SDK creates one rule per unique config (topic + events + prefix). Re-running with 
+ the same config reuses the existing rule. Different configs create separate rules.
+
+ .. note::
+ Supported for SMTJ (serverful and serverless) compute only. HyperPod is not currently supported.
+
+ .. code-block:: python
+
+ trainer = SFTTrainer(
+ model="nova-textgeneration-micro",
+ training_dataset="s3://my-bucket/train.jsonl",
+ accept_eula=True,
+ notifications={
+ "sns_topic_arn": "arn:aws:sns:us-east-1:123456789012:my-topic", # Required
+ "events": ["Completed", "Failed"], # Optional (default: Completed, Failed, Stopped)
+ "job_name_prefix": "my-team-sft-", # Optional: filter by job name
+ "event_bus_arn": "arn:aws:events:us-east-1:123456789012:event-bus/custom-bus" # Optional
+ },
+ )
+ job = trainer.train(wait=False) # Notification sent on completion
+
+ # Access the rule ARN
+ print(trainer.notification_rule_arn)
+
+ # List and manage rules
+ rules = trainer.list_notification_rules()
+ trainer.delete_notification_rule(rule_arn=trainer.notification_rule_arn)
+
+ **Prerequisites:**
+
+ - An SNS topic (and subscription) with a resource policy allowing ``events.amazonaws.com``. 
+ To set up a topic and subscription, see `Creating an SNS topic and subscription <https://docs.aws.amazon.com/sns/latest/dg/sns-create-subscribe-endpoint-to-topic.html>`_. 
+ - IAM permissions: ``events:PutRule``, ``events:PutTargets``, ``events:ListRules``,
+ ``events:RemoveTargets``, ``events:DeleteRule``
+
+**Monitoring: show_metrics()**
+ Plot training metrics after a job completes. Works across all compute types.
+
+ - **Nova models**: Metrics parsed from CloudWatch logs.
+ - **OSS models**: Metrics pulled from MLflow.
+
+ .. code-block:: python
+
+ # Plot all available metrics
+ df = trainer.show_metrics()
+
+ # Plot specific metrics
+ df = trainer.show_metrics(metrics=["training_loss", "lr"])
+
+ # Filter by step range
+ df = trainer.show_metrics(starting_step=10, ending_step=100)
+
+ # Filter by time window
+ from datetime import datetime
+ df = trainer.show_metrics(
+ start_time=datetime(2026, 1, 1, 10, 0, 0),
+ end_time=datetime(2026, 1, 1, 12, 0, 0),
+ )
+
+ **After a kernel restart:**
+
+ .. code-block:: python
+
+ # Standalone (SMTJ)
+ from sagemaker.train import plot_training_metrics
+ plot_training_metrics("my-sft-job")
+
+ # Re-attach (HyperPod — needs cluster name for log group resolution)
+ from sagemaker.train import SFTTrainer
+ from sagemaker.core.training.configs import HyperPodCompute
+
+ trainer = SFTTrainer(
+ model="nova-textgeneration-micro",
+ training_dataset="s3://unused",
+ compute=HyperPodCompute(cluster_name="my-cluster", instance_type="ml.p5.48xlarge")
+ )
+ trainer._latest_training_job = "my-hp-job"
+ df = trainer.show_metrics()
+
+**Monitoring: stream_logs()**
+ Stream CloudWatch logs in real-time while a job is running.
+
+ .. code-block:: python
+
+ # Start training non-blocking
+ job = trainer.train(wait=False)
+
+ # Stream logs (blocks until job completes or Ctrl+C)
+ trainer.stream_logs()
+
+ # Custom polling interval (seconds)
+ trainer.stream_logs(poll=10)
+
+ # Stream from a specific start time - providing this will speed up execution.
+ from datetime import datetime
+ trainer.stream_logs(start_time=datetime(2026, 1, 1, 15, 0, 0))
+
+ .. note::
+
+ - **SMTJ**: Streaming auto-stops when the job reaches a terminal state.
+ - **HyperPod**: Streaming runs until you press Ctrl+C. Logs may take a few minutes
+ to propagate to CloudWatch on first run.
+
+
+----
+
 
 .. toctree::
  :maxdepth: 1