Pipeline TypeError: can only concatenate list (not "NoneType") to list Using SourceCode

[CongaJAMM]

PySDK Version

• PySDK V2 (2.x)
• PySDK V3 (3.3.1)

Describe the bug
Forces a parameter for an optional argument in a SourceCode object when using the Pipeline class upsert method.

To reproduce
A clear, step-by-step set of instructions to reproduce the bug.
The provided code need to be complete and runnable, if additional data is needed, please include them in the issue.
Conda environment with this requirements.yaml file content:
name: aws_env
channels:

• pytorch
• nvidia
• conda-forge
• defaults
  dependencies:
• python=3.12
• pip=25.3
• docker=20.10.9
• imbalanced-learn=0.14.0
• ipykernel=7.1.0
• ipywidgets=8.1.8
• numpy=2.4.0
• pandas=2.3.2
• pillow=10.4.0
• pyarrow=21.0.0
• pytorch=2.5.1
• pytorch-cuda=12.4
• s3fs=2025.10.0
• scikit-learn=1.7.2
• seaborn=0.13.2
• pip:
  • sagemaker==3.3.1

Code -
def main():

src = SourceCode(
source_dir=str(
SOURCE_DIR
), # Everything inside the source_dir will be copied to /opt/ml/input/data/code/**
entry_script="weather_pred/training/train.py", # Relative to the source_dir
# command="python train.py epochs=1",
ignore_patterns=[
".env",
".git",
"pycache",
".DS_Store",
".cache",
".ipynb_checkpoints",
"*.egg-info",
],
)
# ADD THIS LINE to satisfy the internal hashing utility
compute = Compute(
instance_type=TRAIN_INSTANCE_TYPE,
instance_count=TRAIN_INSTANCE_COUNT,
volume_size_in_gb=TRAIN_VOLUME_SIZE_GB,
)
my_output_config = OutputDataConfig(
s3_output_path=TRAIN_S3_OUTPUT_PATH, # Configure path to output artifacts
compression_type="GZIP", # Options: 'GZIP' (default) or 'NONE'
# kms_key_id="your-kms-key-arn", # Optional: for encryption at rest
)

trainer = ModelTrainer(
 role=SAGEMAKER_ROLE,
 sagemaker_session=pipeline_session,
 output_data_config=my_output_config,
 base_job_name="local-trainer",
 source_code=src,
 compute=compute,
 training_image=TRAIN_IMAGE_URI,
 training_input_mode="File",
 environment=env_vars,
)

# Receive the TrainingJobRequest object (indicates everything required to run the training job)
train_step_args = trainer.train()

# Look into cache configs later
# my_train_cache_config = CacheConfig(enable_caching=True, expire_after="P1D")

# Define the Training Step
training_step = TrainingStep(
 name="TrainWeatherModelStep",
 step_args=train_step_args,
 display_name="Training-Phase",
 description="Executing the defined modeltrainer wrapper to build and train an Agent",
 cache_config=None,
 depends_on=None,
 retry_policies=None,
)

# Initialize the Pipeline
pipeline = Pipeline(
 name=PIPELINE_NAME,
 # parameters=None,
 # pipeline_experiment_config=None,
 # mlflow_config=None,
 # steps=[training_step, model_step],
 steps=[training_step],
 sagemaker_session=pipeline_session,
 # pipeline_definition_config=None,
)

# This step either creates a new pipeline in SageMaker or updates an existing one with the same name.
pipeline.upsert(role_arn=SAGEMAKER_ROLE) # <<== PROBLEM HERE

# This command kicks off the actual execution of the pipeline in SageMaker. From this point, SageMaker will orchestrate the execution of each step, managing resources and data flow between steps.
execution = pipeline.start()

Expected behavior
Expected to run without any specific requirements and rely on just the container image's dependencies.

Screenshots or logs
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ in _run_module_as_main:198 │
│ in _run_code:88 │
│ │
│ /mnt/c/Users/brianperez/Desktop/DEV/aws/Weather-Pred-AWS/src/weather_pred/orchestrate/ml-pipelin │
│ e-cloud.py:463 in │
│ │
│ 460 │
│ 461 │
│ 462 if name == "main": │
│ ❱ 463 │ main() │
│ 464 │
│ │
│ /mnt/c/Users/brianperez/Desktop/DEV/aws/Weather-Pred-AWS/src/weather_pred/orchestrate/ml-pipelin │
│ e-cloud.py:387 in main │
│ │
│ 384 │ ) │
│ 385 │ │
│ 386 │ # This step either creates a new pipeline in SageMaker or updates an existing one wi │
│ ❱ 387 │ pipeline.upsert(role_arn=SAGEMAKER_ROLE) │
│ 388 │ │
│ 389 │ # This command kicks off the actual execution of the pipeline in SageMaker. From thi │
│ 390 │ execution = pipeline.start() │
│ │
│ /home/congajamm/miniconda3/envs/aws_env/lib/python3.12/site-packages/sagemaker/mlops/workflow/pi │
│ peline.py:333 in upsert │
│ │
│ 330 │ │ │ # after fetching the config. │
│ 331 │ │ │ raise ValueError("An AWS IAM role is required to create or update a Pipeline │
│ 332 │ │ try: │
│ ❱ 333 │ │ │ response = self.create(role_arn, description, tags, parallelism_config) │
│ 334 │ │ except ClientError as ce: │
│ 335 │ │ │ error_code = ce.response["Error"]["Code"] │
│ 336 │ │ │ error_message = ce.response["Error"]["Message"] │
│ │
│ /home/congajamm/miniconda3/envs/aws_env/lib/python3.12/site-packages/sagemaker/core/telemetry/te │
│ lemetry_logging.py:168 in wrapper │
│ │
│ 165 │ │ │ │ │ caught_ex = e │
│ 166 │ │ │ │ finally: │
│ 167 │ │ │ │ │ if caught_ex: │
│ ❱ 168 │ │ │ │ │ │ raise caught_ex │
│ 169 │ │ │ │ │ return response # pylint: disable=W0150 │
│ 170 │ │ │ else: │
│ 171 │ │ │ │ logger.debug( │
│ │
│ /home/congajamm/miniconda3/envs/aws_env/lib/python3.12/site-packages/sagemaker/core/telemetry/te │
│ lemetry_logging.py:139 in wrapper │
│ │
│ 136 │ │ │ │ start_timer = perf_counter() │
│ 137 │ │ │ │ try: │
│ 138 │ │ │ │ │ # Call the original function │
│ ❱ 139 │ │ │ │ │ response = func(*args, **kwargs) │
│ 140 │ │ │ │ │ stop_timer = perf_counter() │
│ 141 │ │ │ │ │ elapsed = stop_timer - start_timer │
│ 142 │ │ │ │ │ extra += f"&x-latency={round(elapsed, 2)}" │
│ │
│ /home/congajamm/miniconda3/envs/aws_env/lib/python3.12/site-packages/sagemaker/mlops/workflow/pi │
│ peline.py:197 in create │
│ │
│ 194 │ │ tags = format_tags(tags) │
│ 195 │ │ tags = _append_project_tags(tags) │
│ 196 │ │ tags = self.sagemaker_session.append_sagemaker_config_tags(tags, PIPELINE_TAGS │
│ ❱ 197 │ │ kwargs = self._create_args(role_arn, description, parallelism_config) │
│ 198 │ │ update_args( │
│ 199 │ │ │ kwargs, │
│ 200 │ │ │ Tags=tags, │
│ │
│ /home/congajamm/miniconda3/envs/aws_env/lib/python3.12/site-packages/sagemaker/mlops/workflow/pi │
│ peline.py:220 in _create_args │
│ │
│ 217 │ │ Returns: │
│ 218 │ │ │ A keyword argument dict for calling create_pipeline. │
│ 219 │ │ """ │
│ ❱ 220 │ │ pipeline_definition = self.definition() │
│ 221 │ │ kwargs = dict( │
│ 222 │ │ │ PipelineName=self.name, │
│ 223 │ │ │ RoleArn=role_arn, │
│ │
│ /home/congajamm/miniconda3/envs/aws_env/lib/python3.12/site-packages/sagemaker/mlops/workflow/pi │
│ peline.py:448 in definition │
│ │
│ 445 │ │ │ sagemaker_session=self.sagemaker_session, │
│ 446 │ │ │ steps=self.steps, │
│ 447 │ │ │ pipeline_definition_config=self.pipeline_definition_config, │
│ ❱ 448 │ │ ).build() │
│ 449 │ │ │
│ 450 │ │ request_dict = { │
│ 451 │ │ │ "Version": self._version, │
│ │
│ /home/congajamm/miniconda3/envs/aws_env/lib/python3.12/site-packages/sagemaker/mlops/workflow/_s │
│ teps_compiler.py:402 in build │
│ │
│ 399 │ │ if self._build_count > 1: │
│ 400 │ │ │ raise RuntimeError("Cannot build a pipeline more than once with the same com │
│ 401 │ │ │
│ ❱ 402 │ │ return self._initialize_queue_and_build(self._input_steps) │
│ 403 │
│ │
│ /home/congajamm/miniconda3/envs/aws_env/lib/python3.12/site-packages/sagemaker/mlops/workflow/_s │
│ teps_compiler.py:386 in _initialize_queue_and_build │
│ │
│ 383 │ │ │ if isinstance(step, ConditionStep): │
│ 384 │ │ │ │ compiled_steps.append(self._build_condition_step(step)) │
│ 385 │ │ │ else: │
│ ❱ 386 │ │ │ │ compiled_steps.append(self._build_step(step)) │
│ 387 │ │ │
│ 388 │ │ self._set_serialize_output_to_json_flag(compiled_steps) │
│ 389 │ │ return compiled_steps │
│ │
│ /home/congajamm/miniconda3/envs/aws_env/lib/python3.12/site-packages/sagemaker/mlops/workflow/_s │
│ teps_compiler.py:319 in _build_step │
│ │
│ 316 │ │ │ pipeline_name=self.pipeline_name, │
│ 317 │ │ │ step_name=step.name, │
│ 318 │ │ │ sagemaker_session=self.sagemaker_session, │
│ ❱ 319 │ │ │ code_hash=get_code_hash(step), │
│ 320 │ │ │ config_hash=get_config_hash(step), │
│ 321 │ │ │ pipeline_definition_config=self.pipeline_definition_config, │
│ 322 │ │ │ upload_runtime_scripts=self.upload_runtime_scripts, │
│ │
│ /home/congajamm/miniconda3/envs/aws_env/lib/python3.12/site-packages/sagemaker/core/workflow/uti │
│ lities.py:177 in get_code_hash │
│ │
│ 174 │ │ │ source_dir = source_code.source_dir │
│ 175 │ │ │ requirements = source_code.requirements │
│ 176 │ │ │ entry_point = source_code.entry_script │
│ ❱ 177 │ │ │ return get_training_code_hash(entry_point, source_dir, requirements) │
│ 178 │ return None │
│ 179 │
│ 180 │
│ │
│ /home/congajamm/miniconda3/envs/aws_env/lib/python3.12/site-packages/sagemaker/core/workflow/uti │
│ lities.py:251 in get_training_code_hash │
│ │
│ 248 │ │ if source_dir: │
│ 249 │ │ │ source_dir_url = urlparse(source_dir) │
│ 250 │ │ │ if source_dir_url.scheme == "" or source_dir_url.scheme == "file": │
│ ❱ 251 │ │ │ │ return hash_files_or_dirs([source_dir] + dependencies) │
│ 252 │ │ elif entry_point: │
│ 253 │ │ │ entry_point_url = urlparse(entry_point) │
│ 254 │ │ │ if entry_point_url.scheme == "" or entry_point_url.scheme == "file": │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
TypeError: can only concatenate list (not "NoneType") to list
(aws_env) congajamm@CONNB441:/mnt/c/Users/brianperez/Desktop/DEV/aws/Weather-Pred-AWS$

System information
A description of your system. Please provide:

• SageMaker Python SDK version:3.3.1
• Framework name (eg. PyTorch) or algorithm (eg. KMeans):PyTorch
• Framework version: Torchserve version: 0.12.0
• Python version:3.12.12
• CPU or GPU:CPU
• Custom Docker image (Y/N):N

Additional context
Running on Windows 11 with Debian WSL 2 Linux.
(aws_env) congajamm@CONNB441:/mnt/c/Users/brianperez/Desktop/DEV/aws/Weather-Pred-AWS$ python -m src.weather_pred.orchestrate.ml-pipeline-cloud

[ravees]

Facing the same issue when passing source code with requirements.txt. The same error is seen even when the requirements is not explicitly set with requirements.txt file is present in the source location.
source_code=SourceCode(
source_dir='pipeline_code/scripts',
entry_script='train.py',
requirements="requirements.txt"
)

[dgallitelli]

Workaround

The bug is in sagemaker/core/workflow/utilities.py at line 177 where get_code_hash() passes requirements (a str or None - the path to requirements.txt) to get_training_code_hash() which expects dependencies: List[str].

Monkey-patch fix

Add this at the top of your pipeline script before importing other sagemaker modules:

import sagemaker.core.workflow.utilities as _workflow_utils

_original_get_training_code_hash = _workflow_utils.get_training_code_hash

def _patched_get_training_code_hash(entry_point, source_dir, dependencies):
 """Patched version that handles string/None dependencies."""
 if dependencies is None:
 dependencies = []
 elif isinstance(dependencies, str):
 dependencies = [dependencies] if dependencies else []
 return _original_get_training_code_hash(entry_point, source_dir, dependencies)

_workflow_utils.get_training_code_hash = _patched_get_training_code_hash

Suggested fix in SDK

In sagemaker/core/workflow/utilities.py, line 177 should convert requirements to a list before passing:

# Current (broken):
return get_training_code_hash(entry_point, source_dir, requirements)

# Fixed:
deps = [requirements] if requirements else []
return get_training_code_hash(entry_point, source_dir, deps)

Tested with SDK v3.4.1 on Python 3.14.

[doublecompile]

This PR seems to address this bug: #5618

[doublecompile]

Now that the PR is merged and 3.6.0 was released, this bug no longer appears. I think we can consider this issue resolved.