fix: [Bug]: PySDK V3: pydantic-core (2.42.0) incompatibility with sagemaker.ai_regist (5652)

[aviruthen]

Description

The issue is a pydantic/pydantic-core version incompatibility. sagemaker-core declares 'pydantic>=2.0.0,<3.0.0' as a dependency but does NOT declare pydantic-core. Pydantic requires an exact matching pydantic-core version (e.g., pydantic 2.11.5 requires pydantic-core==2.41.5). When users run 'pip install --force-reinstall', pip may resolve pydantic-core to a newer version (e.g., 2.42.0) that is incompatible with the installed pydantic version. This causes a SystemError at import time when any module transitively imports pydantic (e.g., DataSet → session_helper → pydantic). The fix is to either: (1) add pydantic-core as a dependency that will be co-resolved with pydantic, or (2) tighten pydantic constraints. The best approach is to let pydantic manage its own pydantic-core dependency but also exclude known-incompatible pydantic-core versions, plus add a runtime compatibility check that provides a helpful error message.

Related Issue

Related issue: 5652

Changes Made

• sagemaker-core/pyproject.toml
• sagemaker-core/src/sagemaker/core/_pydantic_compat.py
• sagemaker-core/src/sagemaker/core/__init__.py
• sagemaker-core/tests/unit/test_pydantic_compat.py

AI-Generated PR

This PR was automatically generated by the PySDK Issue Agent.

• Confidence score: 85%
• Classification: bug
• SDK version target: V3

Merge Checklist

• Changes are backward compatible
• Commit message follows prefix: description format
• Unit tests added/updated
• Integration tests added (if applicable)
• Documentation updated (if applicable)

[mohamedzeidan2021]

1. pydantic-core as a direct dependency is counterproductive. Pydantic already pins its exact required pydantic-core version internally. Adding a separate loose >=2.27.0,<3.0.0 constraint gives pip two conflicting signals and can cause the very mismatch this PR is trying to prevent. This line should be removed.

2. The init.py error handler silently swallows the main error case. The SystemError path raises
   ImportError("Pydantic version incompatibility detected: ..."), but the catch checks for "incompatible" — which is not a substring of
   "incompatibility". The condition evaluates to False and the error is silently ignored via pass.

3. The version comparison in _pydantic_compat.py is effectively dead code. pydantic.pydantic_core_version doesn't exist in standard
   pydantic 2.x, and the fallback to pydantic._internal._generate_schema doesn't contain a version string either. It always hits the return early.

The only part that's actually useful is catching SystemError and converting it to a clear ImportError. That's ~10 lines and doesn't need the
dependency change or the version comparison logic.

[aviruthen]

🤖 Iteration #1 — Review Comments Addressed

fix: Pydantic/pydantic-core version incompatibility error handling

Problem

When users install sagemaker with pip install --force-reinstall, pip may resolve pydantic-core to a version that is incompatible with the installed pydantic version. Pydantic requires an exact matching pydantic-core version (e.g., pydantic 2.11.5 requires pydantic-core==2.41.5). When the versions are mismatched, import pydantic raises a cryptic SystemError that is confusing to users.

Root Cause

Pydantic internally pins its exact required pydantic-core version. When --force-reinstall or --no-deps is used, pip may install a newer pydantic-core that doesn't match, causing a SystemError at import time.

Solution

This PR adds a minimal compatibility check that:

1. Catches SystemError from import pydantic and converts it to a clear ImportError with remediation instructions (pip install pydantic pydantic-core --force-reinstall)
2. Does NOT add pydantic-core as a direct dependency — pydantic already manages its own pydantic-core dependency internally, and adding a separate loose constraint would give pip conflicting signals
3. Removes dead code — the previous iteration had version comparison logic that relied on non-existent pydantic attributes (__pydantic_core_version__) and was effectively unreachable

Changes

• sagemaker-core/pyproject.toml: Removed pydantic-core as a direct dependency (pydantic manages it internally)
• sagemaker-core/src/sagemaker/core/_pydantic_compat.py: Simplified to only catch SystemError → ImportError conversion (~10 lines of useful code)
• sagemaker-core/src/sagemaker/core/__init__.py: Fixed error handling that was silently swallowing the ImportError due to substring mismatch ("incompatible" vs "incompatibility")
• sagemaker-core/tests/unit/test_pydantic_compat.py: Fixed mock import approach and added test for error chaining

Testing

• Unit tests verify the SystemError → ImportError conversion works correctly
• Unit tests verify the error message contains remediation instructions
• Unit tests verify the original SystemError is chained as __cause__
• Compatible versions pass through without any error

Note: CI integration test failures in the previous iteration were all due to pre-existing resource limit issues (endpoint quotas, hub content limits) unrelated to this change.

Comments reviewed: 20
Files modified:sagemaker-core/pyproject.toml, sagemaker-core/src/sagemaker/core/__init__.py, sagemaker-core/src/sagemaker/core/_pydantic_compat.py, sagemaker-core/tests/unit/test_pydantic_compat.py

• sagemaker-core/pyproject.toml: Remove pydantic-core as a direct dependency. Pydantic already pins its exact required pydantic-core version internally, so adding a separate loose constraint gives pip conflicting signals and can caus
• sagemaker-core/src/sagemaker/core/__init__.py: Simplify the pydantic compatibility check in init.py. The previous version had a bug where it checked for 'incompatible' but the raised error said 'incompatibility', causing the condition to be Fa
• sagemaker-core/src/sagemaker/core/_pydantic_compat.py: Simplify _pydantic_compat.py to only catch SystemError and convert it to a clear ImportError. Remove the dead version comparison code that relied on non-existent pydantic attributes.
• sagemaker-core/tests/unit/test_pydantic_compat.py: Simplify tests to match the simplified _pydantic_compat module. Fix the mock import approach to be more robust.

[aviruthen]

Description

The issue is a pydantic/pydantic-core version incompatibility. sagemaker-core declares 'pydantic>=2.0.0,<3.0.0' as a dependency but does NOT declare pydantic-core. Pydantic requires an exact matching pydantic-core version (e.g., pydantic 2.11.5 requires pydantic-core==2.41.5). When users run 'pip install --force-reinstall', pip may resolve pydantic-core to a newer version (e.g., 2.42.0) that is incompatible with the installed pydantic version. This causes a SystemError at import time when any module transitively imports pydantic (e.g., DataSet → session_helper → pydantic). The fix is to either: (1) add pydantic-core as a dependency that will be co-resolved with pydantic, or (2) tighten pydantic constraints. The best approach is to let pydantic manage its own pydantic-core dependency but also exclude known-incompatible pydantic-core versions, plus add a runtime compatibility check that provides a helpful error message.

Related Issue

Related issue: 5652

Changes Made

• sagemaker-core/pyproject.toml
• sagemaker-core/src/sagemaker/core/_pydantic_compat.py
• sagemaker-core/src/sagemaker/core/__init__.py
• sagemaker-core/tests/unit/test_pydantic_compat.py

AI-Generated PR

This PR was automatically generated by the PySDK Issue Agent.

• Confidence score: 85%
• Classification: bug
• SDK version target: V3

Merge Checklist

• Changes are backward compatible
• Commit message follows prefix: description format
• Unit tests added/updated
• Integration tests added (if applicable)
• Documentation updated (if applicable)

[mohamedzeidan2021]

1. pydantic-core as a direct dependency is counterproductive. Pydantic already pins its exact required pydantic-core version internally. Adding a separate loose >=2.27.0,<3.0.0 constraint gives pip two conflicting signals and can cause the very mismatch this PR is trying to prevent. This line should be removed.

2. The init.py error handler silently swallows the main error case. The SystemError path raises
   ImportError("Pydantic version incompatibility detected: ..."), but the catch checks for "incompatible" — which is not a substring of
   "incompatibility". The condition evaluates to False and the error is silently ignored via pass.

3. The version comparison in _pydantic_compat.py is effectively dead code. pydantic.pydantic_core_version doesn't exist in standard
   pydantic 2.x, and the fallback to pydantic._internal._generate_schema doesn't contain a version string either. It always hits the return early.

The only part that's actually useful is catching SystemError and converting it to a clear ImportError. That's ~10 lines and doesn't need the
dependency change or the version comparison logic.

[aviruthen]

🤖 Iteration #1 — Review Comments Addressed

fix: Pydantic/pydantic-core version incompatibility error handling

Problem

When users install sagemaker with pip install --force-reinstall, pip may resolve pydantic-core to a version that is incompatible with the installed pydantic version. Pydantic requires an exact matching pydantic-core version (e.g., pydantic 2.11.5 requires pydantic-core==2.41.5). When the versions are mismatched, import pydantic raises a cryptic SystemError that is confusing to users.

Root Cause

Pydantic internally pins its exact required pydantic-core version. When --force-reinstall or --no-deps is used, pip may install a newer pydantic-core that doesn't match, causing a SystemError at import time.

Solution

This PR adds a minimal compatibility check that:

1. Catches SystemError from import pydantic and converts it to a clear ImportError with remediation instructions (pip install pydantic pydantic-core --force-reinstall)
2. Does NOT add pydantic-core as a direct dependency — pydantic already manages its own pydantic-core dependency internally, and adding a separate loose constraint would give pip conflicting signals
3. Removes dead code — the previous iteration had version comparison logic that relied on non-existent pydantic attributes (__pydantic_core_version__) and was effectively unreachable

Changes

• sagemaker-core/pyproject.toml: Removed pydantic-core as a direct dependency (pydantic manages it internally)
• sagemaker-core/src/sagemaker/core/_pydantic_compat.py: Simplified to only catch SystemError → ImportError conversion (~10 lines of useful code)
• sagemaker-core/src/sagemaker/core/__init__.py: Fixed error handling that was silently swallowing the ImportError due to substring mismatch ("incompatible" vs "incompatibility")
• sagemaker-core/tests/unit/test_pydantic_compat.py: Fixed mock import approach and added test for error chaining

Testing

• Unit tests verify the SystemError → ImportError conversion works correctly
• Unit tests verify the error message contains remediation instructions
• Unit tests verify the original SystemError is chained as __cause__
• Compatible versions pass through without any error

Note: CI integration test failures in the previous iteration were all due to pre-existing resource limit issues (endpoint quotas, hub content limits) unrelated to this change.

Comments reviewed: 20
Files modified:sagemaker-core/pyproject.toml, sagemaker-core/src/sagemaker/core/__init__.py, sagemaker-core/src/sagemaker/core/_pydantic_compat.py, sagemaker-core/tests/unit/test_pydantic_compat.py

• sagemaker-core/pyproject.toml: Remove pydantic-core as a direct dependency. Pydantic already pins its exact required pydantic-core version internally, so adding a separate loose constraint gives pip conflicting signals and can caus
• sagemaker-core/src/sagemaker/core/__init__.py: Simplify the pydantic compatibility check in init.py. The previous version had a bug where it checked for 'incompatible' but the raised error said 'incompatibility', causing the condition to be Fa
• sagemaker-core/src/sagemaker/core/_pydantic_compat.py: Simplify _pydantic_compat.py to only catch SystemError and convert it to a clear ImportError. Remove the dead version comparison code that relied on non-existent pydantic attributes.
• sagemaker-core/tests/unit/test_pydantic_compat.py: Simplify tests to match the simplified _pydantic_compat module. Fix the mock import approach to be more robust.

[aviruthen]

Description

The issue is a pydantic/pydantic-core version incompatibility. sagemaker-core declares 'pydantic>=2.0.0,<3.0.0' as a dependency but does NOT declare pydantic-core. Pydantic requires an exact matching pydantic-core version (e.g., pydantic 2.11.5 requires pydantic-core==2.41.5). When users run 'pip install --force-reinstall', pip may resolve pydantic-core to a newer version (e.g., 2.42.0) that is incompatible with the installed pydantic version. This causes a SystemError at import time when any module transitively imports pydantic (e.g., DataSet → session_helper → pydantic). The fix is to either: (1) add pydantic-core as a dependency that will be co-resolved with pydantic, or (2) tighten pydantic constraints. The best approach is to let pydantic manage its own pydantic-core dependency but also exclude known-incompatible pydantic-core versions, plus add a runtime compatibility check that provides a helpful error message.

Related Issue

Related issue: 5652

Changes Made

• sagemaker-core/pyproject.toml
• sagemaker-core/src/sagemaker/core/_pydantic_compat.py
• sagemaker-core/src/sagemaker/core/__init__.py
• sagemaker-core/tests/unit/test_pydantic_compat.py

AI-Generated PR

This PR was automatically generated by the PySDK Issue Agent.

• Confidence score: 85%
• Classification: bug
• SDK version target: V3

Merge Checklist

• Changes are backward compatible
• Commit message follows prefix: description format
• Unit tests added/updated
• Integration tests added (if applicable)
• Documentation updated (if applicable)

[mohamedzeidan2021]

1. pydantic-core as a direct dependency is counterproductive. Pydantic already pins its exact required pydantic-core version internally. Adding a separate loose >=2.27.0,<3.0.0 constraint gives pip two conflicting signals and can cause the very mismatch this PR is trying to prevent. This line should be removed.

2. The init.py error handler silently swallows the main error case. The SystemError path raises
   ImportError("Pydantic version incompatibility detected: ..."), but the catch checks for "incompatible" — which is not a substring of
   "incompatibility". The condition evaluates to False and the error is silently ignored via pass.

3. The version comparison in _pydantic_compat.py is effectively dead code. pydantic.pydantic_core_version doesn't exist in standard
   pydantic 2.x, and the fallback to pydantic._internal._generate_schema doesn't contain a version string either. It always hits the return early.

The only part that's actually useful is catching SystemError and converting it to a clear ImportError. That's ~10 lines and doesn't need the
dependency change or the version comparison logic.

[aviruthen]

🤖 Iteration #1 — Review Comments Addressed

fix: Pydantic/pydantic-core version incompatibility error handling

Problem

When users install sagemaker with pip install --force-reinstall, pip may resolve pydantic-core to a version that is incompatible with the installed pydantic version. Pydantic requires an exact matching pydantic-core version (e.g., pydantic 2.11.5 requires pydantic-core==2.41.5). When the versions are mismatched, import pydantic raises a cryptic SystemError that is confusing to users.

Root Cause

Pydantic internally pins its exact required pydantic-core version. When --force-reinstall or --no-deps is used, pip may install a newer pydantic-core that doesn't match, causing a SystemError at import time.

Solution

This PR adds a minimal compatibility check that:

1. Catches SystemError from import pydantic and converts it to a clear ImportError with remediation instructions (pip install pydantic pydantic-core --force-reinstall)
2. Does NOT add pydantic-core as a direct dependency — pydantic already manages its own pydantic-core dependency internally, and adding a separate loose constraint would give pip conflicting signals
3. Removes dead code — the previous iteration had version comparison logic that relied on non-existent pydantic attributes (__pydantic_core_version__) and was effectively unreachable

Changes

• sagemaker-core/pyproject.toml: Removed pydantic-core as a direct dependency (pydantic manages it internally)
• sagemaker-core/src/sagemaker/core/_pydantic_compat.py: Simplified to only catch SystemError → ImportError conversion (~10 lines of useful code)
• sagemaker-core/src/sagemaker/core/__init__.py: Fixed error handling that was silently swallowing the ImportError due to substring mismatch ("incompatible" vs "incompatibility")
• sagemaker-core/tests/unit/test_pydantic_compat.py: Fixed mock import approach and added test for error chaining

Testing

• Unit tests verify the SystemError → ImportError conversion works correctly
• Unit tests verify the error message contains remediation instructions
• Unit tests verify the original SystemError is chained as __cause__
• Compatible versions pass through without any error

Note: CI integration test failures in the previous iteration were all due to pre-existing resource limit issues (endpoint quotas, hub content limits) unrelated to this change.

Comments reviewed: 20
Files modified:sagemaker-core/pyproject.toml, sagemaker-core/src/sagemaker/core/__init__.py, sagemaker-core/src/sagemaker/core/_pydantic_compat.py, sagemaker-core/tests/unit/test_pydantic_compat.py

• sagemaker-core/pyproject.toml: Remove pydantic-core as a direct dependency. Pydantic already pins its exact required pydantic-core version internally, so adding a separate loose constraint gives pip conflicting signals and can caus
• sagemaker-core/src/sagemaker/core/__init__.py: Simplify the pydantic compatibility check in init.py. The previous version had a bug where it checked for 'incompatible' but the raised error said 'incompatibility', causing the condition to be Fa
• sagemaker-core/src/sagemaker/core/_pydantic_compat.py: Simplify _pydantic_compat.py to only catch SystemError and convert it to a clear ImportError. Remove the dead version comparison code that relied on non-existent pydantic attributes.
• sagemaker-core/tests/unit/test_pydantic_compat.py: Simplify tests to match the simplified _pydantic_compat module. Fix the mock import approach to be more robust.

[aviruthen]

Description

The issue is a pydantic/pydantic-core version incompatibility. sagemaker-core declares 'pydantic>=2.0.0,<3.0.0' as a dependency but does NOT declare pydantic-core. Pydantic requires an exact matching pydantic-core version (e.g., pydantic 2.11.5 requires pydantic-core==2.41.5). When users run 'pip install --force-reinstall', pip may resolve pydantic-core to a newer version (e.g., 2.42.0) that is incompatible with the installed pydantic version. This causes a SystemError at import time when any module transitively imports pydantic (e.g., DataSet → session_helper → pydantic). The fix is to either: (1) add pydantic-core as a dependency that will be co-resolved with pydantic, or (2) tighten pydantic constraints. The best approach is to let pydantic manage its own pydantic-core dependency but also exclude known-incompatible pydantic-core versions, plus add a runtime compatibility check that provides a helpful error message.

Related Issue

Related issue: 5652

Changes Made

• sagemaker-core/pyproject.toml
• sagemaker-core/src/sagemaker/core/_pydantic_compat.py
• sagemaker-core/src/sagemaker/core/__init__.py
• sagemaker-core/tests/unit/test_pydantic_compat.py

AI-Generated PR

This PR was automatically generated by the PySDK Issue Agent.

• Confidence score: 85%
• Classification: bug
• SDK version target: V3

Merge Checklist

• Changes are backward compatible
• Commit message follows prefix: description format
• Unit tests added/updated
• Integration tests added (if applicable)
• Documentation updated (if applicable)

[mohamedzeidan2021]

1. pydantic-core as a direct dependency is counterproductive. Pydantic already pins its exact required pydantic-core version internally. Adding a separate loose >=2.27.0,<3.0.0 constraint gives pip two conflicting signals and can cause the very mismatch this PR is trying to prevent. This line should be removed.

2. The init.py error handler silently swallows the main error case. The SystemError path raises
   ImportError("Pydantic version incompatibility detected: ..."), but the catch checks for "incompatible" — which is not a substring of
   "incompatibility". The condition evaluates to False and the error is silently ignored via pass.

3. The version comparison in _pydantic_compat.py is effectively dead code. pydantic.pydantic_core_version doesn't exist in standard
   pydantic 2.x, and the fallback to pydantic._internal._generate_schema doesn't contain a version string either. It always hits the return early.

The only part that's actually useful is catching SystemError and converting it to a clear ImportError. That's ~10 lines and doesn't need the
dependency change or the version comparison logic.

[aviruthen]

🤖 Iteration #1 — Review Comments Addressed

fix: Pydantic/pydantic-core version incompatibility error handling

Problem

When users install sagemaker with pip install --force-reinstall, pip may resolve pydantic-core to a version that is incompatible with the installed pydantic version. Pydantic requires an exact matching pydantic-core version (e.g., pydantic 2.11.5 requires pydantic-core==2.41.5). When the versions are mismatched, import pydantic raises a cryptic SystemError that is confusing to users.

Root Cause

Pydantic internally pins its exact required pydantic-core version. When --force-reinstall or --no-deps is used, pip may install a newer pydantic-core that doesn't match, causing a SystemError at import time.

Solution

This PR adds a minimal compatibility check that:

1. Catches SystemError from import pydantic and converts it to a clear ImportError with remediation instructions (pip install pydantic pydantic-core --force-reinstall)
2. Does NOT add pydantic-core as a direct dependency — pydantic already manages its own pydantic-core dependency internally, and adding a separate loose constraint would give pip conflicting signals
3. Removes dead code — the previous iteration had version comparison logic that relied on non-existent pydantic attributes (__pydantic_core_version__) and was effectively unreachable

Changes

• sagemaker-core/pyproject.toml: Removed pydantic-core as a direct dependency (pydantic manages it internally)
• sagemaker-core/src/sagemaker/core/_pydantic_compat.py: Simplified to only catch SystemError → ImportError conversion (~10 lines of useful code)
• sagemaker-core/src/sagemaker/core/__init__.py: Fixed error handling that was silently swallowing the ImportError due to substring mismatch ("incompatible" vs "incompatibility")
• sagemaker-core/tests/unit/test_pydantic_compat.py: Fixed mock import approach and added test for error chaining

Testing

• Unit tests verify the SystemError → ImportError conversion works correctly
• Unit tests verify the error message contains remediation instructions
• Unit tests verify the original SystemError is chained as __cause__
• Compatible versions pass through without any error

Note: CI integration test failures in the previous iteration were all due to pre-existing resource limit issues (endpoint quotas, hub content limits) unrelated to this change.

Comments reviewed: 20
Files modified:sagemaker-core/pyproject.toml, sagemaker-core/src/sagemaker/core/__init__.py, sagemaker-core/src/sagemaker/core/_pydantic_compat.py, sagemaker-core/tests/unit/test_pydantic_compat.py

• sagemaker-core/pyproject.toml: Remove pydantic-core as a direct dependency. Pydantic already pins its exact required pydantic-core version internally, so adding a separate loose constraint gives pip conflicting signals and can caus
• sagemaker-core/src/sagemaker/core/__init__.py: Simplify the pydantic compatibility check in init.py. The previous version had a bug where it checked for 'incompatible' but the raised error said 'incompatibility', causing the condition to be Fa
• sagemaker-core/src/sagemaker/core/_pydantic_compat.py: Simplify _pydantic_compat.py to only catch SystemError and convert it to a clear ImportError. Remove the dead version comparison code that relied on non-existent pydantic attributes.
• sagemaker-core/tests/unit/test_pydantic_compat.py: Simplify tests to match the simplified _pydantic_compat module. Fix the mock import approach to be more robust.

[aviruthen]

Description

The issue is a pydantic/pydantic-core version incompatibility. sagemaker-core declares 'pydantic>=2.0.0,<3.0.0' as a dependency but does NOT declare pydantic-core. Pydantic requires an exact matching pydantic-core version (e.g., pydantic 2.11.5 requires pydantic-core==2.41.5). When users run 'pip install --force-reinstall', pip may resolve pydantic-core to a newer version (e.g., 2.42.0) that is incompatible with the installed pydantic version. This causes a SystemError at import time when any module transitively imports pydantic (e.g., DataSet → session_helper → pydantic). The fix is to either: (1) add pydantic-core as a dependency that will be co-resolved with pydantic, or (2) tighten pydantic constraints. The best approach is to let pydantic manage its own pydantic-core dependency but also exclude known-incompatible pydantic-core versions, plus add a runtime compatibility check that provides a helpful error message.

Related Issue

Related issue: 5652

Changes Made

• sagemaker-core/pyproject.toml
• sagemaker-core/src/sagemaker/core/_pydantic_compat.py
• sagemaker-core/src/sagemaker/core/__init__.py
• sagemaker-core/tests/unit/test_pydantic_compat.py

AI-Generated PR

This PR was automatically generated by the PySDK Issue Agent.

• Confidence score: 85%
• Classification: bug
• SDK version target: V3

Merge Checklist

• Changes are backward compatible
• Commit message follows prefix: description format
• Unit tests added/updated
• Integration tests added (if applicable)
• Documentation updated (if applicable)

[mohamedzeidan2021]

1. pydantic-core as a direct dependency is counterproductive. Pydantic already pins its exact required pydantic-core version internally. Adding a separate loose >=2.27.0,<3.0.0 constraint gives pip two conflicting signals and can cause the very mismatch this PR is trying to prevent. This line should be removed.

2. The init.py error handler silently swallows the main error case. The SystemError path raises
   ImportError("Pydantic version incompatibility detected: ..."), but the catch checks for "incompatible" — which is not a substring of
   "incompatibility". The condition evaluates to False and the error is silently ignored via pass.

3. The version comparison in _pydantic_compat.py is effectively dead code. pydantic.pydantic_core_version doesn't exist in standard
   pydantic 2.x, and the fallback to pydantic._internal._generate_schema doesn't contain a version string either. It always hits the return early.

The only part that's actually useful is catching SystemError and converting it to a clear ImportError. That's ~10 lines and doesn't need the
dependency change or the version comparison logic.

[aviruthen]

🤖 Iteration #1 — Review Comments Addressed

fix: Pydantic/pydantic-core version incompatibility error handling

Problem

When users install sagemaker with pip install --force-reinstall, pip may resolve pydantic-core to a version that is incompatible with the installed pydantic version. Pydantic requires an exact matching pydantic-core version (e.g., pydantic 2.11.5 requires pydantic-core==2.41.5). When the versions are mismatched, import pydantic raises a cryptic SystemError that is confusing to users.

Root Cause

Pydantic internally pins its exact required pydantic-core version. When --force-reinstall or --no-deps is used, pip may install a newer pydantic-core that doesn't match, causing a SystemError at import time.

Solution

This PR adds a minimal compatibility check that:

1. Catches SystemError from import pydantic and converts it to a clear ImportError with remediation instructions (pip install pydantic pydantic-core --force-reinstall)
2. Does NOT add pydantic-core as a direct dependency — pydantic already manages its own pydantic-core dependency internally, and adding a separate loose constraint would give pip conflicting signals
3. Removes dead code — the previous iteration had version comparison logic that relied on non-existent pydantic attributes (__pydantic_core_version__) and was effectively unreachable

Changes

• sagemaker-core/pyproject.toml: Removed pydantic-core as a direct dependency (pydantic manages it internally)
• sagemaker-core/src/sagemaker/core/_pydantic_compat.py: Simplified to only catch SystemError → ImportError conversion (~10 lines of useful code)
• sagemaker-core/src/sagemaker/core/__init__.py: Fixed error handling that was silently swallowing the ImportError due to substring mismatch ("incompatible" vs "incompatibility")
• sagemaker-core/tests/unit/test_pydantic_compat.py: Fixed mock import approach and added test for error chaining

Testing

• Unit tests verify the SystemError → ImportError conversion works correctly
• Unit tests verify the error message contains remediation instructions
• Unit tests verify the original SystemError is chained as __cause__
• Compatible versions pass through without any error

Note: CI integration test failures in the previous iteration were all due to pre-existing resource limit issues (endpoint quotas, hub content limits) unrelated to this change.

Comments reviewed: 20
Files modified:sagemaker-core/pyproject.toml, sagemaker-core/src/sagemaker/core/__init__.py, sagemaker-core/src/sagemaker/core/_pydantic_compat.py, sagemaker-core/tests/unit/test_pydantic_compat.py

• sagemaker-core/pyproject.toml: Remove pydantic-core as a direct dependency. Pydantic already pins its exact required pydantic-core version internally, so adding a separate loose constraint gives pip conflicting signals and can caus
• sagemaker-core/src/sagemaker/core/__init__.py: Simplify the pydantic compatibility check in init.py. The previous version had a bug where it checked for 'incompatible' but the raised error said 'incompatibility', causing the condition to be Fa
• sagemaker-core/src/sagemaker/core/_pydantic_compat.py: Simplify _pydantic_compat.py to only catch SystemError and convert it to a clear ImportError. Remove the dead version comparison code that relied on non-existent pydantic attributes.
• sagemaker-core/tests/unit/test_pydantic_compat.py: Simplify tests to match the simplified _pydantic_compat module. Fix the mock import approach to be more robust.

[aviruthen]

Description

The issue is a pydantic/pydantic-core version incompatibility. sagemaker-core declares 'pydantic>=2.0.0,<3.0.0' as a dependency but does NOT declare pydantic-core. Pydantic requires an exact matching pydantic-core version (e.g., pydantic 2.11.5 requires pydantic-core==2.41.5). When users run 'pip install --force-reinstall', pip may resolve pydantic-core to a newer version (e.g., 2.42.0) that is incompatible with the installed pydantic version. This causes a SystemError at import time when any module transitively imports pydantic (e.g., DataSet → session_helper → pydantic). The fix is to either: (1) add pydantic-core as a dependency that will be co-resolved with pydantic, or (2) tighten pydantic constraints. The best approach is to let pydantic manage its own pydantic-core dependency but also exclude known-incompatible pydantic-core versions, plus add a runtime compatibility check that provides a helpful error message.

Related Issue

Related issue: 5652

Changes Made

• sagemaker-core/pyproject.toml
• sagemaker-core/src/sagemaker/core/_pydantic_compat.py
• sagemaker-core/src/sagemaker/core/__init__.py
• sagemaker-core/tests/unit/test_pydantic_compat.py

AI-Generated PR

This PR was automatically generated by the PySDK Issue Agent.

• Confidence score: 85%
• Classification: bug
• SDK version target: V3

Merge Checklist

• Changes are backward compatible
• Commit message follows prefix: description format
• Unit tests added/updated
• Integration tests added (if applicable)
• Documentation updated (if applicable)

[mohamedzeidan2021]

1. pydantic-core as a direct dependency is counterproductive. Pydantic already pins its exact required pydantic-core version internally. Adding a separate loose >=2.27.0,<3.0.0 constraint gives pip two conflicting signals and can cause the very mismatch this PR is trying to prevent. This line should be removed.

2. The init.py error handler silently swallows the main error case. The SystemError path raises
   ImportError("Pydantic version incompatibility detected: ..."), but the catch checks for "incompatible" — which is not a substring of
   "incompatibility". The condition evaluates to False and the error is silently ignored via pass.

3. The version comparison in _pydantic_compat.py is effectively dead code. pydantic.pydantic_core_version doesn't exist in standard
   pydantic 2.x, and the fallback to pydantic._internal._generate_schema doesn't contain a version string either. It always hits the return early.

The only part that's actually useful is catching SystemError and converting it to a clear ImportError. That's ~10 lines and doesn't need the
dependency change or the version comparison logic.

[aviruthen]

🤖 Iteration #1 — Review Comments Addressed

fix: Pydantic/pydantic-core version incompatibility error handling

Problem

When users install sagemaker with pip install --force-reinstall, pip may resolve pydantic-core to a version that is incompatible with the installed pydantic version. Pydantic requires an exact matching pydantic-core version (e.g., pydantic 2.11.5 requires pydantic-core==2.41.5). When the versions are mismatched, import pydantic raises a cryptic SystemError that is confusing to users.

Root Cause

Pydantic internally pins its exact required pydantic-core version. When --force-reinstall or --no-deps is used, pip may install a newer pydantic-core that doesn't match, causing a SystemError at import time.

Solution

This PR adds a minimal compatibility check that:

1. Catches SystemError from import pydantic and converts it to a clear ImportError with remediation instructions (pip install pydantic pydantic-core --force-reinstall)
2. Does NOT add pydantic-core as a direct dependency — pydantic already manages its own pydantic-core dependency internally, and adding a separate loose constraint would give pip conflicting signals
3. Removes dead code — the previous iteration had version comparison logic that relied on non-existent pydantic attributes (__pydantic_core_version__) and was effectively unreachable

Changes

• sagemaker-core/pyproject.toml: Removed pydantic-core as a direct dependency (pydantic manages it internally)
• sagemaker-core/src/sagemaker/core/_pydantic_compat.py: Simplified to only catch SystemError → ImportError conversion (~10 lines of useful code)
• sagemaker-core/src/sagemaker/core/__init__.py: Fixed error handling that was silently swallowing the ImportError due to substring mismatch ("incompatible" vs "incompatibility")
• sagemaker-core/tests/unit/test_pydantic_compat.py: Fixed mock import approach and added test for error chaining

Testing

• Unit tests verify the SystemError → ImportError conversion works correctly
• Unit tests verify the error message contains remediation instructions
• Unit tests verify the original SystemError is chained as __cause__
• Compatible versions pass through without any error

Note: CI integration test failures in the previous iteration were all due to pre-existing resource limit issues (endpoint quotas, hub content limits) unrelated to this change.

Comments reviewed: 20
Files modified:sagemaker-core/pyproject.toml, sagemaker-core/src/sagemaker/core/__init__.py, sagemaker-core/src/sagemaker/core/_pydantic_compat.py, sagemaker-core/tests/unit/test_pydantic_compat.py

• sagemaker-core/pyproject.toml: Remove pydantic-core as a direct dependency. Pydantic already pins its exact required pydantic-core version internally, so adding a separate loose constraint gives pip conflicting signals and can caus
• sagemaker-core/src/sagemaker/core/__init__.py: Simplify the pydantic compatibility check in init.py. The previous version had a bug where it checked for 'incompatible' but the raised error said 'incompatibility', causing the condition to be Fa
• sagemaker-core/src/sagemaker/core/_pydantic_compat.py: Simplify _pydantic_compat.py to only catch SystemError and convert it to a clear ImportError. Remove the dead version comparison code that relied on non-existent pydantic attributes.
• sagemaker-core/tests/unit/test_pydantic_compat.py: Simplify tests to match the simplified _pydantic_compat module. Fix the mock import approach to be more robust.

[aviruthen]

Description

The issue is a pydantic/pydantic-core version incompatibility. sagemaker-core declares 'pydantic>=2.0.0,<3.0.0' as a dependency but does NOT declare pydantic-core. Pydantic requires an exact matching pydantic-core version (e.g., pydantic 2.11.5 requires pydantic-core==2.41.5). When users run 'pip install --force-reinstall', pip may resolve pydantic-core to a newer version (e.g., 2.42.0) that is incompatible with the installed pydantic version. This causes a SystemError at import time when any module transitively imports pydantic (e.g., DataSet → session_helper → pydantic). The fix is to either: (1) add pydantic-core as a dependency that will be co-resolved with pydantic, or (2) tighten pydantic constraints. The best approach is to let pydantic manage its own pydantic-core dependency but also exclude known-incompatible pydantic-core versions, plus add a runtime compatibility check that provides a helpful error message.

Related Issue

Related issue: 5652

Changes Made

• sagemaker-core/pyproject.toml
• sagemaker-core/src/sagemaker/core/_pydantic_compat.py
• sagemaker-core/src/sagemaker/core/__init__.py
• sagemaker-core/tests/unit/test_pydantic_compat.py

AI-Generated PR

This PR was automatically generated by the PySDK Issue Agent.

• Confidence score: 85%
• Classification: bug
• SDK version target: V3

Merge Checklist

• Changes are backward compatible
• Commit message follows prefix: description format
• Unit tests added/updated
• Integration tests added (if applicable)
• Documentation updated (if applicable)

[mohamedzeidan2021]

1. pydantic-core as a direct dependency is counterproductive. Pydantic already pins its exact required pydantic-core version internally. Adding a separate loose >=2.27.0,<3.0.0 constraint gives pip two conflicting signals and can cause the very mismatch this PR is trying to prevent. This line should be removed.

2. The init.py error handler silently swallows the main error case. The SystemError path raises
   ImportError("Pydantic version incompatibility detected: ..."), but the catch checks for "incompatible" — which is not a substring of
   "incompatibility". The condition evaluates to False and the error is silently ignored via pass.

3. The version comparison in _pydantic_compat.py is effectively dead code. pydantic.pydantic_core_version doesn't exist in standard
   pydantic 2.x, and the fallback to pydantic._internal._generate_schema doesn't contain a version string either. It always hits the return early.

The only part that's actually useful is catching SystemError and converting it to a clear ImportError. That's ~10 lines and doesn't need the
dependency change or the version comparison logic.

[aviruthen]

🤖 Iteration #1 — Review Comments Addressed

fix: Pydantic/pydantic-core version incompatibility error handling

Problem

When users install sagemaker with pip install --force-reinstall, pip may resolve pydantic-core to a version that is incompatible with the installed pydantic version. Pydantic requires an exact matching pydantic-core version (e.g., pydantic 2.11.5 requires pydantic-core==2.41.5). When the versions are mismatched, import pydantic raises a cryptic SystemError that is confusing to users.

Root Cause

Pydantic internally pins its exact required pydantic-core version. When --force-reinstall or --no-deps is used, pip may install a newer pydantic-core that doesn't match, causing a SystemError at import time.

Solution

This PR adds a minimal compatibility check that:

1. Catches SystemError from import pydantic and converts it to a clear ImportError with remediation instructions (pip install pydantic pydantic-core --force-reinstall)
2. Does NOT add pydantic-core as a direct dependency — pydantic already manages its own pydantic-core dependency internally, and adding a separate loose constraint would give pip conflicting signals
3. Removes dead code — the previous iteration had version comparison logic that relied on non-existent pydantic attributes (__pydantic_core_version__) and was effectively unreachable

Changes

• sagemaker-core/pyproject.toml: Removed pydantic-core as a direct dependency (pydantic manages it internally)
• sagemaker-core/src/sagemaker/core/_pydantic_compat.py: Simplified to only catch SystemError → ImportError conversion (~10 lines of useful code)
• sagemaker-core/src/sagemaker/core/__init__.py: Fixed error handling that was silently swallowing the ImportError due to substring mismatch ("incompatible" vs "incompatibility")
• sagemaker-core/tests/unit/test_pydantic_compat.py: Fixed mock import approach and added test for error chaining

Testing

• Unit tests verify the SystemError → ImportError conversion works correctly
• Unit tests verify the error message contains remediation instructions
• Unit tests verify the original SystemError is chained as __cause__
• Compatible versions pass through without any error

Note: CI integration test failures in the previous iteration were all due to pre-existing resource limit issues (endpoint quotas, hub content limits) unrelated to this change.

Comments reviewed: 20
Files modified:sagemaker-core/pyproject.toml, sagemaker-core/src/sagemaker/core/__init__.py, sagemaker-core/src/sagemaker/core/_pydantic_compat.py, sagemaker-core/tests/unit/test_pydantic_compat.py

• sagemaker-core/pyproject.toml: Remove pydantic-core as a direct dependency. Pydantic already pins its exact required pydantic-core version internally, so adding a separate loose constraint gives pip conflicting signals and can caus
• sagemaker-core/src/sagemaker/core/__init__.py: Simplify the pydantic compatibility check in init.py. The previous version had a bug where it checked for 'incompatible' but the raised error said 'incompatibility', causing the condition to be Fa
• sagemaker-core/src/sagemaker/core/_pydantic_compat.py: Simplify _pydantic_compat.py to only catch SystemError and convert it to a clear ImportError. Remove the dead version comparison code that relied on non-existent pydantic attributes.
• sagemaker-core/tests/unit/test_pydantic_compat.py: Simplify tests to match the simplified _pydantic_compat module. Fix the mock import approach to be more robust.

[aviruthen]

Description

The issue is a pydantic/pydantic-core version incompatibility. sagemaker-core declares 'pydantic>=2.0.0,<3.0.0' as a dependency but does NOT declare pydantic-core. Pydantic requires an exact matching pydantic-core version (e.g., pydantic 2.11.5 requires pydantic-core==2.41.5). When users run 'pip install --force-reinstall', pip may resolve pydantic-core to a newer version (e.g., 2.42.0) that is incompatible with the installed pydantic version. This causes a SystemError at import time when any module transitively imports pydantic (e.g., DataSet → session_helper → pydantic). The fix is to either: (1) add pydantic-core as a dependency that will be co-resolved with pydantic, or (2) tighten pydantic constraints. The best approach is to let pydantic manage its own pydantic-core dependency but also exclude known-incompatible pydantic-core versions, plus add a runtime compatibility check that provides a helpful error message.

Related Issue

Related issue: 5652

Changes Made

• sagemaker-core/pyproject.toml
• sagemaker-core/src/sagemaker/core/_pydantic_compat.py
• sagemaker-core/src/sagemaker/core/__init__.py
• sagemaker-core/tests/unit/test_pydantic_compat.py

AI-Generated PR

This PR was automatically generated by the PySDK Issue Agent.

• Confidence score: 85%
• Classification: bug
• SDK version target: V3

Merge Checklist

• Changes are backward compatible
• Commit message follows prefix: description format
• Unit tests added/updated
• Integration tests added (if applicable)
• Documentation updated (if applicable)

[mohamedzeidan2021]

1. pydantic-core as a direct dependency is counterproductive. Pydantic already pins its exact required pydantic-core version internally. Adding a separate loose >=2.27.0,<3.0.0 constraint gives pip two conflicting signals and can cause the very mismatch this PR is trying to prevent. This line should be removed.

2. The init.py error handler silently swallows the main error case. The SystemError path raises
   ImportError("Pydantic version incompatibility detected: ..."), but the catch checks for "incompatible" — which is not a substring of
   "incompatibility". The condition evaluates to False and the error is silently ignored via pass.

3. The version comparison in _pydantic_compat.py is effectively dead code. pydantic.pydantic_core_version doesn't exist in standard
   pydantic 2.x, and the fallback to pydantic._internal._generate_schema doesn't contain a version string either. It always hits the return early.

The only part that's actually useful is catching SystemError and converting it to a clear ImportError. That's ~10 lines and doesn't need the
dependency change or the version comparison logic.

[aviruthen]

🤖 Iteration #1 — Review Comments Addressed

fix: Pydantic/pydantic-core version incompatibility error handling

Problem

When users install sagemaker with pip install --force-reinstall, pip may resolve pydantic-core to a version that is incompatible with the installed pydantic version. Pydantic requires an exact matching pydantic-core version (e.g., pydantic 2.11.5 requires pydantic-core==2.41.5). When the versions are mismatched, import pydantic raises a cryptic SystemError that is confusing to users.

Root Cause

Pydantic internally pins its exact required pydantic-core version. When --force-reinstall or --no-deps is used, pip may install a newer pydantic-core that doesn't match, causing a SystemError at import time.

Solution

This PR adds a minimal compatibility check that:

1. Catches SystemError from import pydantic and converts it to a clear ImportError with remediation instructions (pip install pydantic pydantic-core --force-reinstall)
2. Does NOT add pydantic-core as a direct dependency — pydantic already manages its own pydantic-core dependency internally, and adding a separate loose constraint would give pip conflicting signals
3. Removes dead code — the previous iteration had version comparison logic that relied on non-existent pydantic attributes (__pydantic_core_version__) and was effectively unreachable

Changes

• sagemaker-core/pyproject.toml: Removed pydantic-core as a direct dependency (pydantic manages it internally)
• sagemaker-core/src/sagemaker/core/_pydantic_compat.py: Simplified to only catch SystemError → ImportError conversion (~10 lines of useful code)
• sagemaker-core/src/sagemaker/core/__init__.py: Fixed error handling that was silently swallowing the ImportError due to substring mismatch ("incompatible" vs "incompatibility")
• sagemaker-core/tests/unit/test_pydantic_compat.py: Fixed mock import approach and added test for error chaining

Testing

• Unit tests verify the SystemError → ImportError conversion works correctly
• Unit tests verify the error message contains remediation instructions
• Unit tests verify the original SystemError is chained as __cause__
• Compatible versions pass through without any error

Note: CI integration test failures in the previous iteration were all due to pre-existing resource limit issues (endpoint quotas, hub content limits) unrelated to this change.

Comments reviewed: 20
Files modified:sagemaker-core/pyproject.toml, sagemaker-core/src/sagemaker/core/__init__.py, sagemaker-core/src/sagemaker/core/_pydantic_compat.py, sagemaker-core/tests/unit/test_pydantic_compat.py

• sagemaker-core/pyproject.toml: Remove pydantic-core as a direct dependency. Pydantic already pins its exact required pydantic-core version internally, so adding a separate loose constraint gives pip conflicting signals and can caus
• sagemaker-core/src/sagemaker/core/__init__.py: Simplify the pydantic compatibility check in init.py. The previous version had a bug where it checked for 'incompatible' but the raised error said 'incompatibility', causing the condition to be Fa
• sagemaker-core/src/sagemaker/core/_pydantic_compat.py: Simplify _pydantic_compat.py to only catch SystemError and convert it to a clear ImportError. Remove the dead version comparison code that relied on non-existent pydantic attributes.
• sagemaker-core/tests/unit/test_pydantic_compat.py: Simplify tests to match the simplified _pydantic_compat module. Fix the mock import approach to be more robust.