gh-130870 Fix _eval_type Handling for GenericAlias with Unflattened Arguments and Union Types

[sharktide]

Fixes the handling of GenericAlias types in _eval_type by correctly unflattening callable arguments when necessary. Also improves the resolution of Union types by evaluating their arguments properly.

Tests (View comments below):
#130897 (comment)

• Issue: typing._eval_type is not preserving GenericAlias subclasses #130870

[bedevere-app]

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

[bedevere-app]

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

[sharktide]

I have made the requested changes; please review again

[bedevere-app]

Thanks for making the requested changes!

@JelleZijlstra: please review the changes made to this pull request.

[sharktide]

@JelleZijlstra

HI! Sorry to disturb you, but could you please re-review this pull request? TiA

[sharktide]

@JelleZijlstra or @AlexWaygood Could you please check this PR again? It has been over a month I have been waiting.

[JelleZijlstra]

The test cases don't use any public APIs so as far as I can see this PR doesn't change any behaviors that we care about.

[sharktide]

The test cases don't use any public APIs so as far as I can see this PR doesn't change any behaviors that we care about.

@JelleZijlstra
While the changes might seem to address internal behaviors, They also contribute to more robust handling of GenericAlias and Union types within the _eval_type function (imo), ensuring that edge cases like callable argument unpacking and recursive type evaluation are properly managed. This improvement could indirectly enhance the reliability of features relying on these internal implementations.

It took me so long to write that :)

[sharktide]

All of these tests still test only _eval_type, which is a private function and doesn't have any guaranteed behavior. Can you demonstrate that this fixes any operation that involves a public API?

@JelleZijlstra Please review again. I added new tests to demonstrate the behavior with a public api (get type hints). They passed.

Note: The fail of the Thread Sanatizer workflow is likely not due to this PR (asynicio test that failed)

[sharktide]

@JelleZijlstra please review. I added the tests and everything works

the test fail is unrelated

[sharktide]

@JelleZijlstra I think you forgot to review again. Please do get to it soon.

[JelleZijlstra]

The reason I've been slow to review this is that I feel I'll basically have to rewrite everything; it's still not clear to me what publicly visible behavior this is supposed to fix.

Start with writing a test case that fails today and that uses only public APIs (get_type_hints or get_args maybe, not _eval_type).

[JelleZijlstra]

All of these tests pass on master today without your changes in typing.py.

[sharktide]

Not the one I sent in the comments!

[ZeroIntensity]

We shouldn't be adding tests that are unrelated to the change.

[bedevere-app]

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

[sharktide]

@JelleZijlstra You want a failing test? I'll give you a failing test:

classTestCallableAlias(BaseTestCase):
deftest_callable_alias_preserves_subclass(self):
C=ABCallable[[str, ForwardRef('int')], int]
classA:
c: C# Explicitly pass global namespace to ensure correct resolutionhints=get_type_hints(A, globalns=globals())
# Ensure evaluated type retains the correct subclass (_CallableGenericAlias)self.assertEqual(hints['c'].__class__, C.__class__)
# Ensure evaluated type retains correct originself.assertEqual(hints['c'].__origin__, C.__origin__)
# Instead of comparing raw ForwardRef, check if the resolution is correctexpected_args=tuple(intifisinstance(arg, ForwardRef) elseargforarginC.__args__)
self.assertEqual(hints['c'].__args__, expected_args)

Out without the fix: (Note that this was when I had f string assertionError messages. I removed them)

FAIL: test_callable_alias_preserves_subclass (__main__.TestCallableAlias.test_callable_alias_preserves_subclass)
----------------------------------------------------------------------
Traceback (most recent call last):
File "C:\Users\rihaan.meher\Documents\GitHub\cpython\zzz.py", line 30, in test_callable_alias_preserves_subclass
self.assertEqual(hints['c'].__class__, C.__class__, f"Expected {C.__class__}, got {hints['c'].__class__}")
AssertionError: <class 'types.GenericAlias'>!= <class 'collections.abc._CallableGenericAlias'>: Expected <class 'collections.abc._CallableGenericAlias'>, got <class 'types.GenericAlias'>

With my PR, this passes.

I added it to test_typing.py

Essentially what this PR does:

• Preserving Type Information: If Callable is incorrectly converted to GenericAlias, it loses its subclass-specific behavior, which could affect type introspection and runtime checks.

• Ensuring Correct Type Resolution: ForwardRef('int') should be properly resolved to int, maintaining expected behavior when evaluating type hints.

• Preventing Silent Type Corruption: If Callable is mishandled, it could lead to unexpected behavior in libraries and applications relying on precise type annotations.

[sharktide]

Test fails are unrelated: 504 gateway timeout in http/urllib tests

[sharktide]

The reason I've been slow to review this is that I feel I'll basically have to rewrite everything; it's still not clear to me what publicly visible behavior this is supposed to fix.

Start with writing a test case that fails today and that uses only public APIs (get_type_hints or get_args maybe, not _eval_type).

Look at the comment I just posted

[sharktide]

No reviewer action in at least 1mo —> @JelleZijlstra please review

[ZeroIntensity]

I agree with Jelle: I don't think this PR is solving an actual problem. This will end up exposing implemenation details, like _CallableGenericAlias. In practice, why do users care about something like _CallableGenericAlias over the (public) GenericAlias?

[ZeroIntensity]

We shouldn't be adding tests that are unrelated to the change.

[sharktide]

I agree with Jelle: I don't think this PR is solving an actual problem. This will end up exposing implemenation details, like _CallableGenericAlias. In practice, why do users care about something like _CallableGenericAlias over the (public) GenericAlias?

Explained in issue

[sharktide]

I have made the requested changes; please review again

[bedevere-app]

Thanks for making the requested changes!

@JelleZijlstra: please review the changes made to this pull request.

[JelleZijlstra]

Superseded by #131583.