PEP 705: Rewrite to focus on TypedDict changes

[alicederyn]

• Change is either:
  • To a Draft PEP
  • To an Accepted or Final PEP, with Steering Council approval
  • To fix an editorial issue (markup, typo, link, header, etc)
• PR title prefixed with PEP number (e.g. PEP 123: Summary of changes)

Feedback on mailing list was to narrow the focus of the PEP to just modifying TypedDict, as that covers all the use-cases we were interested in. If separate use-cases for a separate TypedMapping protocol type come up, they can form the foundation of a separate PEP.

[JelleZijlstra]

This looks great. I have a few minor wording suggestions and two more substantive comments.

I'm also going to try implementing this in typing-extensions.

[JelleZijlstra]

I wrote a draft runtime implementation at python/typing_extensions#284. Please take a look to see if you agree with the choices I made.

It would also be good to mention some of the runtime implementation details in the PEP, like the new attributes __readonly_keys__, __mutable_keys__, __readonly__, and __other_keys__.

A few other notes:

• This PR specifies that it is an error to have both readonly=True and ReadOnly[] on a key. However, PEP 655 accepts the analogous construct (total=False + NotRequired[]). It might be nicer to allow it here too.
• Should a TypedDict that inherits from an other_keys=False TypedDict be required to also set other_keys=False explicitly?

[PIG208]

Once the new draft is merged, should we repurpose the original discussion on TypedMapping, or should a new post be created? Either way, we can perhaps discuss the extra fields/other_keys design there.

[JelleZijlstra]

It probably makes sense to start a new thread since the new version of the PEP is so different. I would prefer a separate thread about __extra__/other_keys since this PEP covers a lot more ground.

[alicederyn]

I'll do the implementation details from python/typing_extensions#284 as a follow-up PR.

[alicederyn]

This PR specifies that it is an error to have both readonly=True and ReadOnly[] on a key. However, PEP 655 accepts the analogous construct (total=False + NotRequired[]). It might be nicer to allow it here too.

I was thinking that it would be important to avoid confusing examples like:

classFoo(TypedDict, readonly=True):
key1: ReadOnly[str]
key2: int

Skimming this, it would be easy for someone to assume key2 was not readonly, missing the readonly=True.

I'm not sure what advantage respecifying ReadOnly might have to counterbalance this?

[PIG208]

Thank you for all the work! I just went through the proposal, and I think the interaction of other_keys=False with other parts of the type system is quite consistent with the draft PEP for __extra__, corresponding to the __extra__: Never case.

While reading this, I feel that other_key sort of relies on readonly (because it defaults extra keys' type to ReadOnly[NotRequired[Any]]), but in other cases, they don't seem to interact with each other a lot.

I'm not sure if having other_key is a requirement for implementing readonly. Potentially, we can limit the scope of PEP 705 to readonly to make it simpler and work on type discrimination on the other PEP. Would be interested in hearing more thoughts on this matter.

[PIG208]

My main concern that led me to add other_keys to PEP-705 is that declaring a TypedDict as @final to indicate other keys won't be present no longer semantically makes sense if keys can be read-only, so we need an officially sanctioned approach.

Just read this comment that explains the motivation for adding other_keys and the discussion on @final. I'm not sure if I understand why read-only keys make a @final annotated type not final anymore? Is it because the user is allowed to refine the types of read-only keys?

[JelleZijlstra]

Just read this comment that explains the motivation for adding other_keys and the discussion on @final. I'm not sure if I understand why read-only keys make a @final annotated type not final anymore? Is it because the user is allowed to refine the types of read-only keys?

Yes, that's right. So you could have this:

classTD1(TypedDict, other_keys=False):
a: ReadOnly[int|None]

And it would be legal to have a subclass:

classTD2(TD1, other_keys=False):
a: ReadOnly[int]

[PIG208]

I found microsoft/pyright#5254 to be helpful in understanding the issue with @final with a bit more context.

[alicederyn]

it defaults extra keys' type to ReadOnly[NotRequired[Any]]

This is supposed to exactly match the existing behaviour of a TypedDict, i.e. with no other_keys/__extra__ set:

• d[k] will be flagged as an error as k may not be present (NotRequired)
• d.get(k) is fine, but the returned type is unknown
• d[k] = ... will be flagged as an error as k is readonly

It seems as if PEP-728 is implicitly making all "extra" keys non-read-only. It would probably be useful to users to be able to use ReadOnly there too:

• subclasses could declare new required keys
• subclasses could refine the type of the extra keys

[JelleZijlstra]

A few small suggestions but this generally looks good, though we still need coordination with PEP 728 and there's a few unresolved threads.

[alicederyn]

Thanks, @JelleZijlstra. I believe I have now resolved all discussion threads? Though of course the text changes may raise more 😄

I think the consensus above re PEP 728 was to go with other_keys=Never, so have modified the proposal accordingly, and made multiple notes in the "rejected ideas" section, as well as linking the PEP 728 PR. I will update the PEP to reflect changes in PEP 728 as they arise.

[JelleZijlstra]

Looks good to me! @pablogsal as the sponsor should approve too.

[pablogsal]

LGTM

Thank you everyone for the fantastic work!