Skip to content

gh-104050: Add type annotations to sentinels in Argument Clinic#104589

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
erlend-aasland wants to merge 5 commits into from
Closed

gh-104050: Add type annotations to sentinels in Argument Clinic #104589

Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
3b12493
gh-104050: Add type annotations to sentinels in Argument Clinic
erlend-aasland May 16, 2023
5c1c2df
Merge branch 'main' into typed-clinic/sentinels
AlexWaygood May 17, 2023
d145ff8
Fix stuff
erlend-aasland May 18, 2023
5833421
Revert unneeded change
erlend-aasland May 18, 2023
703c417
Fix local rebase and write nicer if for line 2700
erlend-aasland May 18, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 21 additions & 20 deletions Tools/clinic/clinic.py
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -12,6 +12,7 @@
import contextlib
import copy
import cpp
import enum
import functools
import hashlib
import inspect
Expand All@@ -28,7 +29,7 @@

from collections.abc import Callable
from types import FunctionType, NoneType
from typing import Any, NamedTuple, NoReturn, Literal, overload
from typing import Any, Final, NamedTuple, NoReturn, Literal, overload

# TODO:
#
Expand DownExpand Up@@ -58,25 +59,22 @@
"return_value",
}

class Unspecified:
def __repr__(self) -> str:
return '<Unspecified>'

unspecified = Unspecified()

class Sentinels(enum.Enum):
unspecified = "unspecified"
NULL = "null"
unknown = "unknown"

class Null:
def __repr__(self) -> str:
return '<Null>'
return f"<{self.value.capitalize()}>"

NULL = Null()

unspecified: Final = Sentinels.unspecified
NULL: Final = Sentinels.NULL
unknown: Final = Sentinels.unknown

class Unknown:
def __repr__(self) -> str:
return '<Unknown>'
NullType = type(Sentinels.NULL)

@AlexWaygoodAlexWaygoodMay 18, 2023•
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, The type of Sentinels.Null is just Sentinels. For Python enums, all enum members are instances of the enum class. (This is why I was wondering if maybe the Null class should just be left as it is; it maybe needs to be its own class, rather than sharing the same class as the other sentinel values.)

@erlend-aaslanderlend-aaslandMay 18, 2023

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, you're right.

@erlend-aaslanderlend-aaslandMay 18, 2023

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps we should amend this instead:

cpython/Tools/clinic/clinic.py

Lines 2605 to 2607 in dcdc90d

# If not None, default must be isinstance() of this type.
# (You can also specify a tuple of types.)
default_type: bltns.type[Any] |tuple[bltns.type[Any], ...] |None=None

@erlend-aaslanderlend-aaslandMay 18, 2023

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think my approach is fundamentally flawed. This is pretty simple really: we simply want distinct (sentinel) types for default_type and the instance check (🥁).

@AlexWaygoodAlexWaygoodMay 18, 2023

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if we're in agreement or not (possibly my brain's going fuzzy at the end of a long day 😆), but I think for this PR, I might keep the refactor for unspecified and unknown, but leave Null as it is on main.

@AlexWaygoodAlexWaygoodMay 18, 2023

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fab, thanks!

@erlend-aaslanderlend-aaslandMay 18, 2023

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The annotation for default_type is probably also incorrect.

cpython/Tools/clinic/clinic.py

Lines 2598 to 2607 in dcdc90d

# The Python default value for this parameter, as a Python value.
# Or the magic value "unspecified" if there is no default.
# Or the magic value "unknown" if this value is a cannot be evaluated
# at Argument-Clinic-preprocessing time (but is presumed to be valid
# at runtime).
default: bool|Unspecified=unspecified
# If not None, default must be isinstance() of this type.
# (You can also specify a tuple of types.)
default_type: bltns.type[Any] |tuple[bltns.type[Any], ...] |None=None

@AlexWaygoodAlexWaygoodMay 18, 2023•
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The annotation for default_type is probably also incorrect.

It looks correct to me based on the comment and usage. But maybe I'm not seeing something. What makes you think it's incorrect?

@erlend-aaslanderlend-aaslandMay 18, 2023

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With a custom converter, you can override pretty much anything (AFAIK), so I thinkbltns.type[Any] is too narrow. I might be wrong.

@AlexWaygoodAlexWaygoodMay 18, 2023

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment indicates that default has to be an instance of default_type if default_type is not None, and isintance(x, Foo) for any given x will fail if Foo is not an instance of type or a tuple of objects which are instances of type


unknown = Unknown()

sig_end_marker = '--'

Expand DownExpand Up@@ -2600,7 +2598,7 @@ class CConverter(metaclass=CConverterAutoRegister):
# Or the magic value "unknown" if this value is a cannot be evaluated
# at Argument-Clinic-preprocessing time (but is presumed to be valid
# at runtime).
default: bool | Unspecified = unspecified
default: bool | Literal[Sentinels.unspecified] = unspecified

# If not None, default must be isinstance() of this type.
# (You can also specify a tuple of types.)
Expand DownExpand Up@@ -2690,7 +2688,7 @@ def __init__(self,
*, # Keyword only args:
c_default: str | None = None,
py_default: str | None = None,
annotation: str | Unspecified = unspecified,
annotation: str | Literal[Sentinels.unspecified] = unspecified,
unused: bool = False,
**kwargs
):
Expand All@@ -2699,7 +2697,10 @@ def __init__(self,
self.unused = unused

if default is not unspecified:
if self.default_type and not isinstance(default, (self.default_type, Unknown)):
if (self.default_type
and default is not unknown
and not isinstance(default, self.default_type)
):
if isinstance(self.default_type, type):
types_str = self.default_type.__name__
else:
Expand DownExpand Up@@ -3487,7 +3488,7 @@ def str_converter_key(types, encoding, zeroes):

class str_converter(CConverter):
type = 'const char *'
default_type = (str, Null, NoneType)
default_type = (str, NullType, NoneType)
format_unit = 's'

def converter_init(
Expand All@@ -3506,7 +3507,7 @@ def converter_init(
self.format_unit = format_unit
self.length = bool(zeroes)
if encoding:
if self.default not in (Null, None, unspecified):
if self.default not in (NULL, None, unspecified):
fail("str_converter: Argument Clinic doesn't support default values for encoded strings")
self.encoding = encoding
self.type = 'char *'
Expand DownExpand Up@@ -3654,7 +3655,7 @@ def parse_arg(self, argname: str, displayname: str) -> str:

class unicode_converter(CConverter):
type = 'PyObject *'
default_type = (str, Null, NoneType)
default_type = (str, NullType, NoneType)
format_unit = 'U'

def parse_arg(self, argname: str, displayname: str) -> str:
Expand All@@ -3678,7 +3679,7 @@ def parse_arg(self, argname: str, displayname: str) -> str:
@add_legacy_c_converter('Z#', accept={str, NoneType}, zeroes=True)
class Py_UNICODE_converter(CConverter):
type = 'const Py_UNICODE *'
default_type = (str, Null, NoneType)
default_type = (str, NullType, NoneType)

def converter_init(
self, *,
Expand Down