Support return annotation in signature for Argument Clinic

[vstinner]

BPO	31939
Nosy	@brettcannon, @ncoghlan, @vstinner, @taleinat, @larryhastings, @bitdancer, @serhiy-storchaka

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2017-11-03.23:31:32.593>
labels = ['type-feature', '3.7', 'expert-argument-clinic']
title = 'Support return annotation in signature for Argument Clinic'
updated_at = <Date 2017-11-06.16:59:38.383>
user = 'https://github.com/vstinner'

bugs.python.org fields:

activity = <Date 2017-11-06.16:59:38.383>
actor = 'yselivanov'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Argument Clinic']
creation = <Date 2017-11-03.23:31:32.593>
creator = 'vstinner'
dependencies = []
files = []
hgrepos = []
issue_num = 31939
keywords = []
message_count = 11.0
messages = ['305525', '305557', '305560', '305563', '305564', '305592', '305600', '305606', '305608', '305649', '305658']
nosy_count = 7.0
nosy_names = ['brett.cannon', 'ncoghlan', 'vstinner', 'taleinat', 'larry', 'r.david.murray', 'serhiy.storchaka']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue31939'
versions = ['Python 3.7']

[vstinner]

Argument Clinic supports a few return types like NoneType, int, bool, etc. But the return type is omitted in the private docstring used to build the __text_signature__, finally used to build a Signature object in inspect.signature().

For example, os.dup() is declared in Modules/posixmodule.c with:

/*[clinic input]
os.dup -> int

fd: int
/

Return a duplicate of a file descriptor.
[clinic start generated code]*/

Currently, Argument Clinic generates:

PyDoc_STRVAR(os_dup__doc__,
"dup($module, fd, /)\n"
"--\n"
"\n"
"Return a duplicate of a file descriptor.");

The return type is omitted in the first line.

Finally, the return type is not documented in the signature:

haypo@selma$ ./python -c "import os,inspect; print(inspect.signature(os.dup))"
(fd, /)

I noticed this limitation while reviewing the PR 4265 which converts the select module to Argument Clinic. Previously, the return type like "-> None" or "-> int" was documented. With Argument Clinic, it's not documented nor generated in the signature, the information is lost.

[taleinat]

I'm not sure that the current concept of return converters in AC is can be used for specifying return value annotations.

For example, what if I want to annotate the return value type with using a return converter?

Another example: The current doc-string for select.select() begins with:

select(rlist, wlist, xlist[, timeout]) -> (rlist, wlist, xlist)

I can't see how describing the return type being a 3-tuple of lists would work with return converters.

[serhiy-storchaka]

Argument Clinic doesn't have any relations to annotations. It is just by accident use the syntax similar to the syntax of annotations in its declarations (and actually use Python parser for parsing them as anotations, but this is an implementation detail). It doesn't set argument annotations in signatures.

For example, chr() is declared with:

/*[clinic input]
chr as builtin_chr

i: int
/

Return a Unicode string of one character with ordinal i; 0 <= i <= 0x10ffff.
[clinic start generated code]*/

Argument Clinic generates:

PyDoc_STRVAR(builtin_chr__doc__,
"chr($module, i, /)\n"
"--\n"
"\n"
"Return a Unicode string of one character with ordinal i; 0 <= i <= 0x10ffff.");

I think it could be possible to make Argument Clinic generating argument annotations basing on the accepted by converters types, but we are far from this.

[vstinner]

I am not asking for a full support of return type annotation. Just export
what is already supported in AC, export it in the signature.

[serhiy-storchaka]

Currently nothing is supported Argument Clinic. Return converters don't contain information that can be used in return type annotation.

[taleinat]

Argument Clinic currently doesn't support input parameter type annotations either, though for this it has more relevant information in many cases.

I think it would be a great boon for AC to enable declaring type annotations, both for input parameters and for return values.

[bitdancer]

I believe we currently have a policy that the python standard library will not include type annotations, that those are provided by typeshed.

[brettcannon]

R. David is correct that currently the policy is there are no type hints in the stdlib. Now the interesting thing about AC is the type hint info could be kept just in the docstring and not added to __annotations__ which somewhat goes around this restriction. Plus I think Victor is only talking about the docstring ATM anyway.

[serhiy-storchaka]

Victor is talking about inspect.signature(). In Python 3.5+ the result is "(fd, /)". In older versions it raises a ValueError. The docstring in 3.4 is 'dup(fd) -> fd2\n\nReturn a duplicate of a file descriptor.' It doesn't contain information that dup() returns int. It contains implicit information that dup() perhaps returns other file descriptor, but this is already explicitly documented by words. I don't see a loss of information here.

[vstinner]

I'm sorry, I'm confused between docstrings and annotations.

My first goal is to not loose the return type from the docstring when the select module is converted to Argument Clinic, at least for the simplest return types. Complex return types can be documented manually in the body of a docstring.

Return *annotation* would be nice to have, but it seems to be opposed to the current policy. I started a discussion on python-dev to ask if we should modify this policy :-)

"[Python-Dev] Allow annotations using basic types in the stdlib?"
https://mail.python.org/pipermail/python-dev/2017-November/150233.html

[taleinat]

My apologies, I seem to have been the source of the confusion. I misunderstood your original meaning.