Derby #18: Convert 31 sites to Argument Clinic across 23 files

[larryhastings]

BPO	20186
Nosy	@birkenfeld, @rhettinger, @jaraco, @taleinat, @larryhastings, @vadmium, @serhiy-storchaka
PRs	bpo-20186: Convert tuple object implementation to Argument Clinic. #614
Dependencies	bpo-34797: Convert heapq to the argument clinic
Files	heapq_clinic.patch modules_issue20186.patch issue20186.mathmodule.patch: updated patch for mathmodule issue20186.enumobject.patch issue20186._operator.patch: AC conversion of nearly all functions in the _operator module issue20186._operator.v2.patch: slightly revised patch after Serhiy's review issue20186._operator.v3.patch: slightly revised (again) patch after Serhiy's additional review issue20186._operator.v4.patch: slightly revised (yet again) patch after Serhiy's latest review issue20186.mathmodule.v2.patch: revised patch for mathmodule.c csv_clinic.patch lsprof_clinic.patch tracemalloc_clinic.patch symtable_clinic.patch enumobject-docstrings.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2014-01-08.00:23:06.113>
labels = ['extension-modules', 'type-feature', '3.7', 'expert-argument-clinic']
title = 'Derby #18: Convert 31 sites to Argument Clinic across 23 files'
updated_at = <Date 2021-03-06.02:14:31.336>
user = 'https://github.com/larryhastings'

bugs.python.org fields:

activity = <Date 2021-03-06.02:14:31.336>
actor = 'jaraco'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Extension Modules', 'Argument Clinic']
creation = <Date 2014-01-08.00:23:06.113>
creator = 'larry'
dependencies = ['34797']
files = ['33425', '33442', '39602', '39603', '39653', '39654', '39655', '39656', '39696', '46357', '46358', '46359', '46360', '46362']
hgrepos = []
issue_num = 20186
keywords = ['patch']
message_count = 55.0
messages = ['207644', '207914', '207917', '207923', '207931', '207933', '207934', '207935', '207936', '207937', '207938', '208009', '208011', '218717', '218718', '224767', '244731', '244736', '244737', '244971', '244972', '244973', '244974', '244975', '244977', '244978', '244987', '244989', '245276', '262010', '262022', '262036', '285798', '285799', '285802', '285803', '285806', '285807', '285808', '285833', '285915', '285916', '285917', '285919', '285920', '286939', '286947', '286950', '286952', '286955', '287062', '287063', '289440', '290163', '388190']
nosy_count = 8.0
nosy_names = ['georg.brandl', 'rhettinger', 'jaraco', 'taleinat', 'larry', 'python-dev', 'martin.panter', 'serhiy.storchaka']
pr_nums = ['614']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue20186'
versions = ['Python 3.7']

[larryhastings]

This issue is part of the Great Argument Clinic Conversion Derby,
where we're trying to convert as much of Python 3.4 to use
Argument Clinic as we can before Release Candidate 1 on January 19.

This issue asks you to change the following bundle of files:
Objects/tupleobject.c: 2 sites
Objects/memoryobject.c: 2 sites
Objects/descrobject.c: 2 sites
Objects/complexobject.c: 2 sites
Modules/_operator.c: 2 sites
Modules/_opcode.c: 2 sites
Modules/_lsprof.c: 2 sites
Modules/_heapqmodule.c: 2 sites
Objects/weakrefobject.c: 1 sites
Objects/structseq.c: 1 sites
Objects/rangeobject.c: 1 sites
Objects/object.c: 1 sites
Objects/moduleobject.c: 1 sites
Objects/funcobject.c: 1 sites
Objects/fileobject.c: 1 sites
Objects/enumobject.c: 1 sites
Objects/codeobject.c: 1 sites
Objects/boolobject.c: 1 sites
Modules/symtablemodule.c: 1 sites
Modules/mathmodule.c: 1 sites
Modules/_tracemalloc.c: 1 sites
Modules/_io/_iomodule.c: 1 sites
Modules/_csv.c: 1 sites

Talk to me (larry) if you only want to attack part of a bundle.

For instructions on how to convert a function to work with Argument
Clinic, read the "howto":
http://docs.python.org/dev/howto/clinic.html

[birkenfeld]

Looking at _csv.c, I see a few functions using PyArg_UnpackTuple. They should be converted too, no?

[birkenfeld]

Attached part 1 of mathmodule (17 functions).

I'm looking forward to a suggestion for handling the rest (see FUNC1/1A/2 macros :)

[larryhastings]

Wow. I never knew about PyArg_UnpackTuple. You're right, those should be converted too. Hooray, more entry points to convert.

I'll write something up for the howto about UnpackTuple.

I just did a quick check, and there are 96 entry points (by my count) that use PyArg_UnpackTuple(). Shall I create Derby issues #19 and #20, or do you have a better idea?

2. For FUNC1 / 1A / 2 macros: right now you'd have to just copy and paste over and over. There might be something you could do with a [python] block where you automatedly reuse the existing sigantures. I was thinking about having Clinic support it directly, maybe with the syntax:

/*[clinic input]
func_name = existing_func_name

docstring goes here
[...]*/

You'd skip the parameters and the return annotation. You could only reuse functions from the current file. Would that be a big boon to you?

[birkenfeld]

Wow. I never knew about PyArg_UnpackTuple. You're right, those
should be converted too. Hooray, more entry points to convert.
I'll write something up for the howto about UnpackTuple.

One thing to note is that (at least in math) many instances of UnpackTuple could have been replaced by ParseTuple. See for example math_hypot: it uses UnpackTuple to get two objects, and then immediately calls PyFloat_AsDouble on them. I've converted these using 'd' and not 'O' specifiers.

I just did a quick check, and there are 96 entry points (by my count)
that use PyArg_UnpackTuple(). Shall I create Derby issues #19 and
#20, or do you have a better idea?

Probably better to add them to the issues that cover their modules, otherwise people might get confused.

2. For FUNC1 / 1A / 2 macros: right now you'd have to just copy and
   paste over and over. There might be something you could do with a
   [python] block where you automatedly reuse the existing sigantures.
   I was thinking about having Clinic support it directly, maybe with
   the syntax:

/*[clinic input]
func_name = existing_func_name

docstring goes here
[...]*/

You'd skip the parameters and the return annotation. You could only
reuse functions from the current file. Would that be a big boon to you?

That sounds good.

On the other hand, if clinic expanded cpp macros we could... *:-)

[birkenfeld]

OK, here's a patch for _csv. Two problems here:

First problem is the __new__ method of the Dialect class:

• it has no docstring and no methoddef entry
• it is a class method, but the first arg is conventionally called "type"

I tried to hack something into clinic with a new decorator, but it may not be how you want it to look, take care.

Second problem is the functions reader(), writer(), register_dialect(): they parse their *args but pass their **kwargs through to another class.
Is there anything like a "**kwds" argument specifier?

BTW, for a module like _csv that is exported through a Python module named csv, should we use the "real" or the "nice" module name?

[birkenfeld]

Tried to tackle symtable -- it uses an O& converter. The clinic howto says

'O&' object(converter='name_of_c_function')

but

Traceback (most recent call last):
 File "Tools/clinic/clinic.py", line 2817, in <module>
 sys.exit(main(sys.argv[1:]))
 File "Tools/clinic/clinic.py", line 2813, in main
 parse_file(filename, output=ns.output, verify=not ns.force)
 File "Tools/clinic/clinic.py", line 1116, in parse_file
 cooked = clinic.parse(raw)
 File "Tools/clinic/clinic.py", line 1066, in parse
 parser.parse(block)
 File "Tools/clinic/clinic.py", line 2109, in parse
 self.state(line)
 File "Tools/clinic/clinic.py", line 2378, in state_parameter
 converter = dict[name](parameter_name, self.function, value, **kwargs)
 File "Tools/clinic/clinic.py", line 1403, in __init__
 self.converter_init(**kwargs)
TypeError: converter_init() got an unexpected keyword argument 'converter'

[birkenfeld]

_tracemalloc converted.

Its existing docstrings did use the

func(arg: argtype) -> rettype

convention. Is there a way in clinic to retain that?

[birkenfeld]

Here's _iomodule. _io.open has a whopping 100-line docstring, which is ... unfortunate ... to have duplicated in the file :)

[birkenfeld]

And _heapq. No problems there, except that it also used "->" return annotations in the docstring.

[birkenfeld]

And lsprof.

[birkenfeld]

OK, new patches coming in.

[birkenfeld]

Actually I put all I have in one. Rietveld doesn't care.

The mathmodule still awaits some kind of solution for the macro atrocities.

Objects will be attacked next.