gh-104683: Argument clinic: remove some unnecessary uses of self.next() in the DSLParser

[AlexWaygood]

Many state_foo methods on the DSLParser class in clinic.py are called only indirectly via the next() method:

cpython/Tools/clinic/clinic.py

Lines 4645 to 4652 in 407d7fd

	defnext(
	self,
	state: StateKeeper,
	line: str|None=None
	) ->None:
	self.state=state
	iflineisnotNone:
	self.state(line)

The reason for this is that the DSLParser parses argument-clinic input a line at a time; but state often persists across lines. (Example: a docstring can span multiple lines; by saving the "state" at the end of one line, the DSLParser is able to resume parsing with the same state when it begins parsing the next line, and therefore knows that it's in the middle of a docstring.)

However, there are two state_foo methods on the DSLParser class that are arguably badly named, and shouldn't be called via self.next(), even though they both currently are:

• state_modulename_name
• state_parameter_docstring_start

Both of these are examples of states that are impossible to span multiple lines. A modulename declaration cannot span multiple lines; nor can the start of a parameter docstring. As such, calling these methods via self.next() is needless indirection, and can be removed.

It is provable that both of these states cannot span multiple lines, due to the fact that neither state_modulename_name nor state_parameter_docstring_start ever exit early before setting self.state to something new (by calling self.next).

• Issue: Modernise code in Tools/clinic/ #104683

[AlexWaygood]

I renamed this function to distinguish it from the state_foo functions, all of which must only ever be called indirectly via self.next()

[erlend-aasland]

What do you think of parse_function_declaration?

Suggested change

	defparse_modulename_name(self, line: str) ->None:
	defparse_function_declaration(self, line: str) ->None:

[erlend-aasland]

I'm not sure (yet) this is a step in the right direction. By consolidating stages in the state machine, debugging it becomes harder. For example, if I add this debug print:

diff --git a/Tools/clinic/clinic.py b/Tools/clinic/clinic.py
index 6c5a2c7c85..b9d47d2e0e 100755
--- a/Tools/clinic/clinic.py+++ b/Tools/clinic/clinic.py@@ -4652,6 +4652,7 @@ def next(
line: str | None = None
) -> None:
self.state = state
+ print(f"--> {state.__name__:25} {line=!r}")
if line is not None:
self.state(line)

I get this:

$ cat test.c/*[clinic input]module mm.func a: int[clinic start generated code]*/static PyObject *m_func_impl(PyObject *module, int a)/*[clinic end generated code: output=188ac0e0fa832273 input=e686625fcf7cad0e]*/
$ python3.12 Tools/clinic/clinic.py test.c--> state_modulename_name line='m.func'--> state_parameters_start line=None--> state_parameter line=' a: int'

However, with this PR, I now get:

$ python3.12 Tools/clinic/clinic.py test.c--> state_parameters_start line=None--> state_parameter line=' a: int'

[AlexWaygood]

Right, that's a very good point, and I think exposes that my mental model of the state machine here was subtly wrong. I think there may be another solution to the "But wait, these branches are unreachable!" problem I came up against in https://github.com/Argument-Clinic/cpython/pull/14; I'll have a play around.