3.10's `make regen-pegen-metaparser` fails with 3.12+

[encukou]

On the 3.10 branch:

$ PYTHON_FOR_REGEN=/usr/bin/python3.11 make regen-pegen-metaparser 
PYTHONPATH=./Tools/peg_generator /usr/bin/python3.11 -m pegen -q python \
./Tools/peg_generator/pegen/metagrammar.gram \
-o ./Tools/peg_generator/pegen/grammar_parser.py.new
/usr/bin/python3.11 ./Tools/scripts/update_file.py ./Tools/peg_generator/pegen/grammar_parser.py \
./Tools/peg_generator/pegen/grammar_parser.py.new

$ PYTHON_FOR_REGEN=/usr/bin/python3.12 make regen-pegen-metaparser 
PYTHONPATH=./Tools/peg_generator /usr/bin/python3.12 -m pegen -q python \
./Tools/peg_generator/pegen/metagrammar.gram \
-o ./Tools/peg_generator/pegen/grammar_parser.py.new
 File "./Tools/peg_generator/pegen/metagrammar.gram", line 87
 | NAME '[' type=NAME '*' ']' '=' ~ item {NamedItem(name.string, item, f"{type.string}*")}
 ^
SyntaxError: pegen parse failure
For full traceback, use -v
make: *** [Makefile:854: regen-pegen-metaparser] Error 1

This now makes PR CI fail, see e.g. https://github.com/python/cpython/actions/runs/11274132145/job/31352641619?pr=125255

Linked PRs

• [3.10] gh-125529: Avoid f-strings in the metagrammar #125582

[lysnikolaou]

That's because of PEP 701 and the fact that f-strings are no longer STRING tokens (grammar actions need to be valid Python tokens and the accepted tokens need to be listed in the actions mini-grammar).

This patch solves the issue, though I don't know how we would fix this now that 3.10 is security-only:

diff --git a/Tools/peg_generator/pegen/grammar_parser.py b/Tools/peg_generator/pegen/grammar_parser.py
index 892df5cf8cc..de735649d46 100644
--- a/Tools/peg_generator/pegen/grammar_parser.py
+++ b/Tools/peg_generator/pegen/grammar_parser.py
@@ -420,7 +420,7 @@ def named_item(self) -> Optional[NamedItem]:
+++ b/Tools/peg_generator/pegen/grammar_parser.py
@@ -420,7 +420,7 @@ def named_item(self) -> Optional[NamedItem]:
 and
 (item := self.item())
 ):
- return NamedItem ( name . string , item , f"{type.string}*" )
+ return NamedItem ( name . string , item , type . string + "*" )
 self.reset(mark)
 if cut: return None
 cut = False
diff --git a/Tools/peg_generator/pegen/metagrammar.gram b/Tools/peg_generator/pegen/metagrammar.gram
index bb4355fd189..77695d5e0a9 100644
--- a/Tools/peg_generator/pegen/metagrammar.gram
+++ b/Tools/peg_generator/pegen/metagrammar.gram
@@ -84,7 +84,7 @@ items[NamedItemList]:
 | named_item { [named_item] }
 
 named_item[NamedItem]:
- | NAME '[' type=NAME '*' ']' '=' ~ item {NamedItem(name.string, item, f"{type.string}*")}
+ | NAME '[' type=NAME '*' ']' '=' ~ item {NamedItem(name.string, item, type.string+"*")}
 | NAME '[' type=NAME ']' '=' ~ item {NamedItem(name.string, item, type.string)}
 | NAME '=' ~ item {NamedItem(name.string, item)}
 | item {NamedItem(None, item)}

[encukou]

Thank you!

Ultimately that's the release manager's call, but I think this is the right solution. I'll prepare a PR and write some reasoning there.