Remove the old parser

[pablogsal]

BPO	40939
Nosy	@gvanrossum, @terryjreedy, @vstinner, @ned-deily, @encukou, @nnemkin, @hroncok, @lysnikolaou, @pablogsal, @miss-islington, @brandtbucher
PRs	bpo-40939: Remove the old parser #20768 bpo-40939: Generate keyword.py using pegen #20800 [3.9] bpo-40939: Generate keyword.py using the new parser (GH-20800) #20801 bpo-40939: Remove PEG parser easter egg #20802 [3.9] bpo-40939: Fix test_keyword for the old parser #20814 bpo-40939: Remove some extra references to PYTHONOLDPARSER #20815 bpo-40939: Clean and adapt the peg_generator directory after deleting the old parser #20822 bpo-40939: Remove the old parser (Part 2) #21005 [3.9] bpo-40939: Deprecate the PyParser_SimpleParse* functions #21012 bpo-40939: Rename PyPegen* functions to PyParser* #21016 [3.9] bpo-40939: Do not emit deprecation warnings inside CPython for old parser APIs #21025 [3.9] bpo-40939: Deprecate PyNode_Compile #21036 bpo-40939: run autoreconf to fix configure{.ac,} disparity #21152 bpo-40939: Delete remaining references to Grammar/Grammar #21624 bpo-40939: Add the new grammar to the grammar specification documentation #19969 [3.9] bpo-40939: Use the new grammar for the grammar specification documentation (GH-19969) #21641 bpo-40939: Remove even more references to the old parser #21642 bpo-41808: Add What's New 3.9 entry missing from master #22294 bpo-40939: Document removal of the old parser in 3.10 whatsnew #23321 bpo-40939: Restore some stable API functions incorrectly deleted #23606 bpo-40939: Remove documentation for PyParser_* & add porting notes #26855 [3.10] bpo-40939: Remove documentation for PyParser_* & add porting notes (GH-26855) #26898
Files	py_limited_api_example.zip: Example of using Py_CompileString with Py_LIMITED_API py3.txt: Imports from python3.dll

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/pablogsal'
closed_at = <Date 2020-12-02.05:17:21.232>
created_at = <Date 2020-06-10.11:25:17.640>
labels = ['interpreter-core', '3.10']
title = 'Remove the old parser'
updated_at = <Date 2021-06-28.09:36:20.843>
user = 'https://github.com/pablogsal'

bugs.python.org fields:

activity = <Date 2021-06-28.09:36:20.843>
actor = 'petr.viktorin'
assignee = 'pablogsal'
closed = True
closed_date = <Date 2020-12-02.05:17:21.232>
closer = 'pablogsal'
components = ['Interpreter Core']
creation = <Date 2020-06-10.11:25:17.640>
creator = 'pablogsal'
dependencies = []
files = ['49642', '49647']
hgrepos = []
issue_num = 40939
keywords = ['patch']
message_count = 69.0
messages = ['371182', '371264', '371265', '371266', '371267', '371268', '371270', '371271', '371288', '371289', '371290', '371291', '371292', '371301', '371307', '371308', '371310', '371311', '371316', '371327', '371929', '371940', '371942', '371953', '371977', '372023', '372025', '372026', '372027', '372370', '374282', '374346', '374410', '374413', '374416', '374417', '377083', '380826', '380828', '380855', '380863', '381095', '381103', '381112', '381136', '381137', '381140', '381172', '381176', '381182', '381227', '381232', '382129', '382130', '382134', '382285', '382287', '382289', '382293', '382299', '382323', '382324', '382325', '382329', '382337', '387017', '396348', '396488', '396622']
nosy_count = 12.0
nosy_names = ['gvanrossum', 'terry.reedy', 'vstinner', 'ned.deily', 'petr.viktorin', 'nnemkin', 'Igor.Skochinsky', 'hroncok', 'lys.nikolaou', 'pablogsal', 'miss-islington', 'brandtbucher']
pr_nums = ['20768', '20800', '20801', '20802', '20814', '20815', '20822', '21005', '21012', '21016', '21025', '21036', '21152', '21624', '19969', '21641', '21642', '22294', '23321', '23606', '26855', '26898']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue40939'
versions = ['Python 3.10']

[pablogsal]

As stated in PEP-617, the old parser will be removed in Python 3.10

[pablogsal]

New changeset 9727694 by Lysandros Nikolaou in branch 'master':
bpo-40939: Generate keyword.py using the new parser (GH-20800)
9727694

[vstinner]

>>> __new_parser__
 File "<stdin>", line 1
 __new_parser__
 ^
SyntaxError: You found it!

"new", "ex" or "ng" are not really future proof names. Can we rename the keyword to "__peg_parser__"?

[lysnikolaou]

Can we rename the keyword to "__peg_parser__"?

I guess we could just remove this, as soon as the old parser is out. We were only using this to differentiate between the two parsers, when we were testing enabling/disabling the old one. I could get a PR ready to be merged after #64967 is there.

[pablogsal]

I guess we could just remove this, as soon as the old parser is out. We were only using this to differentiate between the two parsers, when we were testing enabling/disabling the old one. I could get a PR ready to be merged after #64967 is there.

I would personally would like to keep the easter egg, but I assume is better to rename it to "__peg_parser__".

[miss-islington]

New changeset 961edf7 by Miss Islington (bot) in branch '3.9':
bpo-40939: Generate keyword.py using the new parser (GH-20800)
961edf7

[lysnikolaou]

I would personally would like to keep the easter egg, but I assume is better to rename it to "__peg_parser__".

Ok then! On it.

[vstinner]

I did'nt ask to remove the easter egg. I'm just asking to avoid the "new" name. In my experience, each time that a "new" thing happens, later we have to use "new extended", "new_v2" or worse name :-)

Oh, if the name changes, please change it in 3.9 as well.

Look at this amazing names of the 5 flavors of functions parsing a string:

PyParser_ParseString()
PyParser_ParseStringFlags()
PyParser_ParseStringFlagsFilename()
PyParser_ParseStringFlagsFilenameEx() <= public!
PyParser_ParseStringObject()

Same for parsing a file:

PyParser_ParseFile()
PyParser_ParseFileFlags()
PyParser_ParseFileFlagsEx() <= public!
PyParser_ParseFileObject()

Or PyRun functions:

PyRun_String()
PyRun_AnyFile()
PyRun_AnyFileEx() <= public!
PyRun_AnyFileFlags()
PyRun_SimpleString()
PyRun_SimpleFile()
PyRun_SimpleFileEx() <= public!
PyRun_InteractiveOne()
PyRun_InteractiveLoop()
PyRun_File()
PyRun_FileEx() <= public!
PyRun_FileFlags()

ceval.c:

PyEval_EvalCode()
PyEval_EvalCodeEx() <= public!
_PyEval_EvalCodeWithName()
_PyEval_EvalCode()

I cannot count the number of "Ex" functions that we have :-)

Py_Finalize() -> :Py_FinalizeEx() <= public!
PyErr_Print() -> PyErr_PrintEx() <= public!
PySys_SetArgv() -> PySys_SetArgvEx() <= public!
PyErr_WarnEx() <= public!

_PyBytes_FormatEx()
_PyDict_MergeEx()
_Py_DecodeLocaleEx(), _Py_EncodeLocaleEx()
struct PyMemAllocatorEx
_Py_DecodeUTF8Ex(), _Py_EncodeUTF8Ex()
etc.

[gvanrossum]

Honestly I see no reason to keep that easter egg. Can we remove it please?