Skip to content

gh-127833: Reword and expand the Notation section - #134443

Merged
encukou merged 8 commits into
python:mainfrom
encukou:grammar-notation
Jun 9, 2025
Merged

gh-127833: Reword and expand the Notation section#134443
encukou merged 8 commits into
python:mainfrom
encukou:grammar-notation

Conversation

@encukou

@encukouencukou commented May 21, 2025•
edited by github-actions Bot
Loading

Copy link
Copy Markdown
Member

Prepare the docs for using the notation used in the python.gram file. If we want to sync the two, the meta-syntax should be the same.

Also, remove the distinction between lexical and syntactic rules.
With f- and t-strings, the line between the two is blurry.

📚 Documentation preview 📚: https://cpython-previews--134443.org.readthedocs.build/

@encukou
pythongh-127833: Reword and expand the Notation section
211f83d
Prepare the docs for using the notation used in the `python.gram`
file. If we want to sync the two, the meta-syntax should be the same.
Also, remove the distinction between lexical and syntactic rules.
With f- and t-strings, the line between the two is blurry.
@bedevere-appbedevere-appBot added docs Documentation in the Doc dir skip news labels May 21, 2025
@bedevere-appbedevere-appBot mentioned this pull request May 21, 2025
Open
@encukou@blaisep
Consolidate with the Full Grammar intro
ec90d40
Co-authored-by: Blaise Pabon <blaise@gmail.com>
@encukou

encukou commented May 21, 2025

Copy link
Copy Markdown
MemberAuthor

@lysnikolaou, does this look correct to you?
Rendered docs: notation section, full grammar

@encukouencukou mentioned this pull request May 28, 2025
Merged
@encukou
encukou marked this pull request as draft May 28, 2025 15:43
@encukou@blaisep@AA-Turner
Note practical difference between syntactic & lexical definitions
231e2ba
Co-authored-by: Blaise Pabon <blaise@gmail.com>
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
@encukou
encukou marked this pull request as ready for review May 28, 2025 16:08
@lysnikolaou

lysnikolaou commented May 28, 2025

Copy link
Copy Markdown
Member

I'll have a look at this tomorrow if that's okay.

lysnikolaou
lysnikolaou reviewed May 29, 2025
View reviewed changes

@lysnikolaoulysnikolaou left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great in general! Left a few not-so-significant comments inline

Comment threadDoc/reference/grammar.rst Outdated
Comment threadDoc/reference/introduction.rst Outdated
Comment on lines +132 to +136
* ``"a"..."z"``: Two literal characters separated by three dots mean a choice
of any single character in the given (inclusive) range of ASCII characters.
* ``<...>``: A phrase between angular brackets gives an informal description
of the matched symbol (for example, ``<any ASCII character except "\">``),
or an abbreviation that is defined in nearby text (for example, ``<Lu>``).

@lysnikolaoulysnikolaouMay 29, 2025•
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we be mentioning somewhere that these are not part of the actual Python grammar, but part of the notation to make it easier to describe specific constructs? Maybe as part of the first paragraph that says that this is a mixture of EBNF and PEG?

@encukouencukouJun 4, 2025

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I moved them to the end, and (re-)added notes that they're only used in the lexer definitions.

Comment threadDoc/reference/introduction.rst
The definition to the right of the colon uses the following syntax elements:

* ``name``: A name refers to another rule.
Where possible, it is a link to the rule's definition.

@lysnikolaoulysnikolaouMay 29, 2025

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What possibilities exist, other than referencing another rule?

@encukouencukouJun 3, 2025

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rules that aren't defined by formal grammar in the docs, for example strings in https://docs.python.org/3/reference/compound_stmts.html#literal-patterns
I consider these to be issues that #127833 should eventually fix, but until then I'd keep the weasel word in.

Also: tokens are currently unlinked, and many don't have a (lexical) grammar rule so they should eventually link to prose.

cmarqu
cmarqu reviewed May 29, 2025
View reviewed changes
Comment threadDoc/reference/introduction.rst Outdated
encukouand others added 2 commits June 3, 2025 08:43
@encukou@lysnikolaou@cmarqu
Apply suggestions from code review
fa17c00
Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
Co-authored-by: Colin Marquardt <cmarqu42@gmail.com>
@encukou
Constrain "a"..."z" and <...> to lexical definitions
327b90d
encukou
encukou commented Jun 7, 2025
View reviewed changes
Comment threadDoc/reference/introduction.rst Outdated
@encukou

encukou commented Jun 9, 2025

Copy link
Copy Markdown
MemberAuthor

Thanks for the reviews!
I'll merge; any fixes can go in follow-up PRs.

@encukouencukou added the needs backport to 3.14 bugs and security fixes label Jun 9, 2025
@encukou
encukou merged commit 28d91d0 into python:mainJun 9, 2025
@miss-islington-app

miss-islington-appBot commented Jun 9, 2025

Copy link
Copy Markdown

Thanks @encukou for the PR 🌮🎉.. I'm working now to backport this PR to: 3.14.
🐍🍒⛏🤖

@github-project-automationgithub-project-automationBot moved this from Todo to Done in Docs PRsJun 9, 2025
@encukou
encukou deleted the grammar-notation branch June 9, 2025 13:50
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Jun 9, 2025
@encukou@blaisep
@AA-Turner@lysnikolaou@cmarqu@miss-islington
pythongh-127833: Reword and expand the Notation section (pythonGH-134443
adc4b14
)
Prepare the docs for using the notation used in the `python.gram`
file. If we want to sync the two, the meta-syntax should be the same.
Link the Full Grammar docs here; keep only a few extras.
Also, remove the distinction between lexical and syntactic rules,
except for whitespace handling.
With f- and t-strings, the line between the two is blurry.
(cherry picked from commit 28d91d0)
Co-authored-by: Petr Viktorin <encukou@gmail.com>
Co-authored-by: Blaise Pabon <blaise@gmail.com>
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
Co-authored-by: Colin Marquardt <cmarqu42@gmail.com>
@bedevere-app

bedevere-appBot commented Jun 9, 2025

Copy link
Copy Markdown

GH-135301 is a backport of this pull request to the 3.14 branch.

@bedevere-appbedevere-appBot removed the needs backport to 3.14 bugs and security fixes label Jun 9, 2025
encukou added a commit that referenced this pull request Jun 9, 2025
@miss-islington@encukou
@blaisep@AA-Turner@lysnikolaou@cmarqu
[3.14]gh-127833: Reword and expand the Notation section (GH-134443) (G…
8b35651
…H-135301)
Prepare the docs for using the notation used in the `python.gram`
file. If we want to sync the two, the meta-syntax should be the same.
Link the Full Grammar docs here; keep only a few extras.
Also, remove the distinction between lexical and syntactic rules,
except for whitespace handling.
With f- and t-strings, the line between the two is blurry.
(cherry picked from commit 28d91d0)
Co-authored-by: Petr Viktorin <encukou@gmail.com>
Co-authored-by: Blaise Pabon <blaise@gmail.com>
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
Co-authored-by: Colin Marquardt <cmarqu42@gmail.com>
lkollar pushed a commit to lkollar/cpython that referenced this pull request Jun 19, 2025
@encukou@blaisep
@AA-Turner@lysnikolaou@cmarqu@lkollar
pythongh-127833: Reword and expand the Notation section (pythonGH-134443
f38ea80
)
Prepare the docs for using the notation used in the `python.gram`
file. If we want to sync the two, the meta-syntax should be the same.
Link the Full Grammar docs here; keep only a few extras.
Also, remove the distinction between lexical and syntactic rules,
except for whitespace handling.
With f- and t-strings, the line between the two is blurry.
Co-authored-by: Blaise Pabon <blaise@gmail.com>
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
Co-authored-by: Colin Marquardt <cmarqu42@gmail.com>
Pranjal095 pushed a commit to Pranjal095/cpython that referenced this pull request Jul 12, 2025
@encukou@blaisep
@AA-Turner@lysnikolaou@cmarqu@Pranjal095
pythongh-127833: Reword and expand the Notation section (pythonGH-134443
834b037
)
Prepare the docs for using the notation used in the `python.gram`
file. If we want to sync the two, the meta-syntax should be the same.
Link the Full Grammar docs here; keep only a few extras.
Also, remove the distinction between lexical and syntactic rules,
except for whitespace handling.
With f- and t-strings, the line between the two is blurry.
Co-authored-by: Blaise Pabon <blaise@gmail.com>
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
Co-authored-by: Colin Marquardt <cmarqu42@gmail.com>
taegyunkim pushed a commit to taegyunkim/cpython that referenced this pull request Aug 4, 2025
@encukou@blaisep
@AA-Turner@lysnikolaou@cmarqu@taegyunkim
pythongh-127833: Reword and expand the Notation section (pythonGH-134443
1af6821
)
Prepare the docs for using the notation used in the `python.gram`
file. If we want to sync the two, the meta-syntax should be the same.
Link the Full Grammar docs here; keep only a few extras.
Also, remove the distinction between lexical and syntactic rules,
except for whitespace handling.
With f- and t-strings, the line between the two is blurry.
Co-authored-by: Blaise Pabon <blaise@gmail.com>
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
Co-authored-by: Colin Marquardt <cmarqu42@gmail.com>
Agent-Hellboy pushed a commit to Agent-Hellboy/cpython that referenced this pull request Aug 19, 2025
@encukou@blaisep
@AA-Turner@lysnikolaou@cmarqu@Agent-Hellboy
pythongh-127833: Reword and expand the Notation section (pythonGH-134443
a58f283
)
Prepare the docs for using the notation used in the `python.gram`
file. If we want to sync the two, the meta-syntax should be the same.
Link the Full Grammar docs here; keep only a few extras.
Also, remove the distinction between lexical and syntactic rules,
except for whitespace handling.
With f- and t-strings, the line between the two is blurry.
Co-authored-by: Blaise Pabon <blaise@gmail.com>
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
Co-authored-by: Colin Marquardt <cmarqu42@gmail.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@lysnikolaoulysnikolaoulysnikolaou left review comments
@willingcwillingcAwaiting requested review from willingcwillingc is a code owner
@AA-TurnerAA-TurnerAwaiting requested review from AA-TurnerAA-Turner is a code owner
+1 more reviewer
@cmarqucmarqucmarqu left review comments
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

docsDocumentation in the Doc dirskip news

Projects

Docs PRs
Status: Done

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@encukou@lysnikolaou@cmarqu