Skip to content

Fix RST field list absorbing trailing blocks into last field (#86) - #113

Open
apoorvdarshan wants to merge 1 commit into
rr-:masterfrom
apoorvdarshan:fix/issue-86-rst-field-list-boundary
Open

Fix RST field list absorbing trailing blocks into last field (#86)#113
apoorvdarshan wants to merge 1 commit into
rr-:masterfrom
apoorvdarshan:fix/issue-86-rst-field-list-boundary

Conversation

@apoorvdarshan

Copy link
Copy Markdown

Fixes#86

Root cause

In the ReST parser (docstring_parser/rest.py), the field list is split into individual fields with:

formatchinre.finditer(r"(^:.*?)(?=^:|\Z)", meta_chunk, flags=re.S|re.M):

A field only ends at the next line starting with : or at the end of input. Any trailing block that follows the field list but has no leading : (e.g. an Example section after :rtype: dict) therefore gets pulled into the value of the last field. The last :rtype:/:param: type then swallows everything after it.

Reproduction

fromdocstring_parserimportparsedoc='''Creates a user with the given username, age, and activity status.:param username: The username of the user.:type username: str:return: A dictionary representing the created user.:rtype: dictExample:>>> create_user("Alice", 25){'username': 'Alice'}'''t=parse(doc)
print(repr(t.many_returns[-1].type_name))

Wrong output (before the fix):

'dict\nExample:\n\n>>> create_user("Alice", 25)\n{\'username\': \'Alice\'}'

Corrected output (after the fix):

'dict'

Fix

Terminate a field not only at the next :field: or end of input, but also at a blank line followed by unindented, non-field content — which is where an RST field list ends:

r"(^:.*?)(?=^:|\n[ \t]*\n(?=[^ \t\n:])|\Z)"

The new alternative \n[ \t]*\n(?=[^ \t\n:]) matches a blank line followed by a line that starts at the base indentation and is not another field, so the trailing block is no longer absorbed. This is deliberately narrow:

  • Indented continuation lines (with or without a preceding blank line) still stay in the field body.
  • Unindented continuation lines that follow without a blank separator (already supported behavior) are unaffected.

Tests

Added test_returns_does_not_absorb_trailing_block in docstring_parser/tests/test_rest.py, which asserts that a :rtype: dict followed by a trailing Example block yields type_name == "dict". The test fails on the unmodified code (the type name includes the whole Example block) and passes with the fix.

The full existing test suite passes (255 passed). black, isort, and pylint (repo config) are clean on the changed files.

Disclosure: prepared with AI assistance; reviewed and verified locally.

In the ReST parser, the field-splitting regex only broke a field at the
next `:field:` line or end of input. A trailing block after the field
list (for example an `Example` section following `:rtype: dict`) has no
leading `:`, so it was absorbed into the preceding field's value,
leaving type_name as e.g. "dict\nExample:\n>>> ...".
Terminate a field at a blank line followed by unindented, non-field
content, which ends the RST field list. Indented continuation lines and
unindented continuation lines without a blank separator are unaffected.
Fixesrr-#86
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Parsing of last RST params definition (incorrectly) includes rest-of-docs.

1 participant

@apoorvdarshan