Share the Fixed Fence Parser Across All Three Callers

[ptr727]

Summary

Follow-up to #902, addressing a finding raised on the #900 -> #901 promotion PR review (CodeRabbit).

unfenced_text() (fixed in #902) now handles fence marker family, length, and indentation correctly, but extract_section() and strip_sections() still used the original naive toggle-on-any-marker logic. Verified independently before fixing: a ~~~ line nested inside a ``` block made both exit the fenced state early, so a following ## line could end the region short.

This is not cosmetic for extract_section(): it is what the verbatim byte-for-byte section check hashes, so a nested example inside a fenced code sample could silently truncate what gets compared against the hub canonical.

Extracted the corrected per-line fence logic from unfenced_text() into _fence_step(), a single pure function all three now call, so the fence-matching rule lives in exactly one place instead of three near-duplicates that can drift apart the way the first two already had. Added a regression case covering the nested-marker scenario for both functions, on top of the existing extract_section and strip_sections (via template_ref_outside_verbatim) coverage, which still passes unchanged.

Verification

• python3 spec/audit.py --selftest -> SELFTEST PASS, including the new nested-fence regression and the full pre-existing suite unchanged
• python3 spec/validate.py -> Spec validation OK
• ruff check . / ruff format --check . -> clean
• python3 scripts/prose_lint.py --diff origin/develop spec/audit.py -> clean

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 21d066e1-d0f5-48bb-ad6c-ec316d813e9a

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

_{Comment @coderabbitai help to get the list of available commands.}

[qodo-code-review]

PR Summary by Qodo

Share CommonMark fence parsing across audit section utilities

🐞 Bug fix🧪 Tests🕐 20-40 Minutes

AI Description

• Centralize CommonMark fence state transitions in a shared _fence_step() helper.
• Fix extract_section()/strip_sections() to ignore mismatched nested fences as boundaries.
• Add selftest regression covering ~~~ nested inside ``` fenced blocks.

Diagram

graph TD
MD["Markdown text"] --> ES["extract_section()"] --> FS["_fence_step()"]
MD --> SS["strip_sections()"] --> FS
MD --> UT["unfenced_text()"] --> FS

Loading

High-Level Assessment

The following are alternative approaches to this PR:

1. Keep per-function fence logic (status quo)

• ➕ No new helper function to learn or document
• ➖ Fence rules drift between callers (the original bug)
• ➖ Harder to add/verify new CommonMark edge cases consistently

2. Adopt a full Markdown/CommonMark parser library for fencing/AST

• ➕ Delegates correctness to a well-tested parser
• ➕ Could simplify future markdown structure handling
• ➖ Heavier dependency and integration cost for a small, line-based need
• ➖ May change semantics vs current byte-preserving audit expectations

3. Introduce a small stateful “FenceScanner” object instead of a pure function

• ➕ Encapsulates marker state more explicitly
• ➕ Easier to extend if more scan features are added
• ➖ More code/indirection than needed for three call sites
• ➖ Pure function is already sufficient and easy to unit-test

Recommendation: The chosen approach (a single pure _fence_step() used by all callers) is the best fit: it eliminates duplicated, drifting parsing logic while preserving the existing line-based/byte-preserving audit behavior. A full Markdown parser would likely be overkill and risk semantic shifts for the hashing/audit use cases.

Files changed (1) +55 / -29

Bug fix (1) +55 / -29

audit.pyCentralize CommonMark fence state and add nested-fence regression+55/-29

Centralize CommonMark fence state and add nested-fence regression

• Introduces '_fence_step()' to implement CommonMark-compatible per-line fence transitions and boundary detection. Updates 'extract_section()', 'strip_sections()', and 'unfenced_text()' to use the shared helper, fixing early fence termination on mismatched nested markers. Extends '_selftest()' with a regression that ensures nested '~~~' inside ''' does not truncate sections or stripping behavior.

spec/audit.py

[Copilot]

🟡 Changes recommended

A newly added comment uses a mid-sentence semicolon, which violates the repository's no-semicolon prose rule for agent-authored comments.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR refactors the CommonMark fence parsing logic so extract_section(), strip_sections(), and unfenced_text() share one consistent implementation, preventing nested/mismatched fence markers from prematurely ending fenced regions and truncating section extraction.

Changes:

• Extracted the per-line fence state transition into _fence_step() and updated all three callers to use it.
• Updated extract_section() and strip_sections() to avoid the prior naive toggle-on-any-marker behavior.
• Added a selftest regression covering a mismatched fence marker nested inside another fenced block.

File summaries

File	Description
spec/audit.py	Centralizes fence parsing in _fence_step() and adds a regression test to prevent section truncation due to nested/mismatched fences.

Review details

• Files reviewed: 1/1 changed files
• Comments generated: 1
• Review effort level: Lite

---

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[qodo-code-review]

Code Review by Qodo

🐞 Bugs (0)📘 Rule violations (0)📜 Skill insights (0)

1. Selftest comment contains semicolon✓ Resolved📜 Skill insight✧ Quality

Description

A newly added inline comment uses a semicolon as prose punctuation. This violates the rule against
semicolons in agent-authored prose.

Code

spec/audit.py[2841]

+ # _fence_step carries this contract; extract_section and strip_sections each consume it.

Relevance

●●● Strong

Recent accepted prose-quality feedback in spec/audit.py supports fixing semicolon punctuation in the
new selftest comment.

PR-#555
PR-#402

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

PR Compliance ID 2826756 forbids semicolons in agent-authored prose. The new comment uses ; to
join two independent clauses, which is prose punctuation rather than a code delimiter.

spec/audit.py[2841-2841]
Skill: comment-and-doc-style

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution
## Issue description
An inline comment uses a semicolon as prose punctuation.
## Issue Context
Compliance disallows semicolons in agent-authored prose; rewrite as two sentences or use a period.
## Fix Focus Areas
- spec/audit.py[2840-2842]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

2. _fence_step docstring uses - ✓ Resolved📜 Skill insight✧ Quality

Description

The _fence_step() docstring uses a spaced hyphen ( - ) as a dash inside a sentence. This
violates the prose punctuation rule and should be rewritten as a comma, parentheses, or split
sentences.

Code

spec/audit.py[R397-399]

+ opener's, and nothing but whitespace after that run - a mismatched or shorter marker (a ~~~ example+ inside a ``` block, a ``` inside a longer ````) or trailing text (an opening fence's language tag has+ no closing counterpart) does not close it.

Relevance

●●● Strong

Recent accepted findings show reviewers actively revise prose punctuation and wording in
spec/audit.py.

PR-#555
PR-#402

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

PR Compliance ID 2826777 forbids using  -  as a dash in prose. The added docstring line includes
after that run - a mismatched..., which is exactly the prohibited pattern.

spec/audit.py[397-399]
Skill: comment-and-doc-style

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution
## Issue description
The `_fence_step()` docstring uses ` - ` as a dash to join/interrupt a sentence.
## Issue Context
Compliance forbids spaced-hyphen dashes in prose; use a comma, parentheses, or split into two sentences.
## Fix Focus Areas
- spec/audit.py[395-399]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

3. _fence_step docstring wraps sentences✗ Dismissed📜 Skill insight✧ Quality

Description

The new _fence_step() docstring wraps a single sentence across multiple lines, violating the
one-sentence-per-line comment structure rule. This reduces readability and makes future edits more
error-prone.

Code

spec/audit.py[R392-393]

+ """One line's effect on fence state `(marker, marker_len)`: returns the state after the line, and+ whether the line itself is a fence boundary (opens or closes one) rather than fenced or plain content.

Relevance

●●● Strong

Recent spec/audit prose and grammar findings are accepted, supporting enforcement of documented
comment-style rules.

PR-#555
PR-#402

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

PR Compliance ID 2826725 requires multi-line comments to have one sentence per line and forbids
mid-sentence wrapping. In the added _fence_step() docstring, the opening sentence continues onto
the next line (line 392 into 393), demonstrating a mid-sentence wrap.

spec/audit.py[392-393]
Skill: comment-and-doc-style

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution
## Issue description
The `_fence_step()` docstring wraps sentences across lines, instead of using one sentence per line.
## Issue Context
Compliance requires multi-line comments/docstrings to be structured as exactly one sentence per line, with no mid-sentence wrapping.
## Fix Focus Areas
- spec/audit.py[392-400]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

Context sources

✅ Compliance rules (platform): 67 rules

✅ Skills: 5 invoked

comment-and-doc-style

dotnet-codestyle

python-codestyle

shell-codestyle

workflow-ci-contract

✅ Web pages:

🌐 CommonMark Spec

🌐 Spaces / tabs / code blocks, spec 0.28 - CommonMark Discussion

🌐 Tabs and indentation removal

+6 more

Review mode: ⚖️ Balanced: This is a behavioral parser refactor affecting section extraction, stripping, and verbatim hashing; although localized to one file and well-tested, fence-state edge cases can affect audit correctness and warrant a careful single-pass review.

Tip of the day

💡 Did you know, you can commit Qodo's fix in one click with committable suggestions (GitHub & GitLab)

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

[Copilot]

🟢 Approval recommended

The shared _fence_step() implementation is consistently applied across all three callers and the added regression selftest directly covers the reported failure mode.

Review details

• Files reviewed: 1/1 changed files
• Comments generated: 0 new
• Review effort level: Lite

[Copilot]

🟢 Approval recommended

The shared _fence_step() logic is consistently applied across all three callers and is covered by an added regression selftest targeting the previously verified failure mode.

Review details

• Files reviewed: 1/1 changed files
• Comments generated: 0 new
• Review effort level: Lite