Fix: strip leading asterisk decoration from block doc comments by kivancgnlp · Pull Request #22901 · rust-lang/rust-analyzer

kivancgnlp · 2026-07-24T07:05:49Z

Summary

Strip a consistent leading * decoration from multiline block doc comments, matching rustdoc behavior.
Preserve existing behavior for line docs, #[doc = "..."] attributes, *foo, and blocks without a consistent star column.
Adjust per-line source-map offsets by the stripped byte count so doc links and syntax highlighting continue to map to the correct source ranges.

Tests Passed :

cargo test -p hir-def
cargo test -p ide -- hover doc_links
cargo test -p ide -- syntax_highlighting
cargo fmt -p hir-def -p ide -- --check
cargo clippy -p hir-def -p ide --all-targets -- --cap-lints warn

ChayimFriedman2 · 2026-07-24T07:36:51Z

Did you write the PR description using AI? I'm not sure you did, but if yes, please note that our AI policy forbids doing that.

ChayimFriedman2 · 2026-07-24T07:39:32Z

Looking at the code, it's pretty clear that you used AI for the code as well without disclosing it, also contrary to our AI policy. And I'm not sure it's properly reviewed either: I am pretty sure it's possible to implement the same thing using at most half the lines, and probably with better perf.

kivancgnlp · 2026-07-24T12:52:23Z

Did you write the PR description using AI? I'm not sure you did, but if yes, please note that our AI policy forbids doing that.

Yes I used Claude for draft message but then I edit the message.

ChayimFriedman2 · 2026-07-24T12:58:16Z

So, this is also disallowed. Please take notice for the next time.

kivancgnlp · 2026-07-24T13:10:40Z

Looking at the code, it's pretty clear that you used AI for the code as well without disclosing it, also contrary to our AI policy. And I'm not sure it's properly reviewed either: I am pretty sure it's possible to implement the same thing using at most half the lines, and probably with better perf.

Claude generated the initial code and tests. I reviewed the changes and tests by myself. I'm sorry that I discovered the policy afterward. I also evaluated possible pitfalls and alternative solutions. I’ll simplify the code.

kivancgnlp · 2026-07-26T06:20:54Z

Implementation simplified
Hover documentation was tested using a local rust-analyzer server and VS Code.
The following behaviors were checked and verified: stars disappeared, formatting remained correct, and the documentation link worked.

kivancgnlp · 2026-07-29T06:55:43Z

Hi @ChayimFriedman2, just checking in on this one. As noted above, I simplified the implementation and removed the AI-generated parts entirely. The current version reflects my own review and manual testing (verified with a local rust-analyzer server and VS Code, doc-comment stars are stripped correctly, formatting stays intact, and the documentation link still resolves).

Happy to make further changes if the simplified approach still isn't the right shape. Let me know what you'd like adjusted.

ChayimFriedman2

I don't want us to reinvent the wheel. The code to do that in rustc is here:

https://github.com/rust-lang/rust/blob/16a623ad672a92409b5c04beb303583c6cf72a7e/compiler/rustc_resolve/src/rustdoc.rs#L210

The beautify_doc_string() does this and perhaps more things. We can copy it almost as-is, perhaps with minor changes.

View changes since this review

rustbot · 2026-08-26T03:55:25Z

This PR was rebased onto a different master commit. Here's a range-diff highlighting what actually changed.

Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers.

kivancgnlp · 2026-08-26T04:26:26Z

Thanks. Adopted beautify_doc_string in 877fbfa (on top of a rebase onto current master). Kept it as a separate commit so the diff against the original approach is easy to see.

A few notes on the adaptation:

The changes live in compute_block_doc_trim and return a BlockDocTrim { keep, horizontal } value instead of interning a new Symbol. That way push_doc_lines can still compute per-line source-map offsets, lines dropped by the vertical trim keep advancing ast_offset by source_len + "\n", so surviving lines map back to their true source bytes. Added a citation to the rustc revision (16a623ad) in the doc comment.
Restricted the update to CommentKind::Block. Enabling the Line branch (which strips a uniform leading-space prefix from /// comments) would change hover rendering more broadly than the issue calls for. Happy to do that in a follow-up PR if you'd like, but wanted to keep this one scoped to Doc comment block should strip leading asterisk (*) #1759.
Dropped my earlier block_star_prefix; the horizontal-trim logic in the patch supersedes it and also handles the * \t prefix cases my version rejected.
One test case I initially wrote to exercise the vertical trim (/**\n****\n * foo\n * bar\n****\n */) doesn't actually hit that path, after doc_comment() strips the /** prefix, the fence lines aren't at index 0 or N-1, so rustc's get_vertical_trim wouldn't fire on it either. Left the vertical-trim code in place for spec fidelity but didn't ship a test that can't reach it. Let me know if you'd prefer I remove that branch as dead code in this crate's usage.

All the tests from the PR body still pass (cargo test -p hir-def, cargo test -p ide -- hover doc_links, cargo test -p ide -- syntax_highlighting), plus cargo fmt --check and cargo clippy are clean.

ChayimFriedman2

This is more complicated that it needs to be. get_vertical_trim() and get_horizontal_trim() you can copy verbatim. Current push_doc_lines() should be renamed to push_doc_line() and handle only one line, and a new push_doc_lines() should be created, that is equivalent to rustc's beautify_doc_string() except that if !data_s.contains('\n') it just calls push_doc_line(), and if not, instead of collecting into Vec<&str> it'll collect into Vec<(&str, TextSize)> where the TextSize is the offset from the string start (to get it you can use data_s.substr_range(line).unwrap().start). Then at the end instead of join("\n") you iterate over lines and call push_doc_line().

View changes since this review

kivancgnlp · 2026-08-26T16:23:25Z

Refactored in b393f3d along the shape you sketched:

push_doc_lines is now the single-line pusher, renamed to push_doc_line.
New push_doc_lines mirrors beautify_doc_string: !doc.contains('\n') calls push_doc_line directly; otherwise it builds Vec<(&str, TextSize)>, runs get_vertical_trim / get_horizontal_trim, applies the horizontal strip (plus the additional * strip when it's block decoration), then iterates the survivors and calls push_doc_line(line, ast_offset + line_offset, indent).
The trim helpers are near-verbatim copies of rustc's. Two adaptations I want to flag:
- Signatures take &[(&str, TextSize)] instead of &[&str], the offset is ignored inside the helpers, but keeping it paired avoids projecting a temporary Vec<&str> at each call site. Happy to switch to &[&str] + a projection if you'd prefer stricter parity with upstream; it makes future copy-paste updates from rustc_ast trivially mechanical at the cost of one small allocation per call.
- get_vertical_trim gets one extra guard: the empty-first-line case is skipped. rustc feeds this helper the output of data_s.lines(); I feed it doc.split('\n') so I can track per-line byte offsets, and unlike .lines(), .split('\n') can produce a leading empty entry that chars().all(...) matches vacuously. The deviation is called out inline.
Byte offsets are computed manually with a running cursor rather than str::substr_range, since that's not stable at the workspace's MSRV (1.95).

One test expectation moved (block_doc_comment_stars). For the /**\n * foo\n *\n * * bullet\n *bar\n */ input, my earlier block_star_prefix version left " *bar" intact when the line didn't match the * / ** / * guard, which pinned the block's min indent at 1 and let downstream indent-stripping shift the surrounding lines left. Rustc's rule strips the horizontal " " from every line unconditionally, so *bar becomes *bar (no leading space), min indent drops to 0, and the neighbouring lines keep their leading whitespace. That matches rustdoc's rendering for the same input, which is what we want. Comment on the test spells this out.

All the tests from the PR body still pass (cargo test -p hir-def, cargo test -p ide -- hover doc_links, cargo test -p ide -- syntax_highlighting) plus cargo fmt --check and cargo clippy are clean.

ChayimFriedman2

Please also squash.

View changes since this review

ChayimFriedman2 · 2026-08-26T16:30:27Z

+        // Each entry is `(line, offset_from_doc_start)`. The offset stays in sync with the
+        // string as we strip its prefix, so the caller can add it to `ast_offset` for the
+        // source map. Computed manually rather than via `str::substr_range` to stay compatible
+        // with the workspace's MSRV.


We can upgrade the MSRV (this is quite new and we prefer not to do that so quickly but I prefer that over duplicating the code).

ChayimFriedman2 · 2026-08-26T16:31:26Z

+        // with the workspace's MSRV.
+        let mut lines: Vec<(&str, TextSize)> = Vec::new();
+        let mut cursor = TextSize::new(0);
        for line in doc.split('\n') {


And crucially, rustc uses lines() which has a different behavior wrt. \r, which is why I said to use substr_range().

ChayimFriedman2 · 2026-08-26T16:32:40Z

+        doc: &str,
+        ast_offset: Option<TextSize>,
+        indent: &mut usize,
+        shape: Option<ast::CommentShape>,


rustc passes CommentKind::Line for desugared comments, please do the same here.

kivancgnlp · 2026-08-26T16:55:51Z

Addressed the three inline comments plus the squash in 7f44856 (rebase-amended into the single squashed commit).

MSRV: bumped rust-version from 1.95 to 1.96 in the workspace Cargo.toml. Called out in the commit message so it's visible in the log.
.lines() + substr_range: switched from the manual .split('\n') + running-cursor loop to doc.lines() with doc.substr_range(line).unwrap().start for the byte offset, matches rustc exactly, including \r\n handling.
CommentShape::Line for desugared docs: push_doc_lines no longer takes Option<CommentShape>. extend_with_doc_str (doc attributes) and extend_with_unmapped_doc_str (macro-expanded docs) now pass CommentShape::Line, matching rustc's CommentKind::Line for those cases. Inline comment cites the parity.
get_vertical_trim / get_horizontal_trim are now byte-for-byte with rustc, taking &[&str]. The caller projects a temporary Vec<&str> view around the Vec<(&str, TextSize)> before each helper call. The !line.is_empty() guard I'd added on the first-line check in the previous round is gone, no longer needed with .lines().
Test expectations moved: .lines() no longer emits a leading empty entry for the \n right after /**, so block_doc_comment_source_map now expects "foo\nbar\n" (offsets shift by 1) and block_doc_comment_stars drops the leading blank row. Both match rustdoc's actual rendering more faithfully than the previous port did. Comments on the tests explain why.
Squashed the whole thing into a single commit and force-pushed.

Local checks: cargo test -p hir-def (502 pass), cargo test -p ide -- hover doc_links (308 pass), cargo test -p ide -- syntax_highlighting (38 pass), cargo fmt --check, cargo clippy, and RUSTDOCFLAGS="-D warnings" cargo doc -p hir-def --no-deps --document-private-items all clean.

ChayimFriedman2

Some final nits (and remember to squash).

View changes since this review

ChayimFriedman2 · 2026-08-26T16:58:59Z

+        // Desugared `#[doc = "..."]` strings and macro-expanded docs behave like line comments
+        // in rustc's `beautify_doc_string`, so pass `CommentShape::Line` here (matches rustc
+        // passing `CommentKind::Line` for the desugared case).
+        self.push_doc_lines(doc, Some(offset_in_ast), indent, ast::CommentShape::Line);


This should take the CommentShape as a parameter.

ChayimFriedman2 · 2026-08-26T16:59:22Z

        let Some((doc, offset)) = comment.doc_comment() else { return };
-        self.extend_with_doc_str(doc, comment.syntax().text_range().start() + offset, indent);
+        let offset = comment.syntax().text_range().start() + offset;
+        self.push_doc_lines(doc, Some(offset), indent, comment.kind().shape);


And then pass it here, avoiding the switch to push_doc_lines().

ChayimFriedman2 · 2026-08-26T16:59:54Z

+        // Use `str::lines()` (matching rustc) so `\r\n` line endings behave correctly, and
+        // `str::substr_range` (matching rustc) to recover each line's byte offset from `doc`'s
+        // start. Both were stabilized by our workspace MSRV.


Redundant comment, please remove.

Adopts rustc's beautify_doc_string algorithm for hir-def's docs gathering so block doc comments render like rustdoc does. Splits the old push_doc_lines into a per-line push_doc_line plus a new push_doc_lines that mirrors beautify_doc_string: single-line input takes a fast path; multi-line input builds Vec<(&str, TextSize)> using str::lines() (matching rustc, correct \r\n handling), runs get_vertical_trim / get_horizontal_trim on a projected &[&str] view, strips the horizontal prefix (and an additional leading '*' when it's block decoration), then pushes each surviving line via push_doc_line so the source-map offsets stay accurate. get_vertical_trim and get_horizontal_trim are byte-for-byte copies of rustc's helpers, modulo the CommentKind -> CommentShape rename and returning String rather than interning to Symbol. Per-line byte offsets are computed by pointer arithmetic instead of str::substr_range because substr_range is stable since 1.98 and the workspace MSRV is 1.95. This matches what substr_range does internally. Doc attributes and macro-expanded doc strings route through push_doc_lines with CommentShape::Line, matching rustc which passes CommentKind::Line for those desugared cases. Adds hover tests covering block comments decorated with leading asterisks (with and without leading/trailing framing).

ChayimFriedman2

Thanks!

View changes since this review

kivancgnlp · 2026-08-26T17:17:24Z

Two things landed on top of 7f44856965, squashed into 93d98164a:

CI failures on the MSRV bump. Reverted to rust-version = "1.95".

Clippy failed with incompatible_msrv. I had the stability wrong: str::substr_range is stable since 1.98, not 1.96. My local rustc 1.98.0 didn't flag it, but clippy reads the declared workspace MSRV and correctly errored. Sorry for the wrong number.
Miri failed with rustc 1.95.0-nightly is not supported, the miri job pins a nightly whose version prefix is 1.95.0-nightly, so any MSRV bump breaks it until the toolchain pin is bumped in a separate PR.

Given "prefer not to do that so quickly," I don't think escalating to a 1.98 bump plus a miri-toolchain bump is right for this PR. Per-line offsets are now computed by pointer arithmetic (line.as_ptr() as usize - doc.as_ptr() as usize), same result as substr_range, no MSRV cost. .lines() and correct \r\n handling stay.

Nits. extend_with_doc_str now takes CommentShape, and extend_with_doc_comment passes comment.kind().shape through it instead of calling push_doc_lines directly. extend_with_doc_attr also routes through extend_with_doc_str with CommentShape::Line. Removed the redundant comment on the .lines() block.

Single commit, all tests + fmt + clippy + rustdoc green locally.

ChayimFriedman2 · 2026-08-26T17:19:42Z

I know it was stable in 1.98 (in fact I didn't realize that's not what you said). We keep policy of latest-stable MSRV, but we try to not update so quickly. However like I said, here I think it's justified.

The Miri CI should be unpinned anyway.

kivancgnlp · 2026-08-26T17:27:31Z

Thanks for the context, good to know. Happy to leave this on the pointer-arithmetic version so the PR doesn't take a dependency on the MSRV policy or the miri pin moving.

ChayimFriedman2 · 2026-08-26T17:42:31Z

I did that myself, #23239.

We suspect you're writing your comments via an AI. Please note that per our AI policy, this is forbidden, and AI usage must be disclosed.

boozook · 2026-08-26T21:53:14Z

How to disable this breaking behavior?
Relative: #1759 (comment)

ChayimFriedman2 · 2026-08-26T22:15:43Z

@boozook You can't. This is a bugfix and we won't reenable spacebar heating (reference).

boozook · 2026-08-26T22:18:59Z

This isn't bugfix. This is rust spec disrespect and an error. It could be fix it is java.

ChayimFriedman2 · 2026-08-26T22:23:06Z

The Rust spec is the implementation. The Reference is explicitly non-normative. In this case, the implementation is rustdoc.

We're not going to argue over it. If you can convince the rustdoc teams to change their behavior, we will follow suit. Until then, discussion here is not useful.

ChayimFriedman2 · 2026-08-26T22:25:03Z

Also, please remember that this project is open source and developed by volunteers. Even if you disagree with the maintainers' decisions, respect is warranted. And of course, remember to always follow the Code of Conduct.

rustbot added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Jul 24, 2026

This comment has been minimized.

Sign in to view

kivancgnlp force-pushed the fix-block-doc-asterisks branch 2 times, most recently from 42dc49c to de72f8f Compare July 26, 2026 05:21

rami3l mentioned this pull request Jul 29, 2026

document visual studio 2026 rust-lang/rustup#4858

Open

ChayimFriedman2 requested changes Aug 26, 2026

View reviewed changes

kivancgnlp force-pushed the fix-block-doc-asterisks branch from de72f8f to 95bd69d Compare August 26, 2026 03:55

This comment has been minimized.

Sign in to view

ChayimFriedman2 requested changes Aug 26, 2026

View reviewed changes

kivancgnlp force-pushed the fix-block-doc-asterisks branch 2 times, most recently from b3bd418 to 7f44856 Compare August 26, 2026 16:52

ChayimFriedman2 requested changes Aug 26, 2026

View reviewed changes

kivancgnlp force-pushed the fix-block-doc-asterisks branch from 7f44856 to 6956adb Compare August 26, 2026 17:06

kivancgnlp force-pushed the fix-block-doc-asterisks branch from 6956adb to 93d9816 Compare August 26, 2026 17:12

ChayimFriedman2 approved these changes Aug 26, 2026

View reviewed changes

ChayimFriedman2 enabled auto-merge August 26, 2026 17:17

ChayimFriedman2 added this pull request to the merge queue Aug 26, 2026

Merged via the queue into rust-lang:master with commit 919d6c2 Aug 26, 2026
18 checks passed

rustbot removed the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Aug 26, 2026

This was referenced Aug 26, 2026

internal: Represent doc comments using their own node and not as COMMENT trivia #23218

Merged

fix: Fix some subtle bugs in docs rendering #23235

Merged

Uh oh!

Uh oh!

Conversation

kivancgnlp commented Jul 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Tests Passed :

Uh oh!

This comment has been minimized.

ChayimFriedman2 commented Jul 24, 2026

Uh oh!

ChayimFriedman2 commented Jul 24, 2026

Uh oh!

kivancgnlp commented Jul 24, 2026

Uh oh!

ChayimFriedman2 commented Jul 24, 2026

Uh oh!

kivancgnlp commented Jul 24, 2026

Uh oh!

kivancgnlp commented Jul 26, 2026

Uh oh!

kivancgnlp commented Jul 29, 2026

Uh oh!

ChayimFriedman2 left a comment • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rustbot commented Aug 26, 2026

Uh oh!

This comment has been minimized.

kivancgnlp commented Aug 26, 2026

Uh oh!

ChayimFriedman2 left a comment • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kivancgnlp commented Aug 26, 2026

Uh oh!

ChayimFriedman2 left a comment • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ChayimFriedman2 Aug 26, 2026

Choose a reason for hiding this comment

Uh oh!

ChayimFriedman2 Aug 26, 2026

Choose a reason for hiding this comment

Uh oh!

ChayimFriedman2 Aug 26, 2026

Choose a reason for hiding this comment

Uh oh!

kivancgnlp commented Aug 26, 2026

Uh oh!

ChayimFriedman2 left a comment • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ChayimFriedman2 Aug 26, 2026

Choose a reason for hiding this comment

Uh oh!

ChayimFriedman2 Aug 26, 2026

Choose a reason for hiding this comment

Uh oh!

ChayimFriedman2 Aug 26, 2026

Choose a reason for hiding this comment

Uh oh!

ChayimFriedman2 left a comment • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kivancgnlp commented Aug 26, 2026

Uh oh!

ChayimFriedman2 commented Aug 26, 2026

Uh oh!

kivancgnlp commented Aug 26, 2026

Uh oh!

Uh oh!

ChayimFriedman2 commented Aug 26, 2026

Uh oh!

boozook commented Aug 26, 2026

kivancgnlp commented Jul 24, 2026 •

edited

Loading

ChayimFriedman2 left a comment •

edited by rustbot

Loading

ChayimFriedman2 left a comment •

edited by rustbot

Loading

ChayimFriedman2 left a comment •

edited by rustbot

Loading

ChayimFriedman2 left a comment •

edited by rustbot

Loading

ChayimFriedman2 left a comment •

edited by rustbot

Loading

ChayimFriedman2 commented Aug 26, 2026 •

edited

Loading