Skip to content

coverage: Anonymize line numbers in run-coverage test snapshots - #114875

Merged
bors merged 1 commit into
rust-lang:masterfrom
Zalathar:line-numbers
Aug 17, 2023
Merged

coverage: Anonymize line numbers in run-coverage test snapshots#114875
bors merged 1 commit into
rust-lang:masterfrom
Zalathar:line-numbers

Conversation

@Zalathar

@ZalatharZalathar commented Aug 16, 2023•
edited
Loading

Copy link
Copy Markdown
Member

LLVM's coverage reporter always prints line numbers in its coverage reports.

For testing purposes this is slightly inconvenient, because it means that adding or removing a line in a test file causes all subsequent lines in the snapshot to change. That makes it harder to see the actually meaningful changes in the re-blessed snapshot.

This change fixes that by adding another normalization pass that replaces all line numbers in the coverage reports with LL, which is similar to what UI tests tell the compiler to do when emitting line numbers in error messages.

@rustbot

rustbot commented Aug 16, 2023

Copy link
Copy Markdown
Collaborator

r? @ozkanonur

(rustbot has picked a reviewer for you, use r? to override)

@rustbotrustbot added A-testsuite Area: The testsuite used to check the correctness of rustc S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) A-code-coverage Area: Source-based code coverage (-Cinstrument-coverage) labels Aug 16, 2023
Zalathar
Zalathar commented Aug 16, 2023
View reviewed changes
Comment threadsrc/tools/compiletest/src/runtest.rs Outdated
@Zalathar
Zalatharforce-pushed the line-numbers branch 2 times, most recently from 9ec72a0 to 89155a8CompareAugust 16, 2023 04:30
@onur-ozkan

onur-ozkan commented Aug 16, 2023

Copy link
Copy Markdown
Contributor

Looks good to me, thanks!

rollup=never for smoother rollbacks in case of any regressions or issues.

@bors r+

@bors

bors commented Aug 16, 2023

Copy link
Copy Markdown
Collaborator

📌 Commit 89155a84b9a17413e14b7c2f121629405670adf2 has been approved by ozkanonur

It is now in the queue for this repository.

@borsbors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Aug 16, 2023
@onur-ozkan

onur-ozkan commented Aug 16, 2023

Copy link
Copy Markdown
Contributor

@bors rollup=never

@bors

bors commented Aug 16, 2023

Copy link
Copy Markdown
Collaborator

⌛ Testing commit 89155a84b9a17413e14b7c2f121629405670adf2 with merge af6256d2309a27e05df159fef82a8b236d083870...

@rust-log-analyzer

This comment has been minimized.

Sign in to view
@bors

bors commented Aug 16, 2023

Copy link
Copy Markdown
Collaborator

💔 Test failed - checks-actions

@borsbors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Aug 16, 2023
@onur-ozkan

onur-ozkan commented Aug 16, 2023

Copy link
Copy Markdown
Contributor

@rustbot author

@rustbotrustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Aug 16, 2023
@Zalathar

Zalathar commented Aug 16, 2023

Copy link
Copy Markdown
MemberAuthor

Ah, I forgot to update the run-coverage-rustdoc suite as well, since I rarely need to run it. Fixed.

@rustbot ready

@rustbotrustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Aug 16, 2023
@Zalathar
coverage: Anonymize line numbers in run-coverage test snapshots
bfb1654
This makes the test snapshots less sensitive to lines being added/removed.
@ZalatharZalathar mentioned this pull request Aug 17, 2023
Draft
@onur-ozkan

onur-ozkan commented Aug 17, 2023

Copy link
Copy Markdown
Contributor

@bors retry

@borsbors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Aug 17, 2023
@onur-ozkan

onur-ozkan commented Aug 17, 2023

Copy link
Copy Markdown
Contributor

@bors retry

Didn't work, strange..

@bors r+

@bors

bors commented Aug 17, 2023

Copy link
Copy Markdown
Collaborator

📌 Commit bfb1654 has been approved by ozkanonur

It is now in the queue for this repository.

@bors

bors commented Aug 17, 2023

Copy link
Copy Markdown
Collaborator

⌛ Testing commit bfb1654 with merge aa864a7...

@bors

bors commented Aug 17, 2023

Copy link
Copy Markdown
Collaborator

☀️ Test successful - checks-actions
Approved by: ozkanonur
Pushing aa864a7 to master...

@borsbors added the merged-by-bors This PR was explicitly merged by bors. label Aug 17, 2023
@bors
bors merged commit aa864a7 into rust-lang:masterAug 17, 2023
@rustbotrustbot added this to the 1.73.0 milestone Aug 17, 2023
@Zalathar
Zalathar deleted the line-numbers branch August 17, 2023 07:22
@ZalatharZalathar mentioned this pull request Aug 17, 2023
Merged
@rust-timer

rust-timer commented Aug 17, 2023

Copy link
Copy Markdown
Collaborator

Finished benchmarking commit (aa864a7): comparison URL.

Overall result: no relevant changes - no action needed

@rustbot label: -perf-regression

Instruction count

This benchmark run did not return any relevant results for this metric.

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

meanrangecount
Regressions ❌
(primary)		2.2%[2.2%, 2.2%]1
Regressions ❌
(secondary)		2.2%[2.2%, 2.2%]1
Improvements ✅
(primary)		--0
Improvements ✅
(secondary)		-1.1%[-1.1%, -1.1%]1
All ❌✅ (primary)2.2%[2.2%, 2.2%]1

Cycles

This benchmark run did not return any relevant results for this metric.

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 634.855s -> 634.832s (-0.00%)
Artifact size: 346.99 MiB -> 347.01 MiB (0.01%)

Zalathar added a commit to Zalathar/rust that referenced this pull request Aug 26, 2023
@Zalathar
Tidy up some awkwardly-placed comments in tests
4da38c3
Prior to rust-lang#114875, these tests were very sensitive to lines being added/removed,
so the migration to `run-coverage` in rust-lang#112300 tried hard to avoid disturbing
the existing line numbers. That resulted in some awkward reshuffling when
certain comments/directives needed to be added or moved.
Now that we don't have to worry about preserving line numbers, we can rearrange
those comments into a more conventional layout.
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this pull request Aug 27, 2023
@matthiaskrgr
Rollup merge ofrust-lang#114924- Zalathar:tidy-tests, r=Mark-Simula…
be416b9
…crum
coverage: Tidy up `run-coverage` tests in several small ways
Prior to rust-lang#114875 (anonymized line numbers), these tests were very sensitive to lines being added/removed, since doing so would change the line numbers in every subsequent line of output. Therefore they ended up with a few small style issues that weren't worth the hassle of fixing.
Now that we can freely rearrange lines without needing to re-bless the entire output, we can tidy up the tests in various trivial ways.
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

@onur-ozkanonur-ozkan

Labels

A-code-coverageArea: Source-based code coverage (-Cinstrument-coverage)A-testsuiteArea: The testsuite used to check the correctness of rustcmerged-by-borsThis PR was explicitly merged by bors.S-waiting-on-borsStatus: Waiting on bors to run and complete tests. Bors will change the label on completion.T-bootstrapRelevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)

Projects

None yet

Milestone

1.73.0

Development

Successfully merging this pull request may close these issues.

6 participants

@Zalathar@rustbot@onur-ozkan@bors@rust-log-analyzer@rust-timer