Uh oh!
There was an error while loading. Please reload this page.
Add post-merge analysis CI workflow - #138013
Merged
Merged
Conversation
rustbot
commented
Mar 4, 2025
Collaborator
|
This comment has been minimized.
This comment has been minimized.
bors
commented
Mar 5, 2025
Collaborator
☔ The latest upstream changes (presumably #138021) made this pull request unmergeable. Please resolve the merge conflicts. |
Kobzolforce-pushed
the
ci-post-merge-analysis
branch
from
March 5, 2025 07:34
c40b798 to
6ec42d7CompareKobzol
marked this pull request as ready for review
March 5, 2025 07:41
Kobzol
commented
Mar 5, 2025
MemberAuthor
Rebased. @rustbot ready |
This comment has been minimized.
This comment has been minimized.
bors
commented
Mar 5, 2025
Collaborator
☔ The latest upstream changes (presumably #138058) made this pull request unmergeable. Please resolve the merge conflicts. |
Kobzolforce-pushed
the
ci-post-merge-analysis
branch
from
March 7, 2025 08:25
6ec42d7 to
f04a2ccCompareKobzol
commented
Mar 7, 2025
MemberAuthor
Rebased. |
Kobzolforce-pushed
the
ci-post-merge-analysis
branch
from
March 7, 2025 08:43
f04a2cc to
d5da6b7Compare
marcoieni
left a comment
Member
There was a problem hiding this comment.
very small details that don't matter, I'll approve this PR as it is now
| for job in &job_db.auto_jobs { | ||
| eprintln!("Downloading metrics of job {}", job.name); | ||
| let metrics_parent = match download_job_metrics(&job.name, parent) { |
Member
There was a problem hiding this comment.
you could simplify this match with something like this:download_job_metrics(&job.name, parent).inspect_err( eprintln!()).ok()
Comment on lines
+41
to
+43
| r#"Did not find metrics for job `{}` at `{}`: {error:?}. | ||
| Maybe it was newly added?"#, | ||
| job.name, parent |
Member
There was a problem hiding this comment.
Suggested change
| r#"Did not find metrics for job `{}` at `{}`:{error:?}. | |
| Maybe it was newly added?"#, | |
| job.name, parent | |
| r#"Did not find metrics forjob `{}` at `{parent}`:{error:?}. | |
| Maybe it was newly added?"#, | |
| job.name |
marcoieni
commented
Mar 7, 2025
Member
@bors r+ |
bors
commented
Mar 7, 2025
Collaborator
bors added a commit
to rust-lang-ci/rust
that referenced
this pull request
Mar 8, 2025
Rollup of 12 pull requests Successful merges: - rust-lang#137337 (Add verbatim linker to AIXLinker) - rust-lang#137363 (compiler: factor Windows x86-32 ABI impl into its own file) - rust-lang#137537 (Prevent `rmake.rs` from using unstable features, and fix 3 run-make tests that currently do) - rust-lang#137606 (add a "future" edition) - rust-lang#137957 (Remove i586-pc-windows-msvc) - rust-lang#138000 (atomic: clarify that failing conditional RMW operations are not 'writes') - rust-lang#138013 (Add post-merge analysis CI workflow) - rust-lang#138033 (rustdoc: Add attribute-related tests for rustdoc JSON.) - rust-lang#138137 (setTargetTriple now accepts Triple rather than string) - rust-lang#138173 (Delay bug for negative auto trait rather than ICEing) - rust-lang#138184 (Allow anyone to relabel `CI-spurious-*`) - rust-lang#138187 (remove clones) r? `@ghost` `@rustbot` modify labels: rollup
rust-timer added a commit
to rust-lang-ci/rust
that referenced
this pull request
Mar 8, 2025
Rollup merge of rust-lang#138013 - Kobzol:ci-post-merge-analysis, r=marcoieni Add post-merge analysis CI workflow This PR adds a post-merge analysis workflow, which was discussed [here](https://rust-lang.zulipchat.com/#narrow/channel/242791-t-infra/topic/Reporting.20test.20suite.20statistics.20after.20merge). The workflow currently analyzes test suite results from bootstrap metrics. It downloads metrics for all known jobs in the parent and current (HEAD) commit, compares them and prints a truncated diff. It then posts this diff to the merged PR as a comment. Later I also want to add other statistics to the analysis, e.g. changes in CI/bootstrap step durations. It can be tested locally e.g. using this: ``` cargo run --release --manifest-path src/ci/citool/Cargo.toml post-merge-report 3cb0272fd17dea ``` This uses a slightly older commit as a parent, to have more results in the diff (normally the diff won't be so large). CC `@jieyouxu` r? `@marcoieni`
klensy
reviewed
Mar 13, 2025
Uh oh!
There was an error while loading. Please reload this page.
jhpratt added a commit
to jhpratt/rust
that referenced
this pull request
Mar 14, 2025
…r=jieyouxu Improve post-merge workflow Contains various fixes for the post-merge workflow implemented in rust-lang#138013, which were suggested on Zulip. This PR changes the grouping of test diffs and ignores doctests, as they are too noisy. I'll post an example output (before/after this PR) in comments below. r? `@jieyouxu`
fmease added a commit
to fmease/rust
that referenced
this pull request
Mar 14, 2025
…r=jieyouxu Improve post-merge workflow Contains various fixes for the post-merge workflow implemented in rust-lang#138013, which were suggested on Zulip. This PR changes the grouping of test diffs and ignores doctests, as they are too noisy. I'll post an example output (before/after this PR) in comments below. r? ``@jieyouxu``
fmease added a commit
to fmease/rust
that referenced
this pull request
Mar 14, 2025
…r=jieyouxu Improve post-merge workflow Contains various fixes for the post-merge workflow implemented in rust-lang#138013, which were suggested on Zulip. This PR changes the grouping of test diffs and ignores doctests, as they are too noisy. I'll post an example output (before/after this PR) in comments below. r? ```@jieyouxu```
rust-timer added a commit
to rust-lang-ci/rust
that referenced
this pull request
Mar 15, 2025
Rollup merge of rust-lang#138454 - Kobzol:post-merge-workflow-fixes, r=jieyouxu Improve post-merge workflow Contains various fixes for the post-merge workflow implemented in rust-lang#138013, which were suggested on Zulip. This PR changes the grouping of test diffs and ignores doctests, as they are too noisy. I'll post an example output (before/after this PR) in comments below. r? ```@jieyouxu```
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds a post-merge analysis workflow, which was discussed here.
The workflow currently analyzes test suite results from bootstrap metrics. It downloads metrics for all known jobs in the parent and current (HEAD) commit, compares them and prints a truncated diff. It then posts this diff to the merged PR as a comment. Later I also want to add other statistics to the analysis, e.g. changes in CI/bootstrap step durations.
It can be tested locally e.g. using this:
This uses a slightly older commit as a parent, to have more results in the diff (normally the diff won't be so large).
CC @jieyouxu
r? @marcoieni