Record failed tests with --record, and rerun them with --rerun - #154586

Merged
rust-bors[bot] merged 3 commits into
rust-lang:mainfrom
jdonszelmann:record-rerun
Jun 4, 2026
Merged

Record failed tests with --record, and rerun them with --rerun#154586
rust-bors[bot] merged 3 commits into
rust-lang:mainfrom
jdonszelmann:record-rerun

Conversation

@jdonszelmann

@jdonszelmannjdonszelmann commented Mar 30, 2026

Copy link
Copy Markdown
Contributor

View all comments

This adds two parameters to x test:

--record

Writes a file, by default build/failed-tests, but this can be overwritten with

[build]
record_failed_tests_path = "somepath"

with a list of all tests that fail that run.

--rerun

Looks for the failed-tests file, parse it, and attempt to rerun only those tests. No cli-arguments are necessary, i.e.

x test tests/ui --record
x test --rerun

Will run all failed uitests. No need to pass tests/ui to the rerun invocation.

The last commit is a little awkward, but I think it's the best way to make it so that we first run all tests that have to be rerun, and then rerun tests passed through the cli.

This makes it so:

x test tests/ui --rerun

will first rerun failed tests, some of which may be uitests, if any fail it quits and reports failed tests, but if all pass it will run all normally passed tests. In other words, only if all previously-failed tests pass on the rerun, we then also run uitests.
Without the last commit, this would instead just run all uitests, since the failed tests form a subset of all uitests. I think that's less useful.

@rustbot

Copy link
Copy Markdown
Collaborator

This PR modifies src/bootstrap/src/core/config.

If appropriate, please update CONFIG_CHANGE_HISTORY in src/bootstrap/src/utils/change_tracker.rs.

@rustbotrustbot added A-testsuite Area: The testsuite used to check the correctness of rustc S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) labels Mar 30, 2026
@rustbot

Copy link
Copy Markdown
Collaborator

r? @clubby789

rustbot has assigned @clubby789.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

Why was this reviewer chosen?

The reviewer was selected based on:

  • Owners of files modified in this PR: bootstrap
  • bootstrap expanded to 6 candidates
  • Random selection from Mark-Simulacrum, clubby789, jieyouxu

@rust-log-analyzer

This comment has been minimized.

@jyn514

Copy link
Copy Markdown
Member

Writes a file, by default build/failed-tests, but this can be overwritten with

I wouldn't make this configurable.

What does x test --rerun do if there was no previous --record run? How does cache invalidation work for the failed-tests file, do you just keep it around forever?

@jdonszelmann

Copy link
Copy Markdown
ContributorAuthor

@jyn514

What does x test --rerun do if there was no previous --record run?

It will warn, but treat it as if the file was empty.

How does cache invalidation work for the failed-tests file, do you just keep it around forever?

Keeps it around forever, or at least until you --record again. It prints that it's rerunning, and from what file so it should be easy to delete or even edit manually. Rerunning acts pretty much as if you passed the test paths on the cli, though we do run them before the cli-passed paths.

@rustbot

This comment has been minimized.

@rust-log-analyzer

This comment has been minimized.

@rust-log-analyzer

This comment has been minimized.

@rust-log-analyzer

This comment has been minimized.

@WaffleLapkin

Copy link
Copy Markdown
Member

Rerunning acts pretty much as if you passed the test paths on the cli, though we do run them before the cli-passed paths.

If the build/failed-tests is empty/non-present, does that make x test --rerun run all tests?

@WaffleLapkin

Copy link
Copy Markdown
Member

This is purely a speculation I suppose, but I feel like I will often forget to use --record, before the tests already failed. Would it make sense to save failures of the last command to a separate file and allow moving them to build/failed-tests?

@jyn514

Copy link
Copy Markdown
Member

@WaffleLapkin as long as you don’t modify the compiler, compiletest will remember which tests succeeded the last time, so if you add ––failed it won’t take very long at all to regenerate that list.

@jdonszelmann

Copy link
Copy Markdown
ContributorAuthor

I agree with jyn here, shouldn't make such a big difference

@jdonszelmann

Copy link
Copy Markdown
ContributorAuthor

Rerunning acts pretty much as if you passed the test paths on the cli, though we do run them before the cli-passed paths.

If the build/failed-tests is empty/non-present, does that make x test --rerun run all tests?

that is true I think. What do you expect the behavior to be? maybe if no file is found, and no paths are given, exit, but if some paths are explicitly given run the explicit ones?

@jdonszelmann

Copy link
Copy Markdown
ContributorAuthor

with a warning of course

@jyn514

Copy link
Copy Markdown
Member

I think x test --rerun with no previous --record should behave exactly the same as x test.

@jdonszelmann

Copy link
Copy Markdown
ContributorAuthor

mhm, well that's the current behavior. Except the warning of course, that you passed --rerun with nothing to rerun. But I think that's nice

@jdonszelmann

Copy link
Copy Markdown
ContributorAuthor

cc @jieyouxu (you self assigned the other one, that one was in preparation for this one, also I figured out that bug for this one)

@clubby789

Copy link
Copy Markdown
Contributor

I haven't looked at the full implementation yet, but why not always record failed tests?

@jyn514

jyn514 commented Mar 31, 2026

Copy link
Copy Markdown
Member

Because a later invocation might only rerun a subset of tests. Say you have this series of invocations:

$ x test ui
# ... 34 failures in ui/linkage and ui/attributes
$ x build library # assume there was a compiler change
$ x test --rerun tests/ui/attributes
# ... all tests now pass
$ x test --rerun tests/ui/linkage
# ... no tests are run :( we expected all the failed linkage tests to rerun.

@clubby789

Copy link
Copy Markdown
Contributor

Sorry, always *except when using --rerun 😅

@jieyouxujieyouxu self-assigned this Mar 31, 2026
@rustbotrustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Apr 20, 2026
@rustbot

This comment has been minimized.

@rust-log-analyzer

This comment has been minimized.

@rustbotrustbot removed the S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. label May 12, 2026
@jieyouxujieyouxu reopened this May 12, 2026
@rustbotrustbot added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label May 12, 2026

@jieyouxujieyouxu left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, you can r= clubby and me with one message typo fixed
@rustbot author

View changes since this review

Ok(f) => Some(f),
Err(e) => {
println!(
"Couldn't open file {} to write test failutes to: {e}. (attempted because `--record` was passed). Test failures will not be recorded.",

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: s/failutes/failures/.

@rustbotrustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels May 14, 2026
@jieyouxu

Copy link
Copy Markdown
Member

I thought about #154586 (comment) a bit more, while the parsing is still a bit hacky, it's not that bad since this is more of convenience feature and not correctness-critical (as in, CI will still catch unaddressed failures even if --{record,rerun} isn't fully accurate or drifts).

@rustbot

Copy link
Copy Markdown
Collaborator

This PR was rebased onto a different main commit. Here's a range-diff highlighting what actually changed.

Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers.

@jdonszelmann

Copy link
Copy Markdown
ContributorAuthor

@bors r=clubby789,jieyouxu

@rust-bors

rust-borsBot commented Jun 4, 2026

Copy link
Copy Markdown
Contributor

📌 Commit ab1aa4b has been approved by clubby789,jieyouxu

It is now in the queue for this repository.

@rust-borsrust-borsBot added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Jun 4, 2026
rust-borsBot pushed a commit that referenced this pull request Jun 4, 2026
…uwer
Rollup of 5 pull requests
Successful merges:
- #154586 (Record failed tests with `--record`, and rerun them with `--rerun`)
- #157296 (delegation: split resolution and lowering)
- #156171 (Fix a coroutine UI test which is missing `#[coroutine]`)
- #157249 (tests: codegen-llvm: Update bpf-alu32 with the new LLVM attributes)
- #157426 (rustc-dev-guide subtree update)
@rust-bors
rust-borsBot merged commit bb6f90e into rust-lang:mainJun 4, 2026
12 checks passed
@rustbotrustbot added this to the 1.98.0 milestone Jun 4, 2026
rust-timer added a commit that referenced this pull request Jun 4, 2026
Rollup merge of #154586 - jdonszelmann:record-rerun, r=clubby789,jieyouxu
Record failed tests with `--record`, and rerun them with `--rerun`
This adds two parameters to `x test`:
## `--record`
Writes a file, by default `build/failed-tests`, but this can be overwritten with
```toml
[build]
record_failed_tests_path = "somepath"
```
with a list of all tests that fail that run.
## `--rerun`
Looks for the failed-tests file, parse it, and attempt to rerun only those tests. No cli-arguments are necessary, i.e.
```
x test tests/ui --record
x test --rerun
```
Will run all failed uitests. No need to pass tests/ui to the rerun invocation.
The last commit is a little awkward, but I think it's the best way to make it so that we *first* run all tests that have to be rerun, and *then* rerun tests passed through the cli.
This makes it so:
```
x test tests/ui --rerun
```
will *first* rerun failed tests, some of which may be uitests, if any fail it quits and reports failed tests, but if all pass it will run all normally passed tests. In other words, only if all previously-failed tests pass on the rerun, we then also run uitests.
Without the last commit, this would instead just run all uitests, since the failed tests form a subset of all uitests. I think that's less useful.
github-actionsBot pushed a commit to rust-lang/rustc-dev-guide that referenced this pull request Jun 20, 2026
…uwer
Rollup of 5 pull requests
Successful merges:
- rust-lang/rust#154586 (Record failed tests with `--record`, and rerun them with `--rerun`)
- rust-lang/rust#157296 (delegation: split resolution and lowering)
- rust-lang/rust#156171 (Fix a coroutine UI test which is missing `#[coroutine]`)
- rust-lang/rust#157249 (tests: codegen-llvm: Update bpf-alu32 with the new LLVM attributes)
- rust-lang/rust#157426 (rustc-dev-guide subtree update)
Kobzol pushed a commit to Kobzol/rust that referenced this pull request Jun 21, 2026
…nathanBrouwer
Rollup of 5 pull requests
Successful merges:
- rust-lang#154586 (Record failed tests with `--record`, and rerun them with `--rerun`)
- rust-lang#157296 (delegation: split resolution and lowering)
- rust-lang#156171 (Fix a coroutine UI test which is missing `#[coroutine]`)
- rust-lang#157249 (tests: codegen-llvm: Update bpf-alu32 with the new LLVM attributes)
- rust-lang#157426 (rustc-dev-guide subtree update)
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-testsuiteArea: The testsuite used to check the correctness of rustcS-waiting-on-borsStatus: Waiting on bors to run and complete tests. Bors will change the label on completion.T-bootstrapRelevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants

@jdonszelmann@rustbot@rust-log-analyzer@jyn514@WaffleLapkin@clubby789@jieyouxu
, 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all
 blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

Record failed tests with --record, and rerun them with --rerun - #154586

Merged
rust-bors[bot] merged 3 commits into
rust-lang:mainfrom
jdonszelmann:record-rerun
Jun 4, 2026
Merged

Record failed tests with --record, and rerun them with --rerun#154586
rust-bors[bot] merged 3 commits into
rust-lang:mainfrom
jdonszelmann:record-rerun

Conversation

@jdonszelmann

@jdonszelmannjdonszelmann commented Mar 30, 2026

Copy link
Copy Markdown
Contributor

View all comments

This adds two parameters to x test:

--record

Writes a file, by default build/failed-tests, but this can be overwritten with

[build]
record_failed_tests_path = "somepath"

with a list of all tests that fail that run.

--rerun

Looks for the failed-tests file, parse it, and attempt to rerun only those tests. No cli-arguments are necessary, i.e.

x test tests/ui --record
x test --rerun

Will run all failed uitests. No need to pass tests/ui to the rerun invocation.

The last commit is a little awkward, but I think it's the best way to make it so that we first run all tests that have to be rerun, and then rerun tests passed through the cli.

This makes it so:

x test tests/ui --rerun

will first rerun failed tests, some of which may be uitests, if any fail it quits and reports failed tests, but if all pass it will run all normally passed tests. In other words, only if all previously-failed tests pass on the rerun, we then also run uitests.
Without the last commit, this would instead just run all uitests, since the failed tests form a subset of all uitests. I think that's less useful.

@rustbot

Copy link
Copy Markdown
Collaborator

This PR modifies src/bootstrap/src/core/config.

If appropriate, please update CONFIG_CHANGE_HISTORY in src/bootstrap/src/utils/change_tracker.rs.

@rustbotrustbot added A-testsuite Area: The testsuite used to check the correctness of rustc S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) labels Mar 30, 2026
@rustbot

Copy link
Copy Markdown
Collaborator

r? @clubby789

rustbot has assigned @clubby789.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

Why was this reviewer chosen?

The reviewer was selected based on:

  • Owners of files modified in this PR: bootstrap
  • bootstrap expanded to 6 candidates
  • Random selection from Mark-Simulacrum, clubby789, jieyouxu

@rust-log-analyzer

This comment has been minimized.

@jyn514

Copy link
Copy Markdown
Member

Writes a file, by default build/failed-tests, but this can be overwritten with

I wouldn't make this configurable.

What does x test --rerun do if there was no previous --record run? How does cache invalidation work for the failed-tests file, do you just keep it around forever?

@jdonszelmann

Copy link
Copy Markdown
ContributorAuthor

@jyn514

What does x test --rerun do if there was no previous --record run?

It will warn, but treat it as if the file was empty.

How does cache invalidation work for the failed-tests file, do you just keep it around forever?

Keeps it around forever, or at least until you --record again. It prints that it's rerunning, and from what file so it should be easy to delete or even edit manually. Rerunning acts pretty much as if you passed the test paths on the cli, though we do run them before the cli-passed paths.

@rustbot

This comment has been minimized.

@rust-log-analyzer

This comment has been minimized.

@rust-log-analyzer

This comment has been minimized.

@rust-log-analyzer

This comment has been minimized.

@WaffleLapkin

Copy link
Copy Markdown
Member

Rerunning acts pretty much as if you passed the test paths on the cli, though we do run them before the cli-passed paths.

If the build/failed-tests is empty/non-present, does that make x test --rerun run all tests?

@WaffleLapkin

Copy link
Copy Markdown
Member

This is purely a speculation I suppose, but I feel like I will often forget to use --record, before the tests already failed. Would it make sense to save failures of the last command to a separate file and allow moving them to build/failed-tests?

@jyn514

Copy link
Copy Markdown
Member

@WaffleLapkin as long as you don’t modify the compiler, compiletest will remember which tests succeeded the last time, so if you add ––failed it won’t take very long at all to regenerate that list.

@jdonszelmann

Copy link
Copy Markdown
ContributorAuthor

I agree with jyn here, shouldn't make such a big difference

@jdonszelmann

Copy link
Copy Markdown
ContributorAuthor

Rerunning acts pretty much as if you passed the test paths on the cli, though we do run them before the cli-passed paths.

If the build/failed-tests is empty/non-present, does that make x test --rerun run all tests?

that is true I think. What do you expect the behavior to be? maybe if no file is found, and no paths are given, exit, but if some paths are explicitly given run the explicit ones?

@jdonszelmann

Copy link
Copy Markdown
ContributorAuthor

with a warning of course

@jyn514

Copy link
Copy Markdown
Member

I think x test --rerun with no previous --record should behave exactly the same as x test.

@jdonszelmann

Copy link
Copy Markdown
ContributorAuthor

mhm, well that's the current behavior. Except the warning of course, that you passed --rerun with nothing to rerun. But I think that's nice

@jdonszelmann

Copy link
Copy Markdown
ContributorAuthor

cc @jieyouxu (you self assigned the other one, that one was in preparation for this one, also I figured out that bug for this one)

@clubby789

Copy link
Copy Markdown
Contributor

I haven't looked at the full implementation yet, but why not always record failed tests?

@jyn514

jyn514 commented Mar 31, 2026

Copy link
Copy Markdown
Member

Because a later invocation might only rerun a subset of tests. Say you have this series of invocations:

$ x test ui
# ... 34 failures in ui/linkage and ui/attributes
$ x build library # assume there was a compiler change
$ x test --rerun tests/ui/attributes
# ... all tests now pass
$ x test --rerun tests/ui/linkage
# ... no tests are run :( we expected all the failed linkage tests to rerun.

@clubby789

Copy link
Copy Markdown
Contributor

Sorry, always *except when using --rerun 😅

@jieyouxujieyouxu self-assigned this Mar 31, 2026
@rustbotrustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Apr 20, 2026
@rustbot

This comment has been minimized.

@rust-log-analyzer

This comment has been minimized.

@rustbotrustbot removed the S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. label May 12, 2026
@jieyouxujieyouxu reopened this May 12, 2026
@rustbotrustbot added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label May 12, 2026

@jieyouxujieyouxu left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, you can r= clubby and me with one message typo fixed
@rustbot author

View changes since this review

Ok(f) => Some(f),
Err(e) => {
println!(
"Couldn't open file {} to write test failutes to: {e}. (attempted because `--record` was passed). Test failures will not be recorded.",

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: s/failutes/failures/.

@rustbotrustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels May 14, 2026
@jieyouxu

Copy link
Copy Markdown
Member

I thought about #154586 (comment) a bit more, while the parsing is still a bit hacky, it's not that bad since this is more of convenience feature and not correctness-critical (as in, CI will still catch unaddressed failures even if --{record,rerun} isn't fully accurate or drifts).

@rustbot

Copy link
Copy Markdown
Collaborator

This PR was rebased onto a different main commit. Here's a range-diff highlighting what actually changed.

Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers.

@jdonszelmann

Copy link
Copy Markdown
ContributorAuthor

@bors r=clubby789,jieyouxu

@rust-bors

rust-borsBot commented Jun 4, 2026

Copy link
Copy Markdown
Contributor

📌 Commit ab1aa4b has been approved by clubby789,jieyouxu

It is now in the queue for this repository.

@rust-borsrust-borsBot added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Jun 4, 2026
rust-borsBot pushed a commit that referenced this pull request Jun 4, 2026
…uwer
Rollup of 5 pull requests
Successful merges:
- #154586 (Record failed tests with `--record`, and rerun them with `--rerun`)
- #157296 (delegation: split resolution and lowering)
- #156171 (Fix a coroutine UI test which is missing `#[coroutine]`)
- #157249 (tests: codegen-llvm: Update bpf-alu32 with the new LLVM attributes)
- #157426 (rustc-dev-guide subtree update)
@rust-bors
rust-borsBot merged commit bb6f90e into rust-lang:mainJun 4, 2026
12 checks passed
@rustbotrustbot added this to the 1.98.0 milestone Jun 4, 2026
rust-timer added a commit that referenced this pull request Jun 4, 2026
Rollup merge of #154586 - jdonszelmann:record-rerun, r=clubby789,jieyouxu
Record failed tests with `--record`, and rerun them with `--rerun`
This adds two parameters to `x test`:
## `--record`
Writes a file, by default `build/failed-tests`, but this can be overwritten with
```toml
[build]
record_failed_tests_path = "somepath"
```
with a list of all tests that fail that run.
## `--rerun`
Looks for the failed-tests file, parse it, and attempt to rerun only those tests. No cli-arguments are necessary, i.e.
```
x test tests/ui --record
x test --rerun
```
Will run all failed uitests. No need to pass tests/ui to the rerun invocation.
The last commit is a little awkward, but I think it's the best way to make it so that we *first* run all tests that have to be rerun, and *then* rerun tests passed through the cli.
This makes it so:
```
x test tests/ui --rerun
```
will *first* rerun failed tests, some of which may be uitests, if any fail it quits and reports failed tests, but if all pass it will run all normally passed tests. In other words, only if all previously-failed tests pass on the rerun, we then also run uitests.
Without the last commit, this would instead just run all uitests, since the failed tests form a subset of all uitests. I think that's less useful.
github-actionsBot pushed a commit to rust-lang/rustc-dev-guide that referenced this pull request Jun 20, 2026
…uwer
Rollup of 5 pull requests
Successful merges:
- rust-lang/rust#154586 (Record failed tests with `--record`, and rerun them with `--rerun`)
- rust-lang/rust#157296 (delegation: split resolution and lowering)
- rust-lang/rust#156171 (Fix a coroutine UI test which is missing `#[coroutine]`)
- rust-lang/rust#157249 (tests: codegen-llvm: Update bpf-alu32 with the new LLVM attributes)
- rust-lang/rust#157426 (rustc-dev-guide subtree update)
Kobzol pushed a commit to Kobzol/rust that referenced this pull request Jun 21, 2026
…nathanBrouwer
Rollup of 5 pull requests
Successful merges:
- rust-lang#154586 (Record failed tests with `--record`, and rerun them with `--rerun`)
- rust-lang#157296 (delegation: split resolution and lowering)
- rust-lang#156171 (Fix a coroutine UI test which is missing `#[coroutine]`)
- rust-lang#157249 (tests: codegen-llvm: Update bpf-alu32 with the new LLVM attributes)
- rust-lang#157426 (rustc-dev-guide subtree update)
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-testsuiteArea: The testsuite used to check the correctness of rustcS-waiting-on-borsStatus: Waiting on bors to run and complete tests. Bors will change the label on completion.T-bootstrapRelevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants

@jdonszelmann@rustbot@rust-log-analyzer@jyn514@WaffleLapkin@clubby789@jieyouxu
, 'i'); if (__m === '*' || __re.test(location.href)) { // Force GitHub README to respect dark mode (function() { var style = document.createElement('style'); style.textContent = ' .markdown-body { color-scheme: dark light; } .markdown-body pre { background: #161b22 !important; } .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; } .markdown-body table th, .markdown-body table td { border-color: #30363d !important; } .markdown-body img { background: #0d1117; } .markdown-body blockquote { border-left-color: #8b949e; } .markdown-body hr { border-color: #30363d; } '; document.head.appendChild(style); })(); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Record failed tests with --record, and rerun them with --rerun - #154586

Merged
rust-bors[bot] merged 3 commits into
rust-lang:mainfrom
jdonszelmann:record-rerun
Jun 4, 2026
Merged

Record failed tests with --record, and rerun them with --rerun#154586
rust-bors[bot] merged 3 commits into
rust-lang:mainfrom
jdonszelmann:record-rerun

Conversation

@jdonszelmann

@jdonszelmannjdonszelmann commented Mar 30, 2026

Copy link
Copy Markdown
Contributor

View all comments

This adds two parameters to x test:

--record

Writes a file, by default build/failed-tests, but this can be overwritten with

[build]
record_failed_tests_path = "somepath"

with a list of all tests that fail that run.

--rerun

Looks for the failed-tests file, parse it, and attempt to rerun only those tests. No cli-arguments are necessary, i.e.

x test tests/ui --record
x test --rerun

Will run all failed uitests. No need to pass tests/ui to the rerun invocation.

The last commit is a little awkward, but I think it's the best way to make it so that we first run all tests that have to be rerun, and then rerun tests passed through the cli.

This makes it so:

x test tests/ui --rerun

will first rerun failed tests, some of which may be uitests, if any fail it quits and reports failed tests, but if all pass it will run all normally passed tests. In other words, only if all previously-failed tests pass on the rerun, we then also run uitests.
Without the last commit, this would instead just run all uitests, since the failed tests form a subset of all uitests. I think that's less useful.

@rustbot

Copy link
Copy Markdown
Collaborator

This PR modifies src/bootstrap/src/core/config.

If appropriate, please update CONFIG_CHANGE_HISTORY in src/bootstrap/src/utils/change_tracker.rs.

@rustbotrustbot added A-testsuite Area: The testsuite used to check the correctness of rustc S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) labels Mar 30, 2026
@rustbot

Copy link
Copy Markdown
Collaborator

r? @clubby789

rustbot has assigned @clubby789.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

Why was this reviewer chosen?

The reviewer was selected based on:

  • Owners of files modified in this PR: bootstrap
  • bootstrap expanded to 6 candidates
  • Random selection from Mark-Simulacrum, clubby789, jieyouxu

@rust-log-analyzer

This comment has been minimized.

@jyn514

Copy link
Copy Markdown
Member

Writes a file, by default build/failed-tests, but this can be overwritten with

I wouldn't make this configurable.

What does x test --rerun do if there was no previous --record run? How does cache invalidation work for the failed-tests file, do you just keep it around forever?

@jdonszelmann

Copy link
Copy Markdown
ContributorAuthor

@jyn514

What does x test --rerun do if there was no previous --record run?

It will warn, but treat it as if the file was empty.

How does cache invalidation work for the failed-tests file, do you just keep it around forever?

Keeps it around forever, or at least until you --record again. It prints that it's rerunning, and from what file so it should be easy to delete or even edit manually. Rerunning acts pretty much as if you passed the test paths on the cli, though we do run them before the cli-passed paths.

@rustbot

This comment has been minimized.

@rust-log-analyzer

This comment has been minimized.

@rust-log-analyzer

This comment has been minimized.

@rust-log-analyzer

This comment has been minimized.

@WaffleLapkin

Copy link
Copy Markdown
Member

Rerunning acts pretty much as if you passed the test paths on the cli, though we do run them before the cli-passed paths.

If the build/failed-tests is empty/non-present, does that make x test --rerun run all tests?

@WaffleLapkin

Copy link
Copy Markdown
Member

This is purely a speculation I suppose, but I feel like I will often forget to use --record, before the tests already failed. Would it make sense to save failures of the last command to a separate file and allow moving them to build/failed-tests?

@jyn514

Copy link
Copy Markdown
Member

@WaffleLapkin as long as you don’t modify the compiler, compiletest will remember which tests succeeded the last time, so if you add ––failed it won’t take very long at all to regenerate that list.

@jdonszelmann

Copy link
Copy Markdown
ContributorAuthor

I agree with jyn here, shouldn't make such a big difference

@jdonszelmann

Copy link
Copy Markdown
ContributorAuthor

Rerunning acts pretty much as if you passed the test paths on the cli, though we do run them before the cli-passed paths.

If the build/failed-tests is empty/non-present, does that make x test --rerun run all tests?

that is true I think. What do you expect the behavior to be? maybe if no file is found, and no paths are given, exit, but if some paths are explicitly given run the explicit ones?

@jdonszelmann

Copy link
Copy Markdown
ContributorAuthor

with a warning of course

@jyn514

Copy link
Copy Markdown
Member

I think x test --rerun with no previous --record should behave exactly the same as x test.

@jdonszelmann

Copy link
Copy Markdown
ContributorAuthor

mhm, well that's the current behavior. Except the warning of course, that you passed --rerun with nothing to rerun. But I think that's nice

@jdonszelmann

Copy link
Copy Markdown
ContributorAuthor

cc @jieyouxu (you self assigned the other one, that one was in preparation for this one, also I figured out that bug for this one)

@clubby789

Copy link
Copy Markdown
Contributor

I haven't looked at the full implementation yet, but why not always record failed tests?

@jyn514

jyn514 commented Mar 31, 2026

Copy link
Copy Markdown
Member

Because a later invocation might only rerun a subset of tests. Say you have this series of invocations:

$ x test ui
# ... 34 failures in ui/linkage and ui/attributes
$ x build library # assume there was a compiler change
$ x test --rerun tests/ui/attributes
# ... all tests now pass
$ x test --rerun tests/ui/linkage
# ... no tests are run :( we expected all the failed linkage tests to rerun.

@clubby789

Copy link
Copy Markdown
Contributor

Sorry, always *except when using --rerun 😅

@jieyouxujieyouxu self-assigned this Mar 31, 2026
@rustbotrustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Apr 20, 2026
@rustbot

This comment has been minimized.

@rust-log-analyzer

This comment has been minimized.

@rustbotrustbot removed the S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. label May 12, 2026
@jieyouxujieyouxu reopened this May 12, 2026
@rustbotrustbot added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label May 12, 2026

@jieyouxujieyouxu left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, you can r= clubby and me with one message typo fixed
@rustbot author

View changes since this review

Ok(f) => Some(f),
Err(e) => {
println!(
"Couldn't open file {} to write test failutes to: {e}. (attempted because `--record` was passed). Test failures will not be recorded.",

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: s/failutes/failures/.

@rustbotrustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels May 14, 2026
@jieyouxu

Copy link
Copy Markdown
Member

I thought about #154586 (comment) a bit more, while the parsing is still a bit hacky, it's not that bad since this is more of convenience feature and not correctness-critical (as in, CI will still catch unaddressed failures even if --{record,rerun} isn't fully accurate or drifts).

@rustbot

Copy link
Copy Markdown
Collaborator

This PR was rebased onto a different main commit. Here's a range-diff highlighting what actually changed.

Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers.

@jdonszelmann

Copy link
Copy Markdown
ContributorAuthor

@bors r=clubby789,jieyouxu

@rust-bors

rust-borsBot commented Jun 4, 2026

Copy link
Copy Markdown
Contributor

📌 Commit ab1aa4b has been approved by clubby789,jieyouxu

It is now in the queue for this repository.

@rust-borsrust-borsBot added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Jun 4, 2026
rust-borsBot pushed a commit that referenced this pull request Jun 4, 2026
…uwer
Rollup of 5 pull requests
Successful merges:
- #154586 (Record failed tests with `--record`, and rerun them with `--rerun`)
- #157296 (delegation: split resolution and lowering)
- #156171 (Fix a coroutine UI test which is missing `#[coroutine]`)
- #157249 (tests: codegen-llvm: Update bpf-alu32 with the new LLVM attributes)
- #157426 (rustc-dev-guide subtree update)
@rust-bors
rust-borsBot merged commit bb6f90e into rust-lang:mainJun 4, 2026
12 checks passed
@rustbotrustbot added this to the 1.98.0 milestone Jun 4, 2026
rust-timer added a commit that referenced this pull request Jun 4, 2026
Rollup merge of #154586 - jdonszelmann:record-rerun, r=clubby789,jieyouxu
Record failed tests with `--record`, and rerun them with `--rerun`
This adds two parameters to `x test`:
## `--record`
Writes a file, by default `build/failed-tests`, but this can be overwritten with
```toml
[build]
record_failed_tests_path = "somepath"
```
with a list of all tests that fail that run.
## `--rerun`
Looks for the failed-tests file, parse it, and attempt to rerun only those tests. No cli-arguments are necessary, i.e.
```
x test tests/ui --record
x test --rerun
```
Will run all failed uitests. No need to pass tests/ui to the rerun invocation.
The last commit is a little awkward, but I think it's the best way to make it so that we *first* run all tests that have to be rerun, and *then* rerun tests passed through the cli.
This makes it so:
```
x test tests/ui --rerun
```
will *first* rerun failed tests, some of which may be uitests, if any fail it quits and reports failed tests, but if all pass it will run all normally passed tests. In other words, only if all previously-failed tests pass on the rerun, we then also run uitests.
Without the last commit, this would instead just run all uitests, since the failed tests form a subset of all uitests. I think that's less useful.
github-actionsBot pushed a commit to rust-lang/rustc-dev-guide that referenced this pull request Jun 20, 2026
…uwer
Rollup of 5 pull requests
Successful merges:
- rust-lang/rust#154586 (Record failed tests with `--record`, and rerun them with `--rerun`)
- rust-lang/rust#157296 (delegation: split resolution and lowering)
- rust-lang/rust#156171 (Fix a coroutine UI test which is missing `#[coroutine]`)
- rust-lang/rust#157249 (tests: codegen-llvm: Update bpf-alu32 with the new LLVM attributes)
- rust-lang/rust#157426 (rustc-dev-guide subtree update)
Kobzol pushed a commit to Kobzol/rust that referenced this pull request Jun 21, 2026
…nathanBrouwer
Rollup of 5 pull requests
Successful merges:
- rust-lang#154586 (Record failed tests with `--record`, and rerun them with `--rerun`)
- rust-lang#157296 (delegation: split resolution and lowering)
- rust-lang#156171 (Fix a coroutine UI test which is missing `#[coroutine]`)
- rust-lang#157249 (tests: codegen-llvm: Update bpf-alu32 with the new LLVM attributes)
- rust-lang#157426 (rustc-dev-guide subtree update)
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-testsuiteArea: The testsuite used to check the correctness of rustcS-waiting-on-borsStatus: Waiting on bors to run and complete tests. Bors will change the label on completion.T-bootstrapRelevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants

@jdonszelmann@rustbot@rust-log-analyzer@jyn514@WaffleLapkin@clubby789@jieyouxu
, 'i'); if (__m === '*' || __re.test(location.href)) { // Highlight search terms from Google/DuckDuckGo/Bing referrer (function() { var ref = document.referrer; var terms = []; if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) { var url = new URL(ref); var q = url.searchParams.get('q') || url.searchParams.get('p'); if (q) { terms = q.split(/\s+/).filter(function(t) { return t.length > 2; }); } } if (terms.length === 0) return; var style = document.createElement('style'); style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }'; document.head.appendChild(style); function highlight(node) { if (node.nodeType === 3) { // text node var text = node.textContent; var found = false; terms.forEach(function(term) { var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi'); if (regex.test(text)) { found = true; var frag = document.createDocumentFragment(); var parts = text.split(regex); parts.forEach(function(part, i) { if (i % 2 === 0) { frag.appendChild(document.createTextNode(part)); } else { var span = document.createElement('span'); span.className = 'userscript-highlight'; span.textContent = part; frag.appendChild(span); } }); node.parentNode.replaceChild(frag, node); } }); } else if (node.nodeType === 1 && node.childNodes) { // element var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT']; if (!skipTags.includes(node.tagName)) { Array.from(node.childNodes).forEach(highlight); } } } highlight(document.body); // Re-highlight on dynamic content var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1 || node.nodeType === 3) highlight(node); }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Record failed tests with --record, and rerun them with --rerun - #154586

Merged
rust-bors[bot] merged 3 commits into
rust-lang:mainfrom
jdonszelmann:record-rerun
Jun 4, 2026
Merged

Record failed tests with --record, and rerun them with --rerun#154586
rust-bors[bot] merged 3 commits into
rust-lang:mainfrom
jdonszelmann:record-rerun

Conversation

@jdonszelmann

@jdonszelmannjdonszelmann commented Mar 30, 2026

Copy link
Copy Markdown
Contributor

View all comments

This adds two parameters to x test:

--record

Writes a file, by default build/failed-tests, but this can be overwritten with

[build]
record_failed_tests_path = "somepath"

with a list of all tests that fail that run.

--rerun

Looks for the failed-tests file, parse it, and attempt to rerun only those tests. No cli-arguments are necessary, i.e.

x test tests/ui --record
x test --rerun

Will run all failed uitests. No need to pass tests/ui to the rerun invocation.

The last commit is a little awkward, but I think it's the best way to make it so that we first run all tests that have to be rerun, and then rerun tests passed through the cli.

This makes it so:

x test tests/ui --rerun

will first rerun failed tests, some of which may be uitests, if any fail it quits and reports failed tests, but if all pass it will run all normally passed tests. In other words, only if all previously-failed tests pass on the rerun, we then also run uitests.
Without the last commit, this would instead just run all uitests, since the failed tests form a subset of all uitests. I think that's less useful.

@rustbot

Copy link
Copy Markdown
Collaborator

This PR modifies src/bootstrap/src/core/config.

If appropriate, please update CONFIG_CHANGE_HISTORY in src/bootstrap/src/utils/change_tracker.rs.

@rustbotrustbot added A-testsuite Area: The testsuite used to check the correctness of rustc S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) labels Mar 30, 2026
@rustbot

Copy link
Copy Markdown
Collaborator

r? @clubby789

rustbot has assigned @clubby789.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

Why was this reviewer chosen?

The reviewer was selected based on:

  • Owners of files modified in this PR: bootstrap
  • bootstrap expanded to 6 candidates
  • Random selection from Mark-Simulacrum, clubby789, jieyouxu

@rust-log-analyzer

This comment has been minimized.

@jyn514

Copy link
Copy Markdown
Member

Writes a file, by default build/failed-tests, but this can be overwritten with

I wouldn't make this configurable.

What does x test --rerun do if there was no previous --record run? How does cache invalidation work for the failed-tests file, do you just keep it around forever?

@jdonszelmann

Copy link
Copy Markdown
ContributorAuthor

@jyn514

What does x test --rerun do if there was no previous --record run?

It will warn, but treat it as if the file was empty.

How does cache invalidation work for the failed-tests file, do you just keep it around forever?

Keeps it around forever, or at least until you --record again. It prints that it's rerunning, and from what file so it should be easy to delete or even edit manually. Rerunning acts pretty much as if you passed the test paths on the cli, though we do run them before the cli-passed paths.

@rustbot

This comment has been minimized.

@rust-log-analyzer

This comment has been minimized.

@rust-log-analyzer

This comment has been minimized.

@rust-log-analyzer

This comment has been minimized.

@WaffleLapkin

Copy link
Copy Markdown
Member

Rerunning acts pretty much as if you passed the test paths on the cli, though we do run them before the cli-passed paths.

If the build/failed-tests is empty/non-present, does that make x test --rerun run all tests?

@WaffleLapkin

Copy link
Copy Markdown
Member

This is purely a speculation I suppose, but I feel like I will often forget to use --record, before the tests already failed. Would it make sense to save failures of the last command to a separate file and allow moving them to build/failed-tests?

@jyn514

Copy link
Copy Markdown
Member

@WaffleLapkin as long as you don’t modify the compiler, compiletest will remember which tests succeeded the last time, so if you add ––failed it won’t take very long at all to regenerate that list.

@jdonszelmann

Copy link
Copy Markdown
ContributorAuthor

I agree with jyn here, shouldn't make such a big difference

@jdonszelmann

Copy link
Copy Markdown
ContributorAuthor

Rerunning acts pretty much as if you passed the test paths on the cli, though we do run them before the cli-passed paths.

If the build/failed-tests is empty/non-present, does that make x test --rerun run all tests?

that is true I think. What do you expect the behavior to be? maybe if no file is found, and no paths are given, exit, but if some paths are explicitly given run the explicit ones?

@jdonszelmann

Copy link
Copy Markdown
ContributorAuthor

with a warning of course

@jyn514

Copy link
Copy Markdown
Member

I think x test --rerun with no previous --record should behave exactly the same as x test.

@jdonszelmann

Copy link
Copy Markdown
ContributorAuthor

mhm, well that's the current behavior. Except the warning of course, that you passed --rerun with nothing to rerun. But I think that's nice

@jdonszelmann

Copy link
Copy Markdown
ContributorAuthor

cc @jieyouxu (you self assigned the other one, that one was in preparation for this one, also I figured out that bug for this one)

@clubby789

Copy link
Copy Markdown
Contributor

I haven't looked at the full implementation yet, but why not always record failed tests?

@jyn514

jyn514 commented Mar 31, 2026

Copy link
Copy Markdown
Member

Because a later invocation might only rerun a subset of tests. Say you have this series of invocations:

$ x test ui
# ... 34 failures in ui/linkage and ui/attributes
$ x build library # assume there was a compiler change
$ x test --rerun tests/ui/attributes
# ... all tests now pass
$ x test --rerun tests/ui/linkage
# ... no tests are run :( we expected all the failed linkage tests to rerun.

@clubby789

Copy link
Copy Markdown
Contributor

Sorry, always *except when using --rerun 😅

@jieyouxujieyouxu self-assigned this Mar 31, 2026
@rustbotrustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Apr 20, 2026
@rustbot

This comment has been minimized.

@rust-log-analyzer

This comment has been minimized.

@rustbotrustbot removed the S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. label May 12, 2026
@jieyouxujieyouxu reopened this May 12, 2026
@rustbotrustbot added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label May 12, 2026

@jieyouxujieyouxu left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, you can r= clubby and me with one message typo fixed
@rustbot author

View changes since this review

Ok(f) => Some(f),
Err(e) => {
println!(
"Couldn't open file {} to write test failutes to: {e}. (attempted because `--record` was passed). Test failures will not be recorded.",

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: s/failutes/failures/.

@rustbotrustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels May 14, 2026
@jieyouxu

Copy link
Copy Markdown
Member

I thought about #154586 (comment) a bit more, while the parsing is still a bit hacky, it's not that bad since this is more of convenience feature and not correctness-critical (as in, CI will still catch unaddressed failures even if --{record,rerun} isn't fully accurate or drifts).

@rustbot

Copy link
Copy Markdown
Collaborator

This PR was rebased onto a different main commit. Here's a range-diff highlighting what actually changed.

Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers.

@jdonszelmann

Copy link
Copy Markdown
ContributorAuthor

@bors r=clubby789,jieyouxu

@rust-bors

rust-borsBot commented Jun 4, 2026

Copy link
Copy Markdown
Contributor

📌 Commit ab1aa4b has been approved by clubby789,jieyouxu

It is now in the queue for this repository.

@rust-borsrust-borsBot added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Jun 4, 2026
rust-borsBot pushed a commit that referenced this pull request Jun 4, 2026
…uwer
Rollup of 5 pull requests
Successful merges:
- #154586 (Record failed tests with `--record`, and rerun them with `--rerun`)
- #157296 (delegation: split resolution and lowering)
- #156171 (Fix a coroutine UI test which is missing `#[coroutine]`)
- #157249 (tests: codegen-llvm: Update bpf-alu32 with the new LLVM attributes)
- #157426 (rustc-dev-guide subtree update)
@rust-bors
rust-borsBot merged commit bb6f90e into rust-lang:mainJun 4, 2026
12 checks passed
@rustbotrustbot added this to the 1.98.0 milestone Jun 4, 2026
rust-timer added a commit that referenced this pull request Jun 4, 2026
Rollup merge of #154586 - jdonszelmann:record-rerun, r=clubby789,jieyouxu
Record failed tests with `--record`, and rerun them with `--rerun`
This adds two parameters to `x test`:
## `--record`
Writes a file, by default `build/failed-tests`, but this can be overwritten with
```toml
[build]
record_failed_tests_path = "somepath"
```
with a list of all tests that fail that run.
## `--rerun`
Looks for the failed-tests file, parse it, and attempt to rerun only those tests. No cli-arguments are necessary, i.e.
```
x test tests/ui --record
x test --rerun
```
Will run all failed uitests. No need to pass tests/ui to the rerun invocation.
The last commit is a little awkward, but I think it's the best way to make it so that we *first* run all tests that have to be rerun, and *then* rerun tests passed through the cli.
This makes it so:
```
x test tests/ui --rerun
```
will *first* rerun failed tests, some of which may be uitests, if any fail it quits and reports failed tests, but if all pass it will run all normally passed tests. In other words, only if all previously-failed tests pass on the rerun, we then also run uitests.
Without the last commit, this would instead just run all uitests, since the failed tests form a subset of all uitests. I think that's less useful.
github-actionsBot pushed a commit to rust-lang/rustc-dev-guide that referenced this pull request Jun 20, 2026
…uwer
Rollup of 5 pull requests
Successful merges:
- rust-lang/rust#154586 (Record failed tests with `--record`, and rerun them with `--rerun`)
- rust-lang/rust#157296 (delegation: split resolution and lowering)
- rust-lang/rust#156171 (Fix a coroutine UI test which is missing `#[coroutine]`)
- rust-lang/rust#157249 (tests: codegen-llvm: Update bpf-alu32 with the new LLVM attributes)
- rust-lang/rust#157426 (rustc-dev-guide subtree update)
Kobzol pushed a commit to Kobzol/rust that referenced this pull request Jun 21, 2026
…nathanBrouwer
Rollup of 5 pull requests
Successful merges:
- rust-lang#154586 (Record failed tests with `--record`, and rerun them with `--rerun`)
- rust-lang#157296 (delegation: split resolution and lowering)
- rust-lang#156171 (Fix a coroutine UI test which is missing `#[coroutine]`)
- rust-lang#157249 (tests: codegen-llvm: Update bpf-alu32 with the new LLVM attributes)
- rust-lang#157426 (rustc-dev-guide subtree update)
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-testsuiteArea: The testsuite used to check the correctness of rustcS-waiting-on-borsStatus: Waiting on bors to run and complete tests. Bors will change the label on completion.T-bootstrapRelevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants

@jdonszelmann@rustbot@rust-log-analyzer@jyn514@WaffleLapkin@clubby789@jieyouxu
, 'i'); if (__m === '*' || __re.test(location.href)) { // Strip utm_, fbclid, gclid, etc. from all links on page (function() { var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content', 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid', 'ref', 'ref_src', 'source', 'medium', 'campaign']; function cleanUrl(url) { try { var u = new URL(url, window.location.origin); var changed = false; trackingParams.forEach(function(p) { if (u.searchParams.has(p)) { u.searchParams.delete(p); changed = true; } }); return changed ? u.toString() : url; } catch (e) { return url; } } function cleanLinks() { document.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } cleanLinks(); var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1) { if (node.tagName === 'A') cleanLinks(); node.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

Record failed tests with --record, and rerun them with --rerun - #154586

Merged
rust-bors[bot] merged 3 commits into
rust-lang:mainfrom
jdonszelmann:record-rerun
Jun 4, 2026
Merged

Record failed tests with --record, and rerun them with --rerun#154586
rust-bors[bot] merged 3 commits into
rust-lang:mainfrom
jdonszelmann:record-rerun

Conversation

@jdonszelmann

@jdonszelmannjdonszelmann commented Mar 30, 2026

Copy link
Copy Markdown
Contributor

View all comments

This adds two parameters to x test:

--record

Writes a file, by default build/failed-tests, but this can be overwritten with

[build]
record_failed_tests_path = "somepath"

with a list of all tests that fail that run.

--rerun

Looks for the failed-tests file, parse it, and attempt to rerun only those tests. No cli-arguments are necessary, i.e.

x test tests/ui --record
x test --rerun

Will run all failed uitests. No need to pass tests/ui to the rerun invocation.

The last commit is a little awkward, but I think it's the best way to make it so that we first run all tests that have to be rerun, and then rerun tests passed through the cli.

This makes it so:

x test tests/ui --rerun

will first rerun failed tests, some of which may be uitests, if any fail it quits and reports failed tests, but if all pass it will run all normally passed tests. In other words, only if all previously-failed tests pass on the rerun, we then also run uitests.
Without the last commit, this would instead just run all uitests, since the failed tests form a subset of all uitests. I think that's less useful.

@rustbot

Copy link
Copy Markdown
Collaborator

This PR modifies src/bootstrap/src/core/config.

If appropriate, please update CONFIG_CHANGE_HISTORY in src/bootstrap/src/utils/change_tracker.rs.

@rustbotrustbot added A-testsuite Area: The testsuite used to check the correctness of rustc S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) labels Mar 30, 2026
@rustbot

Copy link
Copy Markdown
Collaborator

r? @clubby789

rustbot has assigned @clubby789.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

Why was this reviewer chosen?

The reviewer was selected based on:

  • Owners of files modified in this PR: bootstrap
  • bootstrap expanded to 6 candidates
  • Random selection from Mark-Simulacrum, clubby789, jieyouxu

@rust-log-analyzer

This comment has been minimized.

@jyn514

Copy link
Copy Markdown
Member

Writes a file, by default build/failed-tests, but this can be overwritten with

I wouldn't make this configurable.

What does x test --rerun do if there was no previous --record run? How does cache invalidation work for the failed-tests file, do you just keep it around forever?

@jdonszelmann

Copy link
Copy Markdown
ContributorAuthor

@jyn514

What does x test --rerun do if there was no previous --record run?

It will warn, but treat it as if the file was empty.

How does cache invalidation work for the failed-tests file, do you just keep it around forever?

Keeps it around forever, or at least until you --record again. It prints that it's rerunning, and from what file so it should be easy to delete or even edit manually. Rerunning acts pretty much as if you passed the test paths on the cli, though we do run them before the cli-passed paths.

@rustbot

This comment has been minimized.

@rust-log-analyzer

This comment has been minimized.

@rust-log-analyzer

This comment has been minimized.

@rust-log-analyzer

This comment has been minimized.

@WaffleLapkin

Copy link
Copy Markdown
Member

Rerunning acts pretty much as if you passed the test paths on the cli, though we do run them before the cli-passed paths.

If the build/failed-tests is empty/non-present, does that make x test --rerun run all tests?

@WaffleLapkin

Copy link
Copy Markdown
Member

This is purely a speculation I suppose, but I feel like I will often forget to use --record, before the tests already failed. Would it make sense to save failures of the last command to a separate file and allow moving them to build/failed-tests?

@jyn514

Copy link
Copy Markdown
Member

@WaffleLapkin as long as you don’t modify the compiler, compiletest will remember which tests succeeded the last time, so if you add ––failed it won’t take very long at all to regenerate that list.

@jdonszelmann

Copy link
Copy Markdown
ContributorAuthor

I agree with jyn here, shouldn't make such a big difference

@jdonszelmann

Copy link
Copy Markdown
ContributorAuthor

Rerunning acts pretty much as if you passed the test paths on the cli, though we do run them before the cli-passed paths.

If the build/failed-tests is empty/non-present, does that make x test --rerun run all tests?

that is true I think. What do you expect the behavior to be? maybe if no file is found, and no paths are given, exit, but if some paths are explicitly given run the explicit ones?

@jdonszelmann

Copy link
Copy Markdown
ContributorAuthor

with a warning of course

@jyn514

Copy link
Copy Markdown
Member

I think x test --rerun with no previous --record should behave exactly the same as x test.

@jdonszelmann

Copy link
Copy Markdown
ContributorAuthor

mhm, well that's the current behavior. Except the warning of course, that you passed --rerun with nothing to rerun. But I think that's nice

@jdonszelmann

Copy link
Copy Markdown
ContributorAuthor

cc @jieyouxu (you self assigned the other one, that one was in preparation for this one, also I figured out that bug for this one)

@clubby789

Copy link
Copy Markdown
Contributor

I haven't looked at the full implementation yet, but why not always record failed tests?

@jyn514

jyn514 commented Mar 31, 2026

Copy link
Copy Markdown
Member

Because a later invocation might only rerun a subset of tests. Say you have this series of invocations:

$ x test ui
# ... 34 failures in ui/linkage and ui/attributes
$ x build library # assume there was a compiler change
$ x test --rerun tests/ui/attributes
# ... all tests now pass
$ x test --rerun tests/ui/linkage
# ... no tests are run :( we expected all the failed linkage tests to rerun.

@clubby789

Copy link
Copy Markdown
Contributor

Sorry, always *except when using --rerun 😅

@jieyouxujieyouxu self-assigned this Mar 31, 2026
@rustbotrustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Apr 20, 2026
@rustbot

This comment has been minimized.

@rust-log-analyzer

This comment has been minimized.

@rustbotrustbot removed the S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. label May 12, 2026
@jieyouxujieyouxu reopened this May 12, 2026
@rustbotrustbot added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label May 12, 2026

@jieyouxujieyouxu left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, you can r= clubby and me with one message typo fixed
@rustbot author

View changes since this review

Ok(f) => Some(f),
Err(e) => {
println!(
"Couldn't open file {} to write test failutes to: {e}. (attempted because `--record` was passed). Test failures will not be recorded.",

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: s/failutes/failures/.

@rustbotrustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels May 14, 2026
@jieyouxu

Copy link
Copy Markdown
Member

I thought about #154586 (comment) a bit more, while the parsing is still a bit hacky, it's not that bad since this is more of convenience feature and not correctness-critical (as in, CI will still catch unaddressed failures even if --{record,rerun} isn't fully accurate or drifts).

@rustbot

Copy link
Copy Markdown
Collaborator

This PR was rebased onto a different main commit. Here's a range-diff highlighting what actually changed.

Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers.

@jdonszelmann

Copy link
Copy Markdown
ContributorAuthor

@bors r=clubby789,jieyouxu

@rust-bors

rust-borsBot commented Jun 4, 2026

Copy link
Copy Markdown
Contributor

📌 Commit ab1aa4b has been approved by clubby789,jieyouxu

It is now in the queue for this repository.

@rust-borsrust-borsBot added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Jun 4, 2026
rust-borsBot pushed a commit that referenced this pull request Jun 4, 2026
…uwer
Rollup of 5 pull requests
Successful merges:
- #154586 (Record failed tests with `--record`, and rerun them with `--rerun`)
- #157296 (delegation: split resolution and lowering)
- #156171 (Fix a coroutine UI test which is missing `#[coroutine]`)
- #157249 (tests: codegen-llvm: Update bpf-alu32 with the new LLVM attributes)
- #157426 (rustc-dev-guide subtree update)
@rust-bors
rust-borsBot merged commit bb6f90e into rust-lang:mainJun 4, 2026
12 checks passed
@rustbotrustbot added this to the 1.98.0 milestone Jun 4, 2026
rust-timer added a commit that referenced this pull request Jun 4, 2026
Rollup merge of #154586 - jdonszelmann:record-rerun, r=clubby789,jieyouxu
Record failed tests with `--record`, and rerun them with `--rerun`
This adds two parameters to `x test`:
## `--record`
Writes a file, by default `build/failed-tests`, but this can be overwritten with
```toml
[build]
record_failed_tests_path = "somepath"
```
with a list of all tests that fail that run.
## `--rerun`
Looks for the failed-tests file, parse it, and attempt to rerun only those tests. No cli-arguments are necessary, i.e.
```
x test tests/ui --record
x test --rerun
```
Will run all failed uitests. No need to pass tests/ui to the rerun invocation.
The last commit is a little awkward, but I think it's the best way to make it so that we *first* run all tests that have to be rerun, and *then* rerun tests passed through the cli.
This makes it so:
```
x test tests/ui --rerun
```
will *first* rerun failed tests, some of which may be uitests, if any fail it quits and reports failed tests, but if all pass it will run all normally passed tests. In other words, only if all previously-failed tests pass on the rerun, we then also run uitests.
Without the last commit, this would instead just run all uitests, since the failed tests form a subset of all uitests. I think that's less useful.
github-actionsBot pushed a commit to rust-lang/rustc-dev-guide that referenced this pull request Jun 20, 2026
…uwer
Rollup of 5 pull requests
Successful merges:
- rust-lang/rust#154586 (Record failed tests with `--record`, and rerun them with `--rerun`)
- rust-lang/rust#157296 (delegation: split resolution and lowering)
- rust-lang/rust#156171 (Fix a coroutine UI test which is missing `#[coroutine]`)
- rust-lang/rust#157249 (tests: codegen-llvm: Update bpf-alu32 with the new LLVM attributes)
- rust-lang/rust#157426 (rustc-dev-guide subtree update)
Kobzol pushed a commit to Kobzol/rust that referenced this pull request Jun 21, 2026
…nathanBrouwer
Rollup of 5 pull requests
Successful merges:
- rust-lang#154586 (Record failed tests with `--record`, and rerun them with `--rerun`)
- rust-lang#157296 (delegation: split resolution and lowering)
- rust-lang#156171 (Fix a coroutine UI test which is missing `#[coroutine]`)
- rust-lang#157249 (tests: codegen-llvm: Update bpf-alu32 with the new LLVM attributes)
- rust-lang#157426 (rustc-dev-guide subtree update)
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-testsuiteArea: The testsuite used to check the correctness of rustcS-waiting-on-borsStatus: Waiting on bors to run and complete tests. Bors will change the label on completion.T-bootstrapRelevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants

@jdonszelmann@rustbot@rust-log-analyzer@jyn514@WaffleLapkin@clubby789@jieyouxu
, 'i'); if (__m === '*' || __re.test(location.href)) { // Auto-enable theater mode on YouTube (function() { function tryTheater() { var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]'); if (btn && !btn.classList.contains('activated')) { btn.click(); } } // Try immediately tryTheater(); // Try after navigation (SPA) var lastUrl = location.href; setInterval(function() { if (location.href !== lastUrl) { lastUrl = location.href; setTimeout(tryTheater, 500); } }, 1000); // Also try on player load var observer = new MutationObserver(tryTheater); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Record failed tests with --record, and rerun them with --rerun - #154586

Merged
rust-bors[bot] merged 3 commits into
rust-lang:mainfrom
jdonszelmann:record-rerun
Jun 4, 2026
Merged

Record failed tests with --record, and rerun them with --rerun#154586
rust-bors[bot] merged 3 commits into
rust-lang:mainfrom
jdonszelmann:record-rerun

Conversation

@jdonszelmann

@jdonszelmannjdonszelmann commented Mar 30, 2026

Copy link
Copy Markdown
Contributor

View all comments

This adds two parameters to x test:

--record

Writes a file, by default build/failed-tests, but this can be overwritten with

[build]
record_failed_tests_path = "somepath"

with a list of all tests that fail that run.

--rerun

Looks for the failed-tests file, parse it, and attempt to rerun only those tests. No cli-arguments are necessary, i.e.

x test tests/ui --record
x test --rerun

Will run all failed uitests. No need to pass tests/ui to the rerun invocation.

The last commit is a little awkward, but I think it's the best way to make it so that we first run all tests that have to be rerun, and then rerun tests passed through the cli.

This makes it so:

x test tests/ui --rerun

will first rerun failed tests, some of which may be uitests, if any fail it quits and reports failed tests, but if all pass it will run all normally passed tests. In other words, only if all previously-failed tests pass on the rerun, we then also run uitests.
Without the last commit, this would instead just run all uitests, since the failed tests form a subset of all uitests. I think that's less useful.

@rustbot

Copy link
Copy Markdown
Collaborator

This PR modifies src/bootstrap/src/core/config.

If appropriate, please update CONFIG_CHANGE_HISTORY in src/bootstrap/src/utils/change_tracker.rs.

@rustbotrustbot added A-testsuite Area: The testsuite used to check the correctness of rustc S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) labels Mar 30, 2026
@rustbot

Copy link
Copy Markdown
Collaborator

r? @clubby789

rustbot has assigned @clubby789.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

Why was this reviewer chosen?

The reviewer was selected based on:

  • Owners of files modified in this PR: bootstrap
  • bootstrap expanded to 6 candidates
  • Random selection from Mark-Simulacrum, clubby789, jieyouxu

@rust-log-analyzer

This comment has been minimized.

@jyn514

Copy link
Copy Markdown
Member

Writes a file, by default build/failed-tests, but this can be overwritten with

I wouldn't make this configurable.

What does x test --rerun do if there was no previous --record run? How does cache invalidation work for the failed-tests file, do you just keep it around forever?

@jdonszelmann

Copy link
Copy Markdown
ContributorAuthor

@jyn514

What does x test --rerun do if there was no previous --record run?

It will warn, but treat it as if the file was empty.

How does cache invalidation work for the failed-tests file, do you just keep it around forever?

Keeps it around forever, or at least until you --record again. It prints that it's rerunning, and from what file so it should be easy to delete or even edit manually. Rerunning acts pretty much as if you passed the test paths on the cli, though we do run them before the cli-passed paths.

@rustbot

This comment has been minimized.

@rust-log-analyzer

This comment has been minimized.

@rust-log-analyzer

This comment has been minimized.

@rust-log-analyzer

This comment has been minimized.

@WaffleLapkin

Copy link
Copy Markdown
Member

Rerunning acts pretty much as if you passed the test paths on the cli, though we do run them before the cli-passed paths.

If the build/failed-tests is empty/non-present, does that make x test --rerun run all tests?

@WaffleLapkin

Copy link
Copy Markdown
Member

This is purely a speculation I suppose, but I feel like I will often forget to use --record, before the tests already failed. Would it make sense to save failures of the last command to a separate file and allow moving them to build/failed-tests?

@jyn514

Copy link
Copy Markdown
Member

@WaffleLapkin as long as you don’t modify the compiler, compiletest will remember which tests succeeded the last time, so if you add ––failed it won’t take very long at all to regenerate that list.

@jdonszelmann

Copy link
Copy Markdown
ContributorAuthor

I agree with jyn here, shouldn't make such a big difference

@jdonszelmann

Copy link
Copy Markdown
ContributorAuthor

Rerunning acts pretty much as if you passed the test paths on the cli, though we do run them before the cli-passed paths.

If the build/failed-tests is empty/non-present, does that make x test --rerun run all tests?

that is true I think. What do you expect the behavior to be? maybe if no file is found, and no paths are given, exit, but if some paths are explicitly given run the explicit ones?

@jdonszelmann

Copy link
Copy Markdown
ContributorAuthor

with a warning of course

@jyn514

Copy link
Copy Markdown
Member

I think x test --rerun with no previous --record should behave exactly the same as x test.

@jdonszelmann

Copy link
Copy Markdown
ContributorAuthor

mhm, well that's the current behavior. Except the warning of course, that you passed --rerun with nothing to rerun. But I think that's nice

@jdonszelmann

Copy link
Copy Markdown
ContributorAuthor

cc @jieyouxu (you self assigned the other one, that one was in preparation for this one, also I figured out that bug for this one)

@clubby789

Copy link
Copy Markdown
Contributor

I haven't looked at the full implementation yet, but why not always record failed tests?

@jyn514

jyn514 commented Mar 31, 2026

Copy link
Copy Markdown
Member

Because a later invocation might only rerun a subset of tests. Say you have this series of invocations:

$ x test ui
# ... 34 failures in ui/linkage and ui/attributes
$ x build library # assume there was a compiler change
$ x test --rerun tests/ui/attributes
# ... all tests now pass
$ x test --rerun tests/ui/linkage
# ... no tests are run :( we expected all the failed linkage tests to rerun.

@clubby789

Copy link
Copy Markdown
Contributor

Sorry, always *except when using --rerun 😅

@jieyouxujieyouxu self-assigned this Mar 31, 2026
@rustbotrustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Apr 20, 2026
@rustbot

This comment has been minimized.

@rust-log-analyzer

This comment has been minimized.

@rustbotrustbot removed the S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. label May 12, 2026
@jieyouxujieyouxu reopened this May 12, 2026
@rustbotrustbot added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label May 12, 2026

@jieyouxujieyouxu left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, you can r= clubby and me with one message typo fixed
@rustbot author

View changes since this review

Ok(f) => Some(f),
Err(e) => {
println!(
"Couldn't open file {} to write test failutes to: {e}. (attempted because `--record` was passed). Test failures will not be recorded.",

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: s/failutes/failures/.

@rustbotrustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels May 14, 2026
@jieyouxu

Copy link
Copy Markdown
Member

I thought about #154586 (comment) a bit more, while the parsing is still a bit hacky, it's not that bad since this is more of convenience feature and not correctness-critical (as in, CI will still catch unaddressed failures even if --{record,rerun} isn't fully accurate or drifts).

@rustbot

Copy link
Copy Markdown
Collaborator

This PR was rebased onto a different main commit. Here's a range-diff highlighting what actually changed.

Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers.

@jdonszelmann

Copy link
Copy Markdown
ContributorAuthor

@bors r=clubby789,jieyouxu

@rust-bors

rust-borsBot commented Jun 4, 2026

Copy link
Copy Markdown
Contributor

📌 Commit ab1aa4b has been approved by clubby789,jieyouxu

It is now in the queue for this repository.

@rust-borsrust-borsBot added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Jun 4, 2026
rust-borsBot pushed a commit that referenced this pull request Jun 4, 2026
…uwer
Rollup of 5 pull requests
Successful merges:
- #154586 (Record failed tests with `--record`, and rerun them with `--rerun`)
- #157296 (delegation: split resolution and lowering)
- #156171 (Fix a coroutine UI test which is missing `#[coroutine]`)
- #157249 (tests: codegen-llvm: Update bpf-alu32 with the new LLVM attributes)
- #157426 (rustc-dev-guide subtree update)
@rust-bors
rust-borsBot merged commit bb6f90e into rust-lang:mainJun 4, 2026
12 checks passed
@rustbotrustbot added this to the 1.98.0 milestone Jun 4, 2026
rust-timer added a commit that referenced this pull request Jun 4, 2026
Rollup merge of #154586 - jdonszelmann:record-rerun, r=clubby789,jieyouxu
Record failed tests with `--record`, and rerun them with `--rerun`
This adds two parameters to `x test`:
## `--record`
Writes a file, by default `build/failed-tests`, but this can be overwritten with
```toml
[build]
record_failed_tests_path = "somepath"
```
with a list of all tests that fail that run.
## `--rerun`
Looks for the failed-tests file, parse it, and attempt to rerun only those tests. No cli-arguments are necessary, i.e.
```
x test tests/ui --record
x test --rerun
```
Will run all failed uitests. No need to pass tests/ui to the rerun invocation.
The last commit is a little awkward, but I think it's the best way to make it so that we *first* run all tests that have to be rerun, and *then* rerun tests passed through the cli.
This makes it so:
```
x test tests/ui --rerun
```
will *first* rerun failed tests, some of which may be uitests, if any fail it quits and reports failed tests, but if all pass it will run all normally passed tests. In other words, only if all previously-failed tests pass on the rerun, we then also run uitests.
Without the last commit, this would instead just run all uitests, since the failed tests form a subset of all uitests. I think that's less useful.
github-actionsBot pushed a commit to rust-lang/rustc-dev-guide that referenced this pull request Jun 20, 2026
…uwer
Rollup of 5 pull requests
Successful merges:
- rust-lang/rust#154586 (Record failed tests with `--record`, and rerun them with `--rerun`)
- rust-lang/rust#157296 (delegation: split resolution and lowering)
- rust-lang/rust#156171 (Fix a coroutine UI test which is missing `#[coroutine]`)
- rust-lang/rust#157249 (tests: codegen-llvm: Update bpf-alu32 with the new LLVM attributes)
- rust-lang/rust#157426 (rustc-dev-guide subtree update)
Kobzol pushed a commit to Kobzol/rust that referenced this pull request Jun 21, 2026
…nathanBrouwer
Rollup of 5 pull requests
Successful merges:
- rust-lang#154586 (Record failed tests with `--record`, and rerun them with `--rerun`)
- rust-lang#157296 (delegation: split resolution and lowering)
- rust-lang#156171 (Fix a coroutine UI test which is missing `#[coroutine]`)
- rust-lang#157249 (tests: codegen-llvm: Update bpf-alu32 with the new LLVM attributes)
- rust-lang#157426 (rustc-dev-guide subtree update)
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-testsuiteArea: The testsuite used to check the correctness of rustcS-waiting-on-borsStatus: Waiting on bors to run and complete tests. Bors will change the label on completion.T-bootstrapRelevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants

@jdonszelmann@rustbot@rust-log-analyzer@jyn514@WaffleLapkin@clubby789@jieyouxu
, 'i'); if (__m === '*' || __re.test(location.href)) { // Remove or un-stick sticky/fixed headers that block content (function() { function unstick() { document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) { if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') { el.style.position = 'static'; el.style.top = 'auto'; el.style.zIndex = 'auto'; } }); } unstick(); var observer = new MutationObserver(unstick); observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] }); })(); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Record failed tests with --record, and rerun them with --rerun - #154586

Merged
rust-bors[bot] merged 3 commits into
rust-lang:mainfrom
jdonszelmann:record-rerun
Jun 4, 2026
Merged

Record failed tests with --record, and rerun them with --rerun#154586
rust-bors[bot] merged 3 commits into
rust-lang:mainfrom
jdonszelmann:record-rerun

Conversation

@jdonszelmann

@jdonszelmannjdonszelmann commented Mar 30, 2026

Copy link
Copy Markdown
Contributor

View all comments

This adds two parameters to x test:

--record

Writes a file, by default build/failed-tests, but this can be overwritten with

[build]
record_failed_tests_path = "somepath"

with a list of all tests that fail that run.

--rerun

Looks for the failed-tests file, parse it, and attempt to rerun only those tests. No cli-arguments are necessary, i.e.

x test tests/ui --record
x test --rerun

Will run all failed uitests. No need to pass tests/ui to the rerun invocation.

The last commit is a little awkward, but I think it's the best way to make it so that we first run all tests that have to be rerun, and then rerun tests passed through the cli.

This makes it so:

x test tests/ui --rerun

will first rerun failed tests, some of which may be uitests, if any fail it quits and reports failed tests, but if all pass it will run all normally passed tests. In other words, only if all previously-failed tests pass on the rerun, we then also run uitests.
Without the last commit, this would instead just run all uitests, since the failed tests form a subset of all uitests. I think that's less useful.

@rustbot

Copy link
Copy Markdown
Collaborator

This PR modifies src/bootstrap/src/core/config.

If appropriate, please update CONFIG_CHANGE_HISTORY in src/bootstrap/src/utils/change_tracker.rs.

@rustbotrustbot added A-testsuite Area: The testsuite used to check the correctness of rustc S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) labels Mar 30, 2026
@rustbot

Copy link
Copy Markdown
Collaborator

r? @clubby789

rustbot has assigned @clubby789.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

Why was this reviewer chosen?

The reviewer was selected based on:

  • Owners of files modified in this PR: bootstrap
  • bootstrap expanded to 6 candidates
  • Random selection from Mark-Simulacrum, clubby789, jieyouxu

@rust-log-analyzer

This comment has been minimized.

@jyn514

Copy link
Copy Markdown
Member

Writes a file, by default build/failed-tests, but this can be overwritten with

I wouldn't make this configurable.

What does x test --rerun do if there was no previous --record run? How does cache invalidation work for the failed-tests file, do you just keep it around forever?

@jdonszelmann

Copy link
Copy Markdown
ContributorAuthor

@jyn514

What does x test --rerun do if there was no previous --record run?

It will warn, but treat it as if the file was empty.

How does cache invalidation work for the failed-tests file, do you just keep it around forever?

Keeps it around forever, or at least until you --record again. It prints that it's rerunning, and from what file so it should be easy to delete or even edit manually. Rerunning acts pretty much as if you passed the test paths on the cli, though we do run them before the cli-passed paths.

@rustbot

This comment has been minimized.

@rust-log-analyzer

This comment has been minimized.

@rust-log-analyzer

This comment has been minimized.

@rust-log-analyzer

This comment has been minimized.

@WaffleLapkin

Copy link
Copy Markdown
Member

Rerunning acts pretty much as if you passed the test paths on the cli, though we do run them before the cli-passed paths.

If the build/failed-tests is empty/non-present, does that make x test --rerun run all tests?

@WaffleLapkin

Copy link
Copy Markdown
Member

This is purely a speculation I suppose, but I feel like I will often forget to use --record, before the tests already failed. Would it make sense to save failures of the last command to a separate file and allow moving them to build/failed-tests?

@jyn514

Copy link
Copy Markdown
Member

@WaffleLapkin as long as you don’t modify the compiler, compiletest will remember which tests succeeded the last time, so if you add ––failed it won’t take very long at all to regenerate that list.

@jdonszelmann

Copy link
Copy Markdown
ContributorAuthor

I agree with jyn here, shouldn't make such a big difference

@jdonszelmann

Copy link
Copy Markdown
ContributorAuthor

Rerunning acts pretty much as if you passed the test paths on the cli, though we do run them before the cli-passed paths.

If the build/failed-tests is empty/non-present, does that make x test --rerun run all tests?

that is true I think. What do you expect the behavior to be? maybe if no file is found, and no paths are given, exit, but if some paths are explicitly given run the explicit ones?

@jdonszelmann

Copy link
Copy Markdown
ContributorAuthor

with a warning of course

@jyn514

Copy link
Copy Markdown
Member

I think x test --rerun with no previous --record should behave exactly the same as x test.

@jdonszelmann

Copy link
Copy Markdown
ContributorAuthor

mhm, well that's the current behavior. Except the warning of course, that you passed --rerun with nothing to rerun. But I think that's nice

@jdonszelmann

Copy link
Copy Markdown
ContributorAuthor

cc @jieyouxu (you self assigned the other one, that one was in preparation for this one, also I figured out that bug for this one)

@clubby789

Copy link
Copy Markdown
Contributor

I haven't looked at the full implementation yet, but why not always record failed tests?

@jyn514

jyn514 commented Mar 31, 2026

Copy link
Copy Markdown
Member

Because a later invocation might only rerun a subset of tests. Say you have this series of invocations:

$ x test ui
# ... 34 failures in ui/linkage and ui/attributes
$ x build library # assume there was a compiler change
$ x test --rerun tests/ui/attributes
# ... all tests now pass
$ x test --rerun tests/ui/linkage
# ... no tests are run :( we expected all the failed linkage tests to rerun.

@clubby789

Copy link
Copy Markdown
Contributor

Sorry, always *except when using --rerun 😅

@jieyouxujieyouxu self-assigned this Mar 31, 2026
@rustbotrustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Apr 20, 2026
@rustbot

This comment has been minimized.

@rust-log-analyzer

This comment has been minimized.

@rustbotrustbot removed the S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. label May 12, 2026
@jieyouxujieyouxu reopened this May 12, 2026
@rustbotrustbot added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label May 12, 2026

@jieyouxujieyouxu left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, you can r= clubby and me with one message typo fixed
@rustbot author

View changes since this review

Ok(f) => Some(f),
Err(e) => {
println!(
"Couldn't open file {} to write test failutes to: {e}. (attempted because `--record` was passed). Test failures will not be recorded.",

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: s/failutes/failures/.

@rustbotrustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels May 14, 2026
@jieyouxu

Copy link
Copy Markdown
Member

I thought about #154586 (comment) a bit more, while the parsing is still a bit hacky, it's not that bad since this is more of convenience feature and not correctness-critical (as in, CI will still catch unaddressed failures even if --{record,rerun} isn't fully accurate or drifts).

@rustbot

Copy link
Copy Markdown
Collaborator

This PR was rebased onto a different main commit. Here's a range-diff highlighting what actually changed.

Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers.

@jdonszelmann

Copy link
Copy Markdown
ContributorAuthor

@bors r=clubby789,jieyouxu

@rust-bors

rust-borsBot commented Jun 4, 2026

Copy link
Copy Markdown
Contributor

📌 Commit ab1aa4b has been approved by clubby789,jieyouxu

It is now in the queue for this repository.

@rust-borsrust-borsBot added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Jun 4, 2026
rust-borsBot pushed a commit that referenced this pull request Jun 4, 2026
…uwer
Rollup of 5 pull requests
Successful merges:
- #154586 (Record failed tests with `--record`, and rerun them with `--rerun`)
- #157296 (delegation: split resolution and lowering)
- #156171 (Fix a coroutine UI test which is missing `#[coroutine]`)
- #157249 (tests: codegen-llvm: Update bpf-alu32 with the new LLVM attributes)
- #157426 (rustc-dev-guide subtree update)
@rust-bors
rust-borsBot merged commit bb6f90e into rust-lang:mainJun 4, 2026
12 checks passed
@rustbotrustbot added this to the 1.98.0 milestone Jun 4, 2026
rust-timer added a commit that referenced this pull request Jun 4, 2026
Rollup merge of #154586 - jdonszelmann:record-rerun, r=clubby789,jieyouxu
Record failed tests with `--record`, and rerun them with `--rerun`
This adds two parameters to `x test`:
## `--record`
Writes a file, by default `build/failed-tests`, but this can be overwritten with
```toml
[build]
record_failed_tests_path = "somepath"
```
with a list of all tests that fail that run.
## `--rerun`
Looks for the failed-tests file, parse it, and attempt to rerun only those tests. No cli-arguments are necessary, i.e.
```
x test tests/ui --record
x test --rerun
```
Will run all failed uitests. No need to pass tests/ui to the rerun invocation.
The last commit is a little awkward, but I think it's the best way to make it so that we *first* run all tests that have to be rerun, and *then* rerun tests passed through the cli.
This makes it so:
```
x test tests/ui --rerun
```
will *first* rerun failed tests, some of which may be uitests, if any fail it quits and reports failed tests, but if all pass it will run all normally passed tests. In other words, only if all previously-failed tests pass on the rerun, we then also run uitests.
Without the last commit, this would instead just run all uitests, since the failed tests form a subset of all uitests. I think that's less useful.
github-actionsBot pushed a commit to rust-lang/rustc-dev-guide that referenced this pull request Jun 20, 2026
…uwer
Rollup of 5 pull requests
Successful merges:
- rust-lang/rust#154586 (Record failed tests with `--record`, and rerun them with `--rerun`)
- rust-lang/rust#157296 (delegation: split resolution and lowering)
- rust-lang/rust#156171 (Fix a coroutine UI test which is missing `#[coroutine]`)
- rust-lang/rust#157249 (tests: codegen-llvm: Update bpf-alu32 with the new LLVM attributes)
- rust-lang/rust#157426 (rustc-dev-guide subtree update)
Kobzol pushed a commit to Kobzol/rust that referenced this pull request Jun 21, 2026
…nathanBrouwer
Rollup of 5 pull requests
Successful merges:
- rust-lang#154586 (Record failed tests with `--record`, and rerun them with `--rerun`)
- rust-lang#157296 (delegation: split resolution and lowering)
- rust-lang#156171 (Fix a coroutine UI test which is missing `#[coroutine]`)
- rust-lang#157249 (tests: codegen-llvm: Update bpf-alu32 with the new LLVM attributes)
- rust-lang#157426 (rustc-dev-guide subtree update)
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-testsuiteArea: The testsuite used to check the correctness of rustcS-waiting-on-borsStatus: Waiting on bors to run and complete tests. Bors will change the label on completion.T-bootstrapRelevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants

@jdonszelmann@rustbot@rust-log-analyzer@jyn514@WaffleLapkin@clubby789@jieyouxu
, 'i'); if (__m === '*' || __re.test(location.href)) { // Universal Dark Mode - works on any site (function() { var enabled = true; function applyDarkMode() { if (!enabled) return; // Create style element if it doesn't exist var style = document.getElementById('universal-dark-mode-style'); if (!style) { style = document.createElement('style'); style.id = 'universal-dark-mode-style'; document.head.appendChild(style); } // Dark mode CSS - inverts colors but preserves images/video style.textContent = ' /* Invert everything except media */ html { filter: invert(1) hue-rotate(180deg) !important; background: #1a1a2e !important; } /* Restore images, videos, iframes, canvas */ img, video, iframe, canvas, svg, picture, [style*="background-image"] { filter: invert(1) hue-rotate(180deg) !important; } /* Preserve specific elements that should not be inverted */ .no-dark-mode, .no-dark-mode *, [data-theme="light"], [data-theme="light"], .ace_editor, .ace_editor *, .CodeMirror, .CodeMirror *, .monaco-editor, .monaco-editor *, .markdown-body pre, .markdown-body pre *, .highlight, .highlight *, pre code, pre code * { filter: none !important; } /* Fix common UI elements */ .modal, .popup, .dropdown-menu, .tooltip, .popover { filter: invert(1) hue-rotate(180deg) !important; background: #2d2d44 !important; border-color: #444 !important; } /* Scrollbars */ ::-webkit-scrollbar { background: #1a1a2e !important; } ::-webkit-scrollbar-thumb { background: #444 !important; } ::-webkit-scrollbar-thumb:hover { background: #555 !important; } /* Selection */ ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; } ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; } '; } function removeDarkMode() { var style = document.getElementById('universal-dark-mode-style'); if (style) style.remove(); } // Toggle with Alt+Shift+D document.addEventListener('keydown', function(e) { if (e.altKey && e.shiftKey && e.key === 'D') { e.preventDefault(); enabled = !enabled; if (enabled) { applyDarkMode(); console.log('[Universal Dark Mode] Enabled'); } else { removeDarkMode(); console.log('[Universal Dark Mode] Disabled'); } } }); // Apply on load applyDarkMode(); // Re-apply on dynamic content var observer = new MutationObserver(function(mutations) { if (enabled && !document.getElementById('universal-dark-mode-style')) { applyDarkMode(); } }); observer.observe(document.head, { childList: true }); console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle'); })(); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

Record failed tests with --record, and rerun them with --rerun - #154586

Merged
rust-bors[bot] merged 3 commits into
rust-lang:mainfrom
jdonszelmann:record-rerun
Jun 4, 2026
Merged

Record failed tests with --record, and rerun them with --rerun#154586
rust-bors[bot] merged 3 commits into
rust-lang:mainfrom
jdonszelmann:record-rerun

Conversation

@jdonszelmann

@jdonszelmannjdonszelmann commented Mar 30, 2026

Copy link
Copy Markdown
Contributor

View all comments

This adds two parameters to x test:

--record

Writes a file, by default build/failed-tests, but this can be overwritten with

[build]
record_failed_tests_path = "somepath"

with a list of all tests that fail that run.

--rerun

Looks for the failed-tests file, parse it, and attempt to rerun only those tests. No cli-arguments are necessary, i.e.

x test tests/ui --record
x test --rerun

Will run all failed uitests. No need to pass tests/ui to the rerun invocation.

The last commit is a little awkward, but I think it's the best way to make it so that we first run all tests that have to be rerun, and then rerun tests passed through the cli.

This makes it so:

x test tests/ui --rerun

will first rerun failed tests, some of which may be uitests, if any fail it quits and reports failed tests, but if all pass it will run all normally passed tests. In other words, only if all previously-failed tests pass on the rerun, we then also run uitests.
Without the last commit, this would instead just run all uitests, since the failed tests form a subset of all uitests. I think that's less useful.

@rustbot

Copy link
Copy Markdown
Collaborator

This PR modifies src/bootstrap/src/core/config.

If appropriate, please update CONFIG_CHANGE_HISTORY in src/bootstrap/src/utils/change_tracker.rs.

@rustbotrustbot added A-testsuite Area: The testsuite used to check the correctness of rustc S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) labels Mar 30, 2026
@rustbot

Copy link
Copy Markdown
Collaborator

r? @clubby789

rustbot has assigned @clubby789.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

Why was this reviewer chosen?

The reviewer was selected based on:

  • Owners of files modified in this PR: bootstrap
  • bootstrap expanded to 6 candidates
  • Random selection from Mark-Simulacrum, clubby789, jieyouxu

@rust-log-analyzer

This comment has been minimized.

@jyn514

Copy link
Copy Markdown
Member

Writes a file, by default build/failed-tests, but this can be overwritten with

I wouldn't make this configurable.

What does x test --rerun do if there was no previous --record run? How does cache invalidation work for the failed-tests file, do you just keep it around forever?

@jdonszelmann

Copy link
Copy Markdown
ContributorAuthor

@jyn514

What does x test --rerun do if there was no previous --record run?

It will warn, but treat it as if the file was empty.

How does cache invalidation work for the failed-tests file, do you just keep it around forever?

Keeps it around forever, or at least until you --record again. It prints that it's rerunning, and from what file so it should be easy to delete or even edit manually. Rerunning acts pretty much as if you passed the test paths on the cli, though we do run them before the cli-passed paths.

@rustbot

This comment has been minimized.

@rust-log-analyzer

This comment has been minimized.

@rust-log-analyzer

This comment has been minimized.

@rust-log-analyzer

This comment has been minimized.

@WaffleLapkin

Copy link
Copy Markdown
Member

Rerunning acts pretty much as if you passed the test paths on the cli, though we do run them before the cli-passed paths.

If the build/failed-tests is empty/non-present, does that make x test --rerun run all tests?

@WaffleLapkin

Copy link
Copy Markdown
Member

This is purely a speculation I suppose, but I feel like I will often forget to use --record, before the tests already failed. Would it make sense to save failures of the last command to a separate file and allow moving them to build/failed-tests?

@jyn514

Copy link
Copy Markdown
Member

@WaffleLapkin as long as you don’t modify the compiler, compiletest will remember which tests succeeded the last time, so if you add ––failed it won’t take very long at all to regenerate that list.

@jdonszelmann

Copy link
Copy Markdown
ContributorAuthor

I agree with jyn here, shouldn't make such a big difference

@jdonszelmann

Copy link
Copy Markdown
ContributorAuthor

Rerunning acts pretty much as if you passed the test paths on the cli, though we do run them before the cli-passed paths.

If the build/failed-tests is empty/non-present, does that make x test --rerun run all tests?

that is true I think. What do you expect the behavior to be? maybe if no file is found, and no paths are given, exit, but if some paths are explicitly given run the explicit ones?

@jdonszelmann

Copy link
Copy Markdown
ContributorAuthor

with a warning of course

@jyn514

Copy link
Copy Markdown
Member

I think x test --rerun with no previous --record should behave exactly the same as x test.

@jdonszelmann

Copy link
Copy Markdown
ContributorAuthor

mhm, well that's the current behavior. Except the warning of course, that you passed --rerun with nothing to rerun. But I think that's nice

@jdonszelmann

Copy link
Copy Markdown
ContributorAuthor

cc @jieyouxu (you self assigned the other one, that one was in preparation for this one, also I figured out that bug for this one)

@clubby789

Copy link
Copy Markdown
Contributor

I haven't looked at the full implementation yet, but why not always record failed tests?

@jyn514

jyn514 commented Mar 31, 2026

Copy link
Copy Markdown
Member

Because a later invocation might only rerun a subset of tests. Say you have this series of invocations:

$ x test ui
# ... 34 failures in ui/linkage and ui/attributes
$ x build library # assume there was a compiler change
$ x test --rerun tests/ui/attributes
# ... all tests now pass
$ x test --rerun tests/ui/linkage
# ... no tests are run :( we expected all the failed linkage tests to rerun.

@clubby789

Copy link
Copy Markdown
Contributor

Sorry, always *except when using --rerun 😅

@jieyouxujieyouxu self-assigned this Mar 31, 2026
@rustbotrustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Apr 20, 2026
@rustbot

This comment has been minimized.

@rust-log-analyzer

This comment has been minimized.

@rustbotrustbot removed the S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. label May 12, 2026
@jieyouxujieyouxu reopened this May 12, 2026
@rustbotrustbot added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label May 12, 2026

@jieyouxujieyouxu left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, you can r= clubby and me with one message typo fixed
@rustbot author

View changes since this review

Ok(f) => Some(f),
Err(e) => {
println!(
"Couldn't open file {} to write test failutes to: {e}. (attempted because `--record` was passed). Test failures will not be recorded.",

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: s/failutes/failures/.

@rustbotrustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels May 14, 2026
@jieyouxu

Copy link
Copy Markdown
Member

I thought about #154586 (comment) a bit more, while the parsing is still a bit hacky, it's not that bad since this is more of convenience feature and not correctness-critical (as in, CI will still catch unaddressed failures even if --{record,rerun} isn't fully accurate or drifts).

@rustbot

Copy link
Copy Markdown
Collaborator

This PR was rebased onto a different main commit. Here's a range-diff highlighting what actually changed.

Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers.

@jdonszelmann

Copy link
Copy Markdown
ContributorAuthor

@bors r=clubby789,jieyouxu

@rust-bors

rust-borsBot commented Jun 4, 2026

Copy link
Copy Markdown
Contributor

📌 Commit ab1aa4b has been approved by clubby789,jieyouxu

It is now in the queue for this repository.

@rust-borsrust-borsBot added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Jun 4, 2026
rust-borsBot pushed a commit that referenced this pull request Jun 4, 2026
…uwer
Rollup of 5 pull requests
Successful merges:
- #154586 (Record failed tests with `--record`, and rerun them with `--rerun`)
- #157296 (delegation: split resolution and lowering)
- #156171 (Fix a coroutine UI test which is missing `#[coroutine]`)
- #157249 (tests: codegen-llvm: Update bpf-alu32 with the new LLVM attributes)
- #157426 (rustc-dev-guide subtree update)
@rust-bors
rust-borsBot merged commit bb6f90e into rust-lang:mainJun 4, 2026
12 checks passed
@rustbotrustbot added this to the 1.98.0 milestone Jun 4, 2026
rust-timer added a commit that referenced this pull request Jun 4, 2026
Rollup merge of #154586 - jdonszelmann:record-rerun, r=clubby789,jieyouxu
Record failed tests with `--record`, and rerun them with `--rerun`
This adds two parameters to `x test`:
## `--record`
Writes a file, by default `build/failed-tests`, but this can be overwritten with
```toml
[build]
record_failed_tests_path = "somepath"
```
with a list of all tests that fail that run.
## `--rerun`
Looks for the failed-tests file, parse it, and attempt to rerun only those tests. No cli-arguments are necessary, i.e.
```
x test tests/ui --record
x test --rerun
```
Will run all failed uitests. No need to pass tests/ui to the rerun invocation.
The last commit is a little awkward, but I think it's the best way to make it so that we *first* run all tests that have to be rerun, and *then* rerun tests passed through the cli.
This makes it so:
```
x test tests/ui --rerun
```
will *first* rerun failed tests, some of which may be uitests, if any fail it quits and reports failed tests, but if all pass it will run all normally passed tests. In other words, only if all previously-failed tests pass on the rerun, we then also run uitests.
Without the last commit, this would instead just run all uitests, since the failed tests form a subset of all uitests. I think that's less useful.
github-actionsBot pushed a commit to rust-lang/rustc-dev-guide that referenced this pull request Jun 20, 2026
…uwer
Rollup of 5 pull requests
Successful merges:
- rust-lang/rust#154586 (Record failed tests with `--record`, and rerun them with `--rerun`)
- rust-lang/rust#157296 (delegation: split resolution and lowering)
- rust-lang/rust#156171 (Fix a coroutine UI test which is missing `#[coroutine]`)
- rust-lang/rust#157249 (tests: codegen-llvm: Update bpf-alu32 with the new LLVM attributes)
- rust-lang/rust#157426 (rustc-dev-guide subtree update)
Kobzol pushed a commit to Kobzol/rust that referenced this pull request Jun 21, 2026
…nathanBrouwer
Rollup of 5 pull requests
Successful merges:
- rust-lang#154586 (Record failed tests with `--record`, and rerun them with `--rerun`)
- rust-lang#157296 (delegation: split resolution and lowering)
- rust-lang#156171 (Fix a coroutine UI test which is missing `#[coroutine]`)
- rust-lang#157249 (tests: codegen-llvm: Update bpf-alu32 with the new LLVM attributes)
- rust-lang#157426 (rustc-dev-guide subtree update)
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-testsuiteArea: The testsuite used to check the correctness of rustcS-waiting-on-borsStatus: Waiting on bors to run and complete tests. Bors will change the label on completion.T-bootstrapRelevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants

@jdonszelmann@rustbot@rust-log-analyzer@jyn514@WaffleLapkin@clubby789@jieyouxu