libtest: expose --fail-fast as an unstable command-line option

[sourcefrog]

This exposes the fail_fast option added in #105153 on the test harness command line, so that workflows that only want to know if any test fails can find out without waiting for everything to run. For example, cargo-mutants just needs to know if any tests fails. It only works with -Zunstable-options.

Tracking issue: #142859

[rustbot]

r? @SparrowLii

rustbot has assigned @SparrowLii.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

[sourcefrog]

I can not yet find any in-tree tests for the libtest command line options...

[sourcefrog]

I locally tested this with rustup toolchain link, and the basic function works as intended: after one test fails, no other tests start.

However that does find a problem that doctests don't accept this option.

[jieyouxu]

cc @rust-lang/testing-devex
r? libs-api (libtest cli)

[epage]

Over at rust-lang/testing-devex-team#5, we're building up a list of all of the problems with the current doctest architecture.

I've noted this for the next testing-devex meeting.

[epage]

As this builds on the original implementation and it was recognized as a hack by both the reviewer and the implementer, it would be good to get their feedback. My main concern is I've found PRs a poor place for collecting requirements / design information because they can come and go as different people work on it or even with the same author. Unsure if we should go ahead and do that discussion here or open an issue.

[jieyouxu]

(It's probably best as its own issue, PR discussions are very easy to become lost)

[sourcefrog]

OK, thanks, I created #142859 for the feature request. I looked but could not find an existing one.

---

After a bit of investigation I suspect the doctest failure might be something about doctest not linking the same library that's should have been used by cargo toolchain link. It doesn't seem to have its own runner or command line parser.

[epage]

FCP has closed on rust-lang/libs-team#633, putting the CLI decision within the responsibility of @rust-lang/testing-devex. We previously discussed this in a meeting and recorded that decision at #142859 (comment).

As we're new to this, I assume this then needs an implementation sign off from a T-libs member?

[dtolnay]

Thanks!
@bors r+

[bors]

📌 Commit 547c729 has been approved by dtolnay

It is now in the queue for this repository.

[sourcefrog]

Thanks, that's great!

I'll follow up about #142807 (comment). It's been a little while but from memory I think doctest targets fail if they see this option. Since it's marked unstable I hope we can just fix forward?

[sourcefrog]

This exposes the fail_fast option added in #105153 on the test harness command line, so that workflows that only want to know if any test fails can find out without waiting for everything to run. For example, cargo-mutants just needs to know if any tests fails. It only works with -Zunstable-options.

Tracking issue: #142859

[rustbot]

r? @SparrowLii

rustbot has assigned @SparrowLii.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

[sourcefrog]

I can not yet find any in-tree tests for the libtest command line options...

[sourcefrog]

I locally tested this with rustup toolchain link, and the basic function works as intended: after one test fails, no other tests start.

However that does find a problem that doctests don't accept this option.

[jieyouxu]

cc @rust-lang/testing-devex
r? libs-api (libtest cli)

[epage]

Over at rust-lang/testing-devex-team#5, we're building up a list of all of the problems with the current doctest architecture.

I've noted this for the next testing-devex meeting.

[epage]

As this builds on the original implementation and it was recognized as a hack by both the reviewer and the implementer, it would be good to get their feedback. My main concern is I've found PRs a poor place for collecting requirements / design information because they can come and go as different people work on it or even with the same author. Unsure if we should go ahead and do that discussion here or open an issue.

[jieyouxu]

(It's probably best as its own issue, PR discussions are very easy to become lost)

[sourcefrog]

OK, thanks, I created #142859 for the feature request. I looked but could not find an existing one.

---

After a bit of investigation I suspect the doctest failure might be something about doctest not linking the same library that's should have been used by cargo toolchain link. It doesn't seem to have its own runner or command line parser.

[epage]

FCP has closed on rust-lang/libs-team#633, putting the CLI decision within the responsibility of @rust-lang/testing-devex. We previously discussed this in a meeting and recorded that decision at #142859 (comment).

As we're new to this, I assume this then needs an implementation sign off from a T-libs member?

[dtolnay]

Thanks!
@bors r+

[bors]

📌 Commit 547c729 has been approved by dtolnay

It is now in the queue for this repository.

[sourcefrog]

Thanks, that's great!

I'll follow up about #142807 (comment). It's been a little while but from memory I think doctest targets fail if they see this option. Since it's marked unstable I hope we can just fix forward?

[sourcefrog]

This exposes the fail_fast option added in #105153 on the test harness command line, so that workflows that only want to know if any test fails can find out without waiting for everything to run. For example, cargo-mutants just needs to know if any tests fails. It only works with -Zunstable-options.

Tracking issue: #142859

[rustbot]

r? @SparrowLii

rustbot has assigned @SparrowLii.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

[sourcefrog]

I can not yet find any in-tree tests for the libtest command line options...

[sourcefrog]

I locally tested this with rustup toolchain link, and the basic function works as intended: after one test fails, no other tests start.

However that does find a problem that doctests don't accept this option.

[jieyouxu]

cc @rust-lang/testing-devex
r? libs-api (libtest cli)

[epage]

Over at rust-lang/testing-devex-team#5, we're building up a list of all of the problems with the current doctest architecture.

I've noted this for the next testing-devex meeting.

[epage]

As this builds on the original implementation and it was recognized as a hack by both the reviewer and the implementer, it would be good to get their feedback. My main concern is I've found PRs a poor place for collecting requirements / design information because they can come and go as different people work on it or even with the same author. Unsure if we should go ahead and do that discussion here or open an issue.

[jieyouxu]

(It's probably best as its own issue, PR discussions are very easy to become lost)

[sourcefrog]

OK, thanks, I created #142859 for the feature request. I looked but could not find an existing one.

---

After a bit of investigation I suspect the doctest failure might be something about doctest not linking the same library that's should have been used by cargo toolchain link. It doesn't seem to have its own runner or command line parser.

[epage]

FCP has closed on rust-lang/libs-team#633, putting the CLI decision within the responsibility of @rust-lang/testing-devex. We previously discussed this in a meeting and recorded that decision at #142859 (comment).

As we're new to this, I assume this then needs an implementation sign off from a T-libs member?

[dtolnay]

Thanks!
@bors r+

[bors]

📌 Commit 547c729 has been approved by dtolnay

It is now in the queue for this repository.

[sourcefrog]

Thanks, that's great!

I'll follow up about #142807 (comment). It's been a little while but from memory I think doctest targets fail if they see this option. Since it's marked unstable I hope we can just fix forward?

[sourcefrog]

This exposes the fail_fast option added in #105153 on the test harness command line, so that workflows that only want to know if any test fails can find out without waiting for everything to run. For example, cargo-mutants just needs to know if any tests fails. It only works with -Zunstable-options.

Tracking issue: #142859

[rustbot]

r? @SparrowLii

rustbot has assigned @SparrowLii.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

[sourcefrog]

I can not yet find any in-tree tests for the libtest command line options...

[sourcefrog]

I locally tested this with rustup toolchain link, and the basic function works as intended: after one test fails, no other tests start.

However that does find a problem that doctests don't accept this option.

[jieyouxu]

cc @rust-lang/testing-devex
r? libs-api (libtest cli)

[epage]

Over at rust-lang/testing-devex-team#5, we're building up a list of all of the problems with the current doctest architecture.

I've noted this for the next testing-devex meeting.

[epage]

As this builds on the original implementation and it was recognized as a hack by both the reviewer and the implementer, it would be good to get their feedback. My main concern is I've found PRs a poor place for collecting requirements / design information because they can come and go as different people work on it or even with the same author. Unsure if we should go ahead and do that discussion here or open an issue.

[jieyouxu]

(It's probably best as its own issue, PR discussions are very easy to become lost)

[sourcefrog]

OK, thanks, I created #142859 for the feature request. I looked but could not find an existing one.

---

After a bit of investigation I suspect the doctest failure might be something about doctest not linking the same library that's should have been used by cargo toolchain link. It doesn't seem to have its own runner or command line parser.

[epage]

FCP has closed on rust-lang/libs-team#633, putting the CLI decision within the responsibility of @rust-lang/testing-devex. We previously discussed this in a meeting and recorded that decision at #142859 (comment).

As we're new to this, I assume this then needs an implementation sign off from a T-libs member?

[dtolnay]

Thanks!
@bors r+

[bors]

📌 Commit 547c729 has been approved by dtolnay

It is now in the queue for this repository.

[sourcefrog]

Thanks, that's great!

I'll follow up about #142807 (comment). It's been a little while but from memory I think doctest targets fail if they see this option. Since it's marked unstable I hope we can just fix forward?

[sourcefrog]

This exposes the fail_fast option added in #105153 on the test harness command line, so that workflows that only want to know if any test fails can find out without waiting for everything to run. For example, cargo-mutants just needs to know if any tests fails. It only works with -Zunstable-options.

Tracking issue: #142859

[rustbot]

r? @SparrowLii

rustbot has assigned @SparrowLii.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

[sourcefrog]

I can not yet find any in-tree tests for the libtest command line options...

[sourcefrog]

I locally tested this with rustup toolchain link, and the basic function works as intended: after one test fails, no other tests start.

However that does find a problem that doctests don't accept this option.

[jieyouxu]

cc @rust-lang/testing-devex
r? libs-api (libtest cli)

[epage]

Over at rust-lang/testing-devex-team#5, we're building up a list of all of the problems with the current doctest architecture.

I've noted this for the next testing-devex meeting.

[epage]

As this builds on the original implementation and it was recognized as a hack by both the reviewer and the implementer, it would be good to get their feedback. My main concern is I've found PRs a poor place for collecting requirements / design information because they can come and go as different people work on it or even with the same author. Unsure if we should go ahead and do that discussion here or open an issue.

[jieyouxu]

(It's probably best as its own issue, PR discussions are very easy to become lost)

[sourcefrog]

OK, thanks, I created #142859 for the feature request. I looked but could not find an existing one.

---

After a bit of investigation I suspect the doctest failure might be something about doctest not linking the same library that's should have been used by cargo toolchain link. It doesn't seem to have its own runner or command line parser.

[epage]

FCP has closed on rust-lang/libs-team#633, putting the CLI decision within the responsibility of @rust-lang/testing-devex. We previously discussed this in a meeting and recorded that decision at #142859 (comment).

As we're new to this, I assume this then needs an implementation sign off from a T-libs member?

[dtolnay]

Thanks!
@bors r+

[bors]

📌 Commit 547c729 has been approved by dtolnay

It is now in the queue for this repository.

[sourcefrog]

Thanks, that's great!

I'll follow up about #142807 (comment). It's been a little while but from memory I think doctest targets fail if they see this option. Since it's marked unstable I hope we can just fix forward?

[sourcefrog]

This exposes the fail_fast option added in #105153 on the test harness command line, so that workflows that only want to know if any test fails can find out without waiting for everything to run. For example, cargo-mutants just needs to know if any tests fails. It only works with -Zunstable-options.

Tracking issue: #142859

[rustbot]

r? @SparrowLii

rustbot has assigned @SparrowLii.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

[sourcefrog]

I can not yet find any in-tree tests for the libtest command line options...

[sourcefrog]

I locally tested this with rustup toolchain link, and the basic function works as intended: after one test fails, no other tests start.

However that does find a problem that doctests don't accept this option.

[jieyouxu]

cc @rust-lang/testing-devex
r? libs-api (libtest cli)

[epage]

Over at rust-lang/testing-devex-team#5, we're building up a list of all of the problems with the current doctest architecture.

I've noted this for the next testing-devex meeting.

[epage]

As this builds on the original implementation and it was recognized as a hack by both the reviewer and the implementer, it would be good to get their feedback. My main concern is I've found PRs a poor place for collecting requirements / design information because they can come and go as different people work on it or even with the same author. Unsure if we should go ahead and do that discussion here or open an issue.

[jieyouxu]

(It's probably best as its own issue, PR discussions are very easy to become lost)

[sourcefrog]

OK, thanks, I created #142859 for the feature request. I looked but could not find an existing one.

---

After a bit of investigation I suspect the doctest failure might be something about doctest not linking the same library that's should have been used by cargo toolchain link. It doesn't seem to have its own runner or command line parser.

[epage]

FCP has closed on rust-lang/libs-team#633, putting the CLI decision within the responsibility of @rust-lang/testing-devex. We previously discussed this in a meeting and recorded that decision at #142859 (comment).

As we're new to this, I assume this then needs an implementation sign off from a T-libs member?

[dtolnay]

Thanks!
@bors r+

[bors]

📌 Commit 547c729 has been approved by dtolnay

It is now in the queue for this repository.

[sourcefrog]

Thanks, that's great!

I'll follow up about #142807 (comment). It's been a little while but from memory I think doctest targets fail if they see this option. Since it's marked unstable I hope we can just fix forward?

[sourcefrog]

This exposes the fail_fast option added in #105153 on the test harness command line, so that workflows that only want to know if any test fails can find out without waiting for everything to run. For example, cargo-mutants just needs to know if any tests fails. It only works with -Zunstable-options.

Tracking issue: #142859

[rustbot]

r? @SparrowLii

rustbot has assigned @SparrowLii.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

[sourcefrog]

I can not yet find any in-tree tests for the libtest command line options...

[sourcefrog]

I locally tested this with rustup toolchain link, and the basic function works as intended: after one test fails, no other tests start.

However that does find a problem that doctests don't accept this option.

[jieyouxu]

cc @rust-lang/testing-devex
r? libs-api (libtest cli)

[epage]

Over at rust-lang/testing-devex-team#5, we're building up a list of all of the problems with the current doctest architecture.

I've noted this for the next testing-devex meeting.

[epage]

As this builds on the original implementation and it was recognized as a hack by both the reviewer and the implementer, it would be good to get their feedback. My main concern is I've found PRs a poor place for collecting requirements / design information because they can come and go as different people work on it or even with the same author. Unsure if we should go ahead and do that discussion here or open an issue.

[jieyouxu]

(It's probably best as its own issue, PR discussions are very easy to become lost)

[sourcefrog]

OK, thanks, I created #142859 for the feature request. I looked but could not find an existing one.

---

After a bit of investigation I suspect the doctest failure might be something about doctest not linking the same library that's should have been used by cargo toolchain link. It doesn't seem to have its own runner or command line parser.

[epage]

FCP has closed on rust-lang/libs-team#633, putting the CLI decision within the responsibility of @rust-lang/testing-devex. We previously discussed this in a meeting and recorded that decision at #142859 (comment).

As we're new to this, I assume this then needs an implementation sign off from a T-libs member?

[dtolnay]

Thanks!
@bors r+

[bors]

📌 Commit 547c729 has been approved by dtolnay

It is now in the queue for this repository.

[sourcefrog]

Thanks, that's great!

I'll follow up about #142807 (comment). It's been a little while but from memory I think doctest targets fail if they see this option. Since it's marked unstable I hope we can just fix forward?

[sourcefrog]

This exposes the fail_fast option added in #105153 on the test harness command line, so that workflows that only want to know if any test fails can find out without waiting for everything to run. For example, cargo-mutants just needs to know if any tests fails. It only works with -Zunstable-options.

Tracking issue: #142859

[rustbot]

r? @SparrowLii

rustbot has assigned @SparrowLii.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

[sourcefrog]

I can not yet find any in-tree tests for the libtest command line options...

[sourcefrog]

I locally tested this with rustup toolchain link, and the basic function works as intended: after one test fails, no other tests start.

However that does find a problem that doctests don't accept this option.

[jieyouxu]

cc @rust-lang/testing-devex
r? libs-api (libtest cli)

[epage]

Over at rust-lang/testing-devex-team#5, we're building up a list of all of the problems with the current doctest architecture.

I've noted this for the next testing-devex meeting.

[epage]

As this builds on the original implementation and it was recognized as a hack by both the reviewer and the implementer, it would be good to get their feedback. My main concern is I've found PRs a poor place for collecting requirements / design information because they can come and go as different people work on it or even with the same author. Unsure if we should go ahead and do that discussion here or open an issue.

[jieyouxu]

(It's probably best as its own issue, PR discussions are very easy to become lost)

[sourcefrog]

OK, thanks, I created #142859 for the feature request. I looked but could not find an existing one.

---

After a bit of investigation I suspect the doctest failure might be something about doctest not linking the same library that's should have been used by cargo toolchain link. It doesn't seem to have its own runner or command line parser.

[epage]

FCP has closed on rust-lang/libs-team#633, putting the CLI decision within the responsibility of @rust-lang/testing-devex. We previously discussed this in a meeting and recorded that decision at #142859 (comment).

As we're new to this, I assume this then needs an implementation sign off from a T-libs member?

[dtolnay]

Thanks!
@bors r+

[bors]

📌 Commit 547c729 has been approved by dtolnay

It is now in the queue for this repository.

[sourcefrog]

Thanks, that's great!

I'll follow up about #142807 (comment). It's been a little while but from memory I think doctest targets fail if they see this option. Since it's marked unstable I hope we can just fix forward?