Replace tape with tap

[NatalieWolfe]

The tap module is a more fully featured TAP framework that executes its tests in separate child processes.

Resolves#1123

[bnoordhuis]

Thanks, this is great! I wonder though, does tap.test() actually run tests in a separate process? I glean from node-tap's source that only tap.spawn() does that.

[NatalieWolfe]

Correct, tap.test doesn't run each in their own process, but $ tap test/test-* will spawn a separate child process for each file that matched. Was it per-individual-test that needed process isolation?

[bnoordhuis]

Yes.

[NatalieWolfe]

Is it possible to move the tests which need isolation into individual files? Which tests need this? I can look into doing that step as part of this ticket.

[bnoordhuis]

I'm going to say 'all of them' - it's not really true but isolation never hurts. It's my original plan of attack for #1123.

[NatalieWolfe]

Is that necessary up front? That seems like the kind of thing that could be resolved on a case-by-case basis when colliding tests are discovered. Moving individual tests into separate files isn't that hard, but seems like overkill if you don't actually need them isolated.

[bnoordhuis]

Such things tend to go unnoticed, though. I raised #1123 after I found two unrelated tests had been clobbering each other undetected for some time.

Another anecdote: a project I was involved in had a bug caused by lack of isolation that stopped half the tests from running but it went undiscovered for a year because the CI was green all the time.

[refack]

IMHO need to add -Rtap

[refack]

@NatalieWolfe do you need help? I can help...

[maclover7]

ping @NatalieWolfe

[rvagg]

needs to be updated, either @NatalieWolfe needs to revisit or someone else should open a PR, I'd +1 on merging this if it was ready

[cclauss]

@NatalieWolfe Is this already done elsewhere or is there more to do here?

[richardlau]

This was done in #1795.

[NatalieWolfe]

The tap module is a more fully featured TAP framework that executes its tests in separate child processes.

Resolves#1123

[bnoordhuis]

Thanks, this is great! I wonder though, does tap.test() actually run tests in a separate process? I glean from node-tap's source that only tap.spawn() does that.

[NatalieWolfe]

Correct, tap.test doesn't run each in their own process, but $ tap test/test-* will spawn a separate child process for each file that matched. Was it per-individual-test that needed process isolation?

[bnoordhuis]

Yes.

[NatalieWolfe]

Is it possible to move the tests which need isolation into individual files? Which tests need this? I can look into doing that step as part of this ticket.

[bnoordhuis]

I'm going to say 'all of them' - it's not really true but isolation never hurts. It's my original plan of attack for #1123.

[NatalieWolfe]

Is that necessary up front? That seems like the kind of thing that could be resolved on a case-by-case basis when colliding tests are discovered. Moving individual tests into separate files isn't that hard, but seems like overkill if you don't actually need them isolated.

[bnoordhuis]

Such things tend to go unnoticed, though. I raised #1123 after I found two unrelated tests had been clobbering each other undetected for some time.

Another anecdote: a project I was involved in had a bug caused by lack of isolation that stopped half the tests from running but it went undiscovered for a year because the CI was green all the time.

[refack]

IMHO need to add -Rtap

[refack]

@NatalieWolfe do you need help? I can help...

[maclover7]

ping @NatalieWolfe

[rvagg]

needs to be updated, either @NatalieWolfe needs to revisit or someone else should open a PR, I'd +1 on merging this if it was ready

[cclauss]

@NatalieWolfe Is this already done elsewhere or is there more to do here?

[richardlau]

This was done in #1795.

[NatalieWolfe]

The tap module is a more fully featured TAP framework that executes its tests in separate child processes.

Resolves#1123

[bnoordhuis]

Thanks, this is great! I wonder though, does tap.test() actually run tests in a separate process? I glean from node-tap's source that only tap.spawn() does that.

[NatalieWolfe]

Correct, tap.test doesn't run each in their own process, but $ tap test/test-* will spawn a separate child process for each file that matched. Was it per-individual-test that needed process isolation?

[bnoordhuis]

Yes.

[NatalieWolfe]

Is it possible to move the tests which need isolation into individual files? Which tests need this? I can look into doing that step as part of this ticket.

[bnoordhuis]

I'm going to say 'all of them' - it's not really true but isolation never hurts. It's my original plan of attack for #1123.

[NatalieWolfe]

Is that necessary up front? That seems like the kind of thing that could be resolved on a case-by-case basis when colliding tests are discovered. Moving individual tests into separate files isn't that hard, but seems like overkill if you don't actually need them isolated.

[bnoordhuis]

Such things tend to go unnoticed, though. I raised #1123 after I found two unrelated tests had been clobbering each other undetected for some time.

Another anecdote: a project I was involved in had a bug caused by lack of isolation that stopped half the tests from running but it went undiscovered for a year because the CI was green all the time.

[refack]

IMHO need to add -Rtap

[refack]

@NatalieWolfe do you need help? I can help...

[maclover7]

ping @NatalieWolfe

[rvagg]

needs to be updated, either @NatalieWolfe needs to revisit or someone else should open a PR, I'd +1 on merging this if it was ready

[cclauss]

@NatalieWolfe Is this already done elsewhere or is there more to do here?

[richardlau]

This was done in #1795.

[NatalieWolfe]

The tap module is a more fully featured TAP framework that executes its tests in separate child processes.

Resolves#1123

[bnoordhuis]

Thanks, this is great! I wonder though, does tap.test() actually run tests in a separate process? I glean from node-tap's source that only tap.spawn() does that.

[NatalieWolfe]

Correct, tap.test doesn't run each in their own process, but $ tap test/test-* will spawn a separate child process for each file that matched. Was it per-individual-test that needed process isolation?

[bnoordhuis]

Yes.

[NatalieWolfe]

Is it possible to move the tests which need isolation into individual files? Which tests need this? I can look into doing that step as part of this ticket.

[bnoordhuis]

I'm going to say 'all of them' - it's not really true but isolation never hurts. It's my original plan of attack for #1123.

[NatalieWolfe]

Is that necessary up front? That seems like the kind of thing that could be resolved on a case-by-case basis when colliding tests are discovered. Moving individual tests into separate files isn't that hard, but seems like overkill if you don't actually need them isolated.

[bnoordhuis]

Such things tend to go unnoticed, though. I raised #1123 after I found two unrelated tests had been clobbering each other undetected for some time.

Another anecdote: a project I was involved in had a bug caused by lack of isolation that stopped half the tests from running but it went undiscovered for a year because the CI was green all the time.

[refack]

IMHO need to add -Rtap

[refack]

@NatalieWolfe do you need help? I can help...

[maclover7]

ping @NatalieWolfe

[rvagg]

needs to be updated, either @NatalieWolfe needs to revisit or someone else should open a PR, I'd +1 on merging this if it was ready

[cclauss]

@NatalieWolfe Is this already done elsewhere or is there more to do here?

[richardlau]

This was done in #1795.

[NatalieWolfe]

The tap module is a more fully featured TAP framework that executes its tests in separate child processes.

Resolves#1123

[bnoordhuis]

Thanks, this is great! I wonder though, does tap.test() actually run tests in a separate process? I glean from node-tap's source that only tap.spawn() does that.

[NatalieWolfe]

Correct, tap.test doesn't run each in their own process, but $ tap test/test-* will spawn a separate child process for each file that matched. Was it per-individual-test that needed process isolation?

[bnoordhuis]

Yes.

[NatalieWolfe]

Is it possible to move the tests which need isolation into individual files? Which tests need this? I can look into doing that step as part of this ticket.

[bnoordhuis]

I'm going to say 'all of them' - it's not really true but isolation never hurts. It's my original plan of attack for #1123.

[NatalieWolfe]

Is that necessary up front? That seems like the kind of thing that could be resolved on a case-by-case basis when colliding tests are discovered. Moving individual tests into separate files isn't that hard, but seems like overkill if you don't actually need them isolated.

[bnoordhuis]

Such things tend to go unnoticed, though. I raised #1123 after I found two unrelated tests had been clobbering each other undetected for some time.

Another anecdote: a project I was involved in had a bug caused by lack of isolation that stopped half the tests from running but it went undiscovered for a year because the CI was green all the time.

[refack]

IMHO need to add -Rtap

[refack]

@NatalieWolfe do you need help? I can help...

[maclover7]

ping @NatalieWolfe

[rvagg]

needs to be updated, either @NatalieWolfe needs to revisit or someone else should open a PR, I'd +1 on merging this if it was ready

[cclauss]

@NatalieWolfe Is this already done elsewhere or is there more to do here?

[richardlau]

This was done in #1795.

[NatalieWolfe]

The tap module is a more fully featured TAP framework that executes its tests in separate child processes.

Resolves#1123

[bnoordhuis]

Thanks, this is great! I wonder though, does tap.test() actually run tests in a separate process? I glean from node-tap's source that only tap.spawn() does that.

[NatalieWolfe]

Correct, tap.test doesn't run each in their own process, but $ tap test/test-* will spawn a separate child process for each file that matched. Was it per-individual-test that needed process isolation?

[bnoordhuis]

Yes.

[NatalieWolfe]

Is it possible to move the tests which need isolation into individual files? Which tests need this? I can look into doing that step as part of this ticket.

[bnoordhuis]

I'm going to say 'all of them' - it's not really true but isolation never hurts. It's my original plan of attack for #1123.

[NatalieWolfe]

Is that necessary up front? That seems like the kind of thing that could be resolved on a case-by-case basis when colliding tests are discovered. Moving individual tests into separate files isn't that hard, but seems like overkill if you don't actually need them isolated.

[bnoordhuis]

Such things tend to go unnoticed, though. I raised #1123 after I found two unrelated tests had been clobbering each other undetected for some time.

Another anecdote: a project I was involved in had a bug caused by lack of isolation that stopped half the tests from running but it went undiscovered for a year because the CI was green all the time.

[refack]

IMHO need to add -Rtap

[refack]

@NatalieWolfe do you need help? I can help...

[maclover7]

ping @NatalieWolfe

[rvagg]

needs to be updated, either @NatalieWolfe needs to revisit or someone else should open a PR, I'd +1 on merging this if it was ready

[cclauss]

@NatalieWolfe Is this already done elsewhere or is there more to do here?

[richardlau]

This was done in #1795.

[NatalieWolfe]

The tap module is a more fully featured TAP framework that executes its tests in separate child processes.

Resolves#1123

[bnoordhuis]

Thanks, this is great! I wonder though, does tap.test() actually run tests in a separate process? I glean from node-tap's source that only tap.spawn() does that.

[NatalieWolfe]

Correct, tap.test doesn't run each in their own process, but $ tap test/test-* will spawn a separate child process for each file that matched. Was it per-individual-test that needed process isolation?

[bnoordhuis]

Yes.

[NatalieWolfe]

Is it possible to move the tests which need isolation into individual files? Which tests need this? I can look into doing that step as part of this ticket.

[bnoordhuis]

I'm going to say 'all of them' - it's not really true but isolation never hurts. It's my original plan of attack for #1123.

[NatalieWolfe]

Is that necessary up front? That seems like the kind of thing that could be resolved on a case-by-case basis when colliding tests are discovered. Moving individual tests into separate files isn't that hard, but seems like overkill if you don't actually need them isolated.

[bnoordhuis]

Such things tend to go unnoticed, though. I raised #1123 after I found two unrelated tests had been clobbering each other undetected for some time.

Another anecdote: a project I was involved in had a bug caused by lack of isolation that stopped half the tests from running but it went undiscovered for a year because the CI was green all the time.

[refack]

IMHO need to add -Rtap

[refack]

@NatalieWolfe do you need help? I can help...

[maclover7]

ping @NatalieWolfe

[rvagg]

needs to be updated, either @NatalieWolfe needs to revisit or someone else should open a PR, I'd +1 on merging this if it was ready

[cclauss]

@NatalieWolfe Is this already done elsewhere or is there more to do here?

[richardlau]

This was done in #1795.

[NatalieWolfe]

The tap module is a more fully featured TAP framework that executes its tests in separate child processes.

Resolves#1123

[bnoordhuis]

Thanks, this is great! I wonder though, does tap.test() actually run tests in a separate process? I glean from node-tap's source that only tap.spawn() does that.

[NatalieWolfe]

Correct, tap.test doesn't run each in their own process, but $ tap test/test-* will spawn a separate child process for each file that matched. Was it per-individual-test that needed process isolation?

[bnoordhuis]

Yes.

[NatalieWolfe]

Is it possible to move the tests which need isolation into individual files? Which tests need this? I can look into doing that step as part of this ticket.

[bnoordhuis]

I'm going to say 'all of them' - it's not really true but isolation never hurts. It's my original plan of attack for #1123.

[NatalieWolfe]

Is that necessary up front? That seems like the kind of thing that could be resolved on a case-by-case basis when colliding tests are discovered. Moving individual tests into separate files isn't that hard, but seems like overkill if you don't actually need them isolated.

[bnoordhuis]

Such things tend to go unnoticed, though. I raised #1123 after I found two unrelated tests had been clobbering each other undetected for some time.

Another anecdote: a project I was involved in had a bug caused by lack of isolation that stopped half the tests from running but it went undiscovered for a year because the CI was green all the time.

[refack]

IMHO need to add -Rtap

[refack]

@NatalieWolfe do you need help? I can help...

[maclover7]

ping @NatalieWolfe

[rvagg]

needs to be updated, either @NatalieWolfe needs to revisit or someone else should open a PR, I'd +1 on merging this if it was ready

[cclauss]

@NatalieWolfe Is this already done elsewhere or is there more to do here?

[richardlau]

This was done in #1795.