Version
20.11.0
Platform
Darwin MacBook-Pro-4.local 23.1.0 Darwin Kernel Version 23.1.0: Mon Oct 9 21:27:27 PDT 2023; root:xnu-10002.41.9~6/RELEASE_X86_64 x86_64
Subsystem
stream
What steps will reproduce the bug?
When calling destroy(err) on a stream created using stream.compose() some streams that may still be actively processing data do not have _destroy called and do not emit an error event. Illustrated by the code below.
If there is a stream that slowly processes data as the last entry in compose(...) and all it's input has been written (finish event has fired) and then destroy() is called on the composed stream, I was expecting the slow processing stream to also be destroyed and emit the error.
Inserting a PassThrough stream as the last entry in the compose chain seems to fix the issue (uncomment lines below in sample code), but it's unclear to me why that is. Either I've misunderstood something about compose()ing streams or there's a bug.
import{PassThrough,Duplex,compose}from'stream'classSlowProcessorextendsDuplex{constructor(options){super({ ...options,objectMode: true})this.stuff=[]}_write(message,encoding,callback){this.stuff.push(message)callback()}_destroy(err,cb){console.log('SlowProcessor _destroy called',err)cb(err)}_read(){// emulate some slow processingsetTimeout(()=>{if(this.stuff.length){this.push(this.stuff.shift())}elseif(this.writableEnded){this.push(null)}else{setTimeout(()=>this._read(),100)}},100)}}constcomposed=compose(newPassThrough({objectMode:true}).on('finish',()=>console.log('input passthrough finish')).on('end',()=>console.log('input passthrough end')).on('error',err=>console.log('input passthrough error',err)),newSlowProcessor().on('finish',()=>console.log('SlowProcessor finish')).on('end',()=>console.log('SlowProcessor end')).on('error',err=>console.log('SlowProcessor error',err)),//// UNCOMMENT THIS AND IT WORKS AS EXPECTED//// new PassThrough({ objectMode:true })// .on('finish', () => console.log('terminal passthrough finish'))// .on('end', () => console.log('terminal passthrough end'))// .on('error', err => console.log('terminal passthrough error', err))).on('finish',()=>console.log('composed finish')).on('error',err=>console.log('composed',err)).on('end',()=>console.log('composed end')).on('data',d=>console.log('data:',d))composed.write('hello')composed.write('world')composed.end()setTimeout(()=>{composed.destroy(newError('Uh-oh'))},100)How often does it reproduce? Is there a required condition?
Reliably reproduces using sample above.
What is the expected behavior? Why is that the expected behavior?
I'd expect any stream that's part of a compose(...) chain that is still processing data to emit errors when the outer stream is destroyed.
I'd expect output from the above script something like:
input passthrough endinput passthrough finishSlowProcessor finish_destroy called Error: Uh-ohSlowProcessor error Error: Uh-ohcomposed Error: Uh-oh
What do you see instead?
The SlowProcessor stream does not get destroyed even though its Readable side has not ended.
input passthrough endinput passthrough finishSlowProcessor finish// <=== I'd expect an error event from the SlowProcessor here as it hasn't endedcomposed Error: Uh-oh
When there's an extra PassThrough inserted as the last in the compose chain, the results look as I'd expect:
input passthrough endinput passthrough finishSlowProcessor finishterminal passthrough error Error: Uh-ohSlowProcessor _destroy called Error: Uh-ohSlowProcessor error Error: Uh-ohcomposed Error: Uh-oh
Additional information
No response
Version
20.11.0
Platform
Darwin MacBook-Pro-4.local 23.1.0 Darwin Kernel Version 23.1.0: Mon Oct 9 21:27:27 PDT 2023; root:xnu-10002.41.9~6/RELEASE_X86_64 x86_64
Subsystem
stream
What steps will reproduce the bug?
When calling
destroy(err)on a stream created usingstream.compose()some streams that may still be actively processing data do not have_destroycalled and do not emit anerrorevent. Illustrated by the code below.If there is a stream that slowly processes data as the last entry in
compose(...)and all it's input has been written (finishevent has fired) and thendestroy()is called on the composed stream, I was expecting the slow processing stream to also be destroyed and emit the error.Inserting a
PassThroughstream as the last entry in the compose chain seems to fix the issue (uncomment lines below in sample code), but it's unclear to me why that is. Either I've misunderstood something aboutcompose()ing streams or there's a bug.How often does it reproduce? Is there a required condition?
Reliably reproduces using sample above.
What is the expected behavior? Why is that the expected behavior?
I'd expect any stream that's part of a
compose(...)chain that is still processing data to emit errors when the outer stream is destroyed.I'd expect output from the above script something like:
What do you see instead?
The
SlowProcessorstream does not get destroyed even though itsReadableside has notended.When there's an extra
PassThroughinserted as the last in the compose chain, the results look as I'd expect:Additional information
No response