StringDecoder#end doesn't flush state

[jridgewell]

• v8.6.0
• Darwin Kernel Version 16.6.0: Fri Apr 14 16:21:16 PDT 2017; root:xnu-3789.60.24~6/RELEASE_X86_64 x86_64
• StringDecoder

StringDecoder#end doesn't flush the state, allowing it to accept the completing byte afterwards:

const { StringDecoder } = require('string_decoder');
const decoder = new StringDecoder('utf8');

decoder.write(Buffer.from([0xE2, 0x82])); // => ''
// We're waiting for the final byte (currently at byte 2 of a 3 byte sequence)
// Now, end the buffer because, eg, the client disconnected.
decoder.end(); // => '�' (UTF-8 Replacement Character)

Now, there's our setup. If we've ended the buffer and it has outputted a replacement character, it kind of implies it's back at its initial state. But:

decoder.write(Buffer.of(0x61)); // => '�a'

Or, we could "finish" that 3 byte sequence from above (if we haven't already written the 'a'):

decoder.write(Buffer.of(0xAC)); // => '€'

[addaleax]

@jridgewell Do you have any suggestions on what to do here? Should .write() after .end() throw, like it does for streams?

[TimothyGu]

TextDecoder resets its state after the equivalent of an end call, and functions as a new TextDecoder instance basically. That sounds fair to me.

[mscdex]

Make sure to check benchmarks on this one because of how widely string_decoder is used.

[jridgewell]

I think throwing is better than the current behavior, but I’d prefer it just reset and behave like a brand new instance.

[thefourtheye]

Proposed a fix in #16594. I chose to reset silently instead of throwing.