`<fstream>`: seeking 0 with std::ios::cur ignored with bidirectional streams

[Bo98]

Describe the bug
C++ Standard, [filebuf], paragraph 2 states that:

The restrictions on reading and writing a sequence controlled by an object of class basic_filebuf<charT, traits> are the same as for reading and writing with the C standard library FILEs

So when referring to C standard §7.19.5.3, paragraph 6 states that:

However, output shall not be directly followed by input without an intervening call to the fflush function or to a file positioning function (fseek, fsetpos, or rewind), and input shall not be directly followed by output without an intervening call to a file positioning function, unless the input operation encounters end-of-file

One could therefore expect that for a bidirectional std::fstream you could perform stream.seekp(0, std::ios::cur) to satisfy this. In libstdc++ and libc++, this works but this does not work with MSVC STL.

Note that stream.seekp(stream.tellp()) works. So does a +1 and -1 on std::ios:cur.

Command-line test case

#include <fstream>
#include <iostream>

int main() {
	std::ofstream init("test.txt");
	init << "abcd";
	init.close();

	char buffer[5] = {};

	std::fstream inout("test.txt");
	inout.read(buffer, 2);
	inout.seekp(0, std::ios::cur); // Should allow switch to write.
	inout << "ba";
	inout.close();

	std::ifstream out("test.txt");
	out.read(buffer, 4);
	std::cout << buffer << std::endl;
	out.close();
}

Expected behavior
The output is "abba".

MSVC STL outputs "abcd" - the write operation is completely ignored.

STL version

Microsoft Visual Studio Enterprise 2022
Version 17.5.2

Additional context

C++ standard, [filebuf.virtuals], paragraph 13 states:

Let width denote a_codecvt.encoding(). If is_open() == false, or off != 0 && width <= 0, then the positioning operation fails. Otherwise, if way != basic_ios::cur or off != 0, and if the last operation was output, then update the output sequence and write any unshift sequence. Next, seek to the new position: if width > 0, call fseek(file, width * off, whence), otherwise call fseek(file, 0, whence)

If I've not misread this: although stream.seekp(0, std::ios::cur) should not touch the output/unshift sequence, it still should call fseek to my understanding.

[StephanTLavavej]

Thanks, this does seem like a bug. We'll need to investigate where this is happening - from a brief glance at the code, it looks like we're not short-circuiting for 0 and are passing it to the UCRT's fsetpos().

[cpplearner]

So this is related to #2477: UCRT sometimes gets the position wrong in text mode, and thus fseek(f, 0, SEEK_CUR) may change the file position in an unexpected way.

This in particular affects tellg, which internally calls seekoff(0, cur, in). (#2477 and #3840 show that tellg currently returns the wrong value, but at least it doesn't change the file position.)

[Bo98]

Right, yeah. UCRT doesn't really support files with LF in text mode. If you're writing on C level, the expectation is to use binary mode if you use LF files on Windows, which is not exactly great but it's the way it's been forever. I suppose that expectation hasn't really translated fully in C++ land, and I agree the UCRT behaviour isn't ideal. It all comes down to this behaviour of the UCRT:

Carriage return-line feed (CR-LF) combinations are translated into single line feeds (LF) on input and LF characters are translated to CR-LF combinations on output. Also, CTRL+Z is interpreted as an end-of-file character on input.

That's fine but the problem is that it strongly assumes the input file is CRLF, rather than supporting both.

There's two main layers in UCRT. You have the low level IO and you have the FILE level IO. Assuming the string: hello\r\n, the buffers used for low level IO is as-is with the CR and LF characters, but the buffer used in FILE level is hello\n.

There's a few problems. Here's one example:

    // If this is a binary mode stream, we can simply subtract this from the
    // lowio position, and combine it with the buffer offset to get the stdio
    // position:
    if ((_osfile(fh) & FTEXT) == 0)
    {
        return lowio_position
            - (bytes_read    / translation_factor)
            + (buffer_offset / translation_factor);
    }

    // If this is a text mode stream, we need to adjust the number of bytes that
    // were read into the buffer to account for newline translation.
    //
    // If we are _not_ at EOF, the number of untranslated characters read is the
    // buffer size.  However, if we are not at EOF, the buffer may not be full,
    // so we need to scan the buffer to count newline characters.  (Note:  we
    // only count newline characters if the stream is at EOF, because doing so
    // is more expensive than seeking to the end and seeking back).

    // Seek to the end of the file.  If the current position is the end of the
    // file, then scan the buffer for newlines and adjust bytes_read:
    if (_lseeki64_internal(fh, 0, SEEK_END, ptd) == lowio_position)
    {
        bytes_read += count_newline_bytes(stream->_base, stream->_base + bytes_read, text_mode);

        // If the last byte was a ^Z, that character will not be present in the
        // buffer (it is omitted by lowio):
        if (stream.ctrl_z())
        {
            bytes_read += buffer_character_size(text_mode);
        }
    }
    else

count_newline_bytes counts the number of newlines on FILE level, where \r is stripped. It does this and adds it on to the bytes read because it assumes all \n is preceded by \r when operating in text mode, so each newline should add an additional byte. These calculations then go on to be used alongside the low level IO position and sizes, which can cause things to go negative when it thinks it has read two bytes (CR-LF) on the file level, but the low level IO has only actually read one byte (LF).

There's a few other cases like this:

        // If the first byte in the untranslated buffer was a '\n', we assume it
        // was preceded by a '\r', which was discarded by the previous read
        // operation:
        if (_osfile(fh) & FCRLF)
        {
            bytes_read += buffer_character_size(text_mode);
        }

which explicitly says they are assuming CRLF is always what is used while operating under text mode (FCRLF is not some magical file detection flag - it's simply something to denote that the first character of the buffer is LF).

There's other issues like #1784 too, that demonstrate how LF text file position calculations have never really been supported.

Binary mode is unaffected in all this. Binary mode on LF files should work roughly the same way as text mode does on CRLF files.