msgpack unit test failures on ppc64 arch

[dkopecek]

Hi, I was trying to build[1] the newest release on multiple architectures (Fedora build architectures). There seems to be some msgpack related unit test failure on ppc64. See the attached build log:

json-ppc64-build.log.txt

[1] https://koji.fedoraproject.org/koji/taskinfo?taskID=18476755

[nlohmann]

Thanks for the report. Could this be an endianess issue?

[dkopecek]

Yeah, that would be my guess too. Consider:

src/unit-msgpack.cpp:210: FAILED:
  CHECK( json::from_msgpack(result) == j )
with expansion:
  1 == 256

which looks like 0x0001 == 0x0100

[dkopecek]

ppc64 is big endian

[user1095108]

if you decide to do the swaps don't forget about bit_cast<>().

[nlohmann]

I currently do not have the means to test this. Is there a way to have a big endian system with Vagrant?

[user1095108]

Apparently the users, test this for you, you don't need to test yourself. Otherwise, try qemu, install a big endian distro and start testing. 2017-04-01 1:08 GMT+02:00 Niels Lohmann <notifications@github.com>:

…

I currently do not have the means to test this. Is there a way to have a big endian system with Vagrant? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#524 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AH6jVM6a_CIFxnG3AQFUuhJPrTh_kJCBks5rrYdigaJpZM4Mh1xK> .

[nlohmann]

In the end, I need a test setup to at least verify a fix.

I could get qemu to work, but I failed to get a recent Linux with a recent compiler to work, let alone properly integrate it into my network. Hints greatly appreciated!

[user1095108]

Its very involved, I don't even remember how I did it myself, but I managed at last. You should use Debian and follow some online tuts.

…

On Apr 3, 2017 22:31, "Niels Lohmann" ***@***.***> wrote: In the end, I need a test setup to at least verify a fix. I could get qemu to work, but I failed to get a recent Linux with a recent compiler to work, let alone properly integrate it into my network. Hints greatly appreciated! — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#524 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AH6jVNy1ltorQmhawYxdQLn7SktGNza1ks5rsVckgaJpZM4Mh1xK> .

[nlohmann]

Hey @dkopecek, could you please try https://github.com/nlohmann/json/blob/feature/manual_lexer/src/json.hpp ? I could not test it with a big-endian system myself.

It is a reimplementation of the binary readers/writers which should respect the byte order.

The main functions are:

// from http://stackoverflow.com/a/1001328/266378
static bool little_endianess()
{
    int num = 1;
    return (*reinterpret_cast<char*>(&num) == 1);
}

// writes a number of type T to vector v
template<typename T>
void write_number(T n)
{
    std::array<uint8_t, sizeof(T)> vec;
    std::memcpy(vec.data(), &n, sizeof(T));

    for (size_t i = 0; i < sizeof(T); ++i)
    {
        // reverse byte order prior to conversion if necessary
        if (is_little_endian)
        {
            v.push_back(vec[sizeof(T) - i - 1]);
        }
        else
        {
            v.push_back(vec[i]);
        }
    }
}

// reads a number of type T from input
template<typename T>
T get_number()
{
    std::array<uint8_t, sizeof(T)> vec;
    for (size_t i = 0; i < sizeof(T); ++i)
    {
        get();
        check_eof();

        // reverse byte order prior to conversion if necessary
        if (is_little_endian)
        {
            vec[sizeof(T) - i - 1] = static_cast<uint8_t>(current);
        }
        else
        {
            vec[i] = static_cast<uint8_t>(current);
        }
    }

    T result;
    std::memcpy(&result, vec.data(), sizeof(T));
    return result;
}

[user1095108]

This code is inefficient considering that endianess is a compile-time constant. You could do away with the if.

…

On Apr 5, 2017 06:58, "Niels Lohmann" ***@***.***> wrote: Hey @dkopecek <https://github.com/dkopecek>, could you please try https://github.com/nlohmann/json/blob/feature/manual_lexer/src/json.hpp ? I could not test it with a big-endian system myself. It is a reimplementation of the binary readers/writers which should respect the byte order. The main functions are: // from http://stackoverflow.com/a/1001328/266378static bool little_endianess() { int num = 1; return (*reinterpret_cast<char*>(&num) == 1); } // writes a number of type T to vector vtemplate<typename T>void write_number(T n) { std::array<uint8_t, sizeof(T)> vec; std::memcpy(vec.data(), &n, sizeof(T)); for (size_t i = 0; i < sizeof(T); ++i) { // reverse byte order prior to conversion if necessary if (is_little_endian) { v.push_back(vec[sizeof(T) - i - 1]); } else { v.push_back(vec[i]); } } } // reads a number of type T from inputtemplate<typename T> T get_number() { std::array<uint8_t, sizeof(T)> vec; for (size_t i = 0; i < sizeof(T); ++i) { get(); check_eof(); // reverse byte order prior to conversion if necessary if (is_little_endian) { vec[sizeof(T) - i - 1] = static_cast<uint8_t>(current); } else { vec[i] = static_cast<uint8_t>(current); } } T result; std::memcpy(&result, vec.data(), sizeof(T)); return result; } — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#524 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AH6jVK0SirO4aXnunKTw7iKvxEGUVaMCks5rsx9agaJpZM4Mh1xK> .

[nlohmann]

I could not find such a constant in the standard, and I would like to avoid dozens of #ifdefs to cover all compilers. I'm curious whether this fix works in the first place.