Skip to content

dump's parameter "ensure_ascii" creates too long sequences #656

Description

@nlohmann

I detected a problem in the code of PR #654:

The code seems to created too long \uxxxx sequences. Take the sign for instance. It is U+20AC and should be encoded as string "\u20ac". The current code encodes it as "\u00e2\u0082\u00ac". This is incorrect, as this does not roundtrip.

Example:

#include <iostream>
#include <fstream>
#include "json.hpp"

using json = nlohmann::json;

int main() {
    json j1 = u8"";
    std::cout << j1.dump(0, ' ', false) << std::endl;
    std::cout << j1.dump(0, ' ', true) << std::endl;
    
    json j2 = json::parse("\"\\u20ac\"");
    std::cout << j2.dump(0, ' ', false) << std::endl;
    std::cout << j2.dump(0, ' ', true) << std::endl;
    
    json j3 = json::parse(j1.dump(0, ' ', true));
    std::cout << j3.dump(0, ' ', false) << std::endl;
    std::cout << j3.dump(0, ' ', true) << std::endl;
}

Output:

"€"
"\u00e2\u0082\u00ac"
"€"
"\u00e2\u0082\u00ac"
"�"
"\u00c3\u00a2\u00c2\u0082\u00c2\u00ac"

Expected output:

"€"
"\u20ac"
"€"
"\u20ac"
"€"
"\u20ac"

Sorry for not detecting this earlier. The provided test case was correct as it coped with Emojis which created longer sequences anyway.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions