Inclusion of nlohmann/json.hpp causes OS/ABI to change on Linux

[jongough]

• What is the issue you have?
  Trying to incorporate a json schema validator using nlohmann/json.hpp into C++ code causes the linux gcc compiler/linker to change the OSABI from UNIX - System V (ELF OS/ABI value 0) to UNIX - GNU (ELF OS/ABI value 3). This makes it impossible to run with existing programs as it fails the compliance test at runtime.

• Please describe the steps to reproduce the issue. Can you provide a small but working code example?
  A simple program called testOSABI.cpp:

    //#include "nlohmann/json.hpp"
    int main (){
        return 0;
    }

If compiled with:
gcc testOSABI.cpp -o testOSABI.o -Inlohmann_json/single_include
Then use readelf
readelf -h testOSABI.o
will give:

ELF Header:
  Magic:   7f 45 4c 46 02 01 01 00 00 00 00 00 00 00 00 00 
  Class:                             ELF64
  Data:                              2's complement, little endian
  Version:                           1 (current)
  OS/ABI:                            UNIX - System V
  ABI Version:                       0
  Type:                              DYN (Shared object file)
  Machine:                           Advanced Micro Devices X86-64
  Version:                           0x1
  Entry point address:               0x4f0
  Start of program headers:          64 (bytes into file)
  Start of section headers:          6376 (bytes into file)
  Flags:                             0x0
  Size of this header:               64 (bytes)
  Size of program headers:           56 (bytes)
  Number of program headers:         9
  Size of section headers:           64 (bytes)
  Number of section headers:         28
  Section header string table index: 27

If the program has the #include statement uncommented and the same process followed the readelf will give:

  Magic:   7f 45 4c 46 02 01 01 03 00 00 00 00 00 00 00 00 
  Class:                             ELF64
  Data:                              2's complement, little endian
  Version:                           1 (current)
  OS/ABI:                            UNIX - GNU
  ABI Version:                       0
  Type:                              DYN (Shared object file)
  Machine:                           Advanced Micro Devices X86-64
  Version:                           0x1
  Entry point address:               0x520
  Start of program headers:          64 (bytes into file)
  Start of section headers:          7032 (bytes into file)
  Flags:                             0x0
  Size of this header:               64 (bytes)
  Size of program headers:           56 (bytes)
  Number of program headers:         9
  Size of section headers:           64 (bytes)
  Number of section headers:         29
  Section header string table index: 28

• What is the expected behaviour?
  The two outputs should be the same for the OS/ABI value

• And what is the actual behaviour instead?
  The two OS/ABI values are different. This will stop a program with json.hpp included from interacting successfully with one that does not have json.hpp included.

• Which compiler and operating system are you using? Is it a supported compiler?
  gcc 7.3.0 on Ubuntu based Linux Mint 19 with kernel 4.18.0-13-generic

• Did you use a released version of the library or the version from the develop branch?
  Release

[nlohmann]

Thanks for reporting! I have no idea how to proceed here.

[jongough]

I have looked around to see if there is anyway to set the OS/ABI with the compiler/linker options, but it seems to be poorly documented. I have tried some suggestion, i.e. -fno-jump-tables, but none of them worked.
The only thing that I can think of is to try cutting down the json.hpp file to see what capability you are using that changes the ELF flavour. I have also tried compiling my project for windows and had a similar result i.e. incompatible DLL is generated and causes a crash.

The code I am trying to integrate your json.hpp with is quite large with multiple modules generated by many people. In Linux the main program and all the loadable libraries have System V ABI, so I need to be able to generate my loadable with json.hpp the same.

[nickaein]

To track down the issue, we can use nm tool to see the symbols in the object:

nm --demangle testOSABI.o

Comparing its output to the case when jhon.hpp is not included, there are two symbols with u type which we hadn't before:

0000000000491f15 u nlohmann::detail::static_const<nlohmann::detail::to_json_fn>::value
0000000000491f14 u nlohmann::detail::static_const<nlohmann::detail::from_json_fn>::value

For u symbols nm man page says (emphasis mine):

The symbol is a unique global symbol. This is a GNU extension to the standard set of ELF symbol bindings. For such a symbol the dynamic linker will make sure that in the entire process there is just one symbol with this name and type in use.

In the source code, to_json_fn and from_json_fn are the only types that are referenced inside a static_const template which in turn defined as:

template<typename T>
struct static_const
{
    static constexpr T value{};
};

template<typename T>
constexpr T static_const<T>::value;

I suspect the static constexpr translate as unique global variable to the linker so it uses unique global symbol for these types. The root cause could be new revisions to the standard abbout static variables, their initialization their thread-safety. I might be wrong on this though.

However, I'm afraid this is not limited only to from_json and to_json.

Take the following code for instance:

#include <string>

int main()
{
    std::string str;

    size_t pos = str.find("Hello");
    if(pos == std::string::npos)
        return 0;

    return 1;
}

Compiling this code with GCC 7.3 also lead to a binary with header OS/ABI: UNIX - GNU. The nm output confirms it too as there is a std::string::npos symbol with type u. This means that it is probably defined as static constexpr or something similar to that in the std::string source code.

This is not always true. A better code sample is posted in my next comment: #1410 (comment)

[jongough]

In my program with json.hpp included I get the following " u " values

00000000005d10c8 u guard variable for std::__detail::_AnyMatcher<std::__cxx11::regex_traits<char>, false, false, true>::operator()(char) const::__nul
00000000005d10d8 u guard variable for std::__detail::_AnyMatcher<std::__cxx11::regex_traits<char>, false, true, false>::operator()(char) const::__nul
00000000005d10e8 u guard variable for std::__detail::_AnyMatcher<std::__cxx11::regex_traits<char>, false, true, true>::operator()(char) const::__nul
00000000005b77e0 u std::__cxx11::regex_traits<char>::lookup_classname<char const*>(char const*, char const*, bool) const::__classnames
00000000005cfcc0 u std::__cxx11::regex_traits<char>::lookup_collatename<char const*>(char const*, char const*) const::__collatenames
00000000005d10b0 u std::__detail::_AnyMatcher<std::__cxx11::regex_traits<char>, false, false, false>::operator()(char) const::__nul
00000000005d10c0 u std::__detail::_AnyMatcher<std::__cxx11::regex_traits<char>, false, false, true>::operator()(char) const::__nul
00000000005d10d0 u std::__detail::_AnyMatcher<std::__cxx11::regex_traits<char>, false, true, false>::operator()(char) const::__nul
00000000005d10e0 u std::__detail::_AnyMatcher<std::__cxx11::regex_traits<char>, false, true, true>::operator()(char) const::__nul
000000000032a178 u std::__detail::_Executor<__gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::__cxx11::sub_match<__gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > >, std::__cxx11::regex_traits<char>, false>::_M_is_word(char) const::__s
000000000032a17a u std::__detail::_Executor<__gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::__cxx11::sub_match<__gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > >, std::__cxx11::regex_traits<char>, true>::_M_is_word(char) const::__s

Is there any ability to change this so that GNU extensions are not used? The code my module is interacting with is UNIX - System V and cannot be changed to UNIX - GNU. The code is also used on osX and Windows, and I know windows gets a 'similar' result, although the error codes are different.

Regards
Jon

[nickaein]

In my program with json.hpp included I get the following " u " values

The "u" symbols present in the object may differ based on your usage. That's because the linker can removed unused symbols from the final executable.

It is interesting that all "u" symbols showing up for you is from std namespace though. It's weird that there are no "u" symbols from nlohmann json namespace as in my experience a couple of them should up. Maybe linker removed it's unused symbols?

Is there any ability to change this so that GNU extensions are not used?

I am no expert on this but and it seems not easy. Here are some posts discussing this issue:

https://stackoverflow.com/questions/11931420
https://gcc.gnu.org/ml/gcc-help/2013-01/msg00008.html
https://cygwin.com/ml/binutils/2011-10/msg00276.html

There are probably various cases that such symbol shows up in the binary object. The previous sample code I posted at #1410 (comment) doesn't always work.

The following code (which is very simple singleton pattern) seems trigger "u" symbol generation too.
This happens even on c++98 standard (g++ -std=c++98 -O3 main.cpp -o main.o):

class singleton {
public:
    static int& getInstance()
    {
        static int number = 1;

        return number;
    }
};

int main()
{
    int& ref = singleton::getInstance();

    return ref;
}

[FrancoisChabot]

This can be fixed by simply wrapping static_const in an annonymous namespace, thus preventing the symbols from being exported altogether (which should be fine for a constexpr value like this):

namespace {
template<typename T>
struct static_const
{
    static constexpr T value{};
};

template<typename T>
constexpr T static_const<T>::value;
}
}  // namespace detail
}  // namespace nlohmann

Whether this should be done for the library is potentially contentious since it's an import from ranges-V3 and consistency might be desirable. However, it's a change you can make to your own copy of json.hpp very easily.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[jongough]

I was away so I could not do a test before the bot auto closed this issue.

I have tried this, but it still shows the same " u " strings. So there must be something else in the json.hpp which is triggering this behaviour.

[nlohmann]

Should I reopen? Does anyone has an idea?

[jongough]

It needs someone to look at it as just including json.hpp should leave the 'flavour' of the environment as it was, i.e. it should add no 'special' processing. If this is not going to be the case then it makes it very difficult for developers to use this code to assist with json processing.