Stack-overflow (OSS-Fuzz 4234)

[nlohmann]

Detailed report: https://oss-fuzz.com/testcase?key=5854339613589504

Project: json
Fuzzer: libFuzzer_json_parse_cbor_fuzzer
Job Type: libfuzzer_asan_json
Platform Id: linux

Crash Type: Stack-overflow
Crash Address: 0x7ffc55e22ce8
Crash State:
nlohmann::basic_json<std::__1::map, std::__1::vector, std::__1::basic_string<cha
nlohmann::detail::binary_reader<nlohmann::basic_json<std::__1::map, std::__1::ve
nlohmann::basic_json<std::__1::map, std::__1::vector, std::__1::basic_string<cha

Sanitizer: address (ASAN)

Regressed: https://oss-fuzz.com/revisions?job=libfuzzer_asan_json&range=201701031958:201701032147

Reproducer Testcase: https://oss-fuzz.com/download?testcase_id=5854339613589504

Issue filed automatically.

See https://github.com/google/oss-fuzz/blob/master/docs/reproducing.md for more information.

This bug is subject to a 90 day disclosure deadline. If 90 days elapse
without an upstream patch, then the bug report will automatically
become visible to the public.

clusterfuzz-testcase-minimized-5854339613589504.dms.zip

Stack trace:

AddressSanitizer:DEADLYSIGNAL
--
  | =================================================================
  | ==1==ERROR: AddressSanitizer: stack-overflow on address 0x7ffc55e22ce8 (pc 0x000000515f22 bp 0x7ffc55e23530 sp 0x7ffc55e22cd0 T0)
  | SCARINESS: 10 (stack-overflow)
  | #0 0x515f21 in operator new(unsigned long) _asan_rtl_
  | #1 0x5a6470 in __allocate /usr/local/include/c++/v1/new:226:10
  | #2 0x5a6470 in allocate /usr/local/include/c++/v1/memory:1747
  | #3 0x5a6470 in std::__1::vector<nlohmann::basic_json<std::__1::map, std::__1::vector, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, bool, long, unsigned long, double, std::__1::allocator, nlohmann::adl_serializer>, std::__1::allocator<nlohmann::basic_json<std::__1::map, std::__1::vector, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, bool, long, unsigned long, double, std::__1::allocator, nlohmann::adl_serializer> > >* nlohmann::basic_json<std::__1::map, std::__1::vector, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, bool, long, unsigned long, double, std::__1::allocator, nlohmann::adl_serializer>::create<std::__1::vector<nlohmann::basic_json<std::__1::map, std::__1::vector, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, bool, long, unsigned long, double, std::__1::allocator, nlohmann::adl_serializer>, std::__1::allocator<nlohmann::basic_json<std::__1::map, std::__1::vector, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, bool, long, unsigned long, double, std::__1::allocator, nlohmann::adl_serializer> > >>() json/src/json.hpp:7939
  | #4 0x5a5fcf in nlohmann::basic_json<std::__1::map, std::__1::vector, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, bool, long, unsigned long, double, std::__1::allocator, nlohmann::adl_serializer>::json_value::json_value(nlohmann::detail::value_t) json/src/json.hpp:8013:29
  | #5 0x5a0061 in nlohmann::basic_json<std::__1::map, std::__1::vector, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, bool, long, unsigned long, double, std::__1::allocator, nlohmann::adl_serializer>::basic_json(nlohmann::detail::value_t) json/src/json.hpp:8249:22
  | #6 0x59ed6a in nlohmann::basic_json<std::__1::map, std::__1::vector, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, bool, long, unsigned long, double, std::__1::allocator, nlohmann::adl_serializer> nlohmann::detail::binary_reader<nlohmann::basic_json<std::__1::map, std::__1::vector, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, bool, long, unsigned long, double, std::__1::allocator, nlohmann::adl_serializer> >::get_cbor_array<int>(int) json/src/json.hpp:5374:32
  | #7 0x59bd2e in nlohmann::detail::binary_reader<nlohmann::basic_json<std::__1::map, std::__1::vector, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, bool, long, unsigned long, double, std::__1::allocator, nlohmann::adl_serializer> >::parse_cbor_internal(bool) json/src/json.hpp:4717:24
  | #8 0x5a4bd0 in nlohmann::basic_json<std::__1::map, std::__1::vector, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, bool, long, unsigned long, double, std::__1::allocator, nlohmann::adl_serializer> nlohmann::detail::binary_reader<nlohmann::basic_json<std::__1::map, std::__1::vector, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, bool, long, unsigned long, double, std::__1::allocator, nlohmann::adl_serializer> >::get_cbor_array<int>(int)::{lambda()#1}::operator()() const json/src/json.hpp:5377:20
  | #9 0x59ee2e in generate_n<std::__1::back_insert_iterator<std::__1::vector<nlohmann::basic_json<std::map, std::vector, std::__1::basic_string<char>, bool, long, unsigned long, double, std::allocator, adl_serializer>, std::__1::allocator<nlohmann::basic_json<std::map, std::vector, std::__1::basic_string<char>, bool, long, unsigned long, double, std::allocator, adl_serializer> > > >, int, (lambda at ../src/json.hpp:5375:73)> /usr/local/include/c++/v1/algorithm:2186:20
  | #10 0x59ee2e in nlohmann::basic_json<std::__1::map, std::__1::vector, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, bool, long, unsigned long, double, std::__1::allocator, nlohmann::adl_serializer> nlohmann::detail::binary_reader<nlohmann::basic_json<std::__1::map, std::__1::vector, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, bool, long, unsigned long, double, std::__1::allocator, nlohmann::adl_serializer> >::get_cbor_array<int>(int) json/src/json.hpp:5375
  | #11 0x59bd2e in nlohmann::detail::binary_reader<nlohmann::basic_json<std::__1::map, std::__1::vector, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, bool, long, unsigned long, double, std::__1::allocator, nlohmann::adl_serializer> >::parse_cbor_internal(bool) json/src/json.hpp:4717:24
  | #12 0x5a4bd0 in nlohmann::basic_json<std::__1::map, std::__1::vector, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, bool, long, unsigned long, double, std::__1::allocator, nlohmann::adl_serializer> nlohmann::detail::binary_reader<nlohmann::basic_json<std::__1::map, std::__1::vector, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, bool, long, unsigned long, double, std::__1::allocator, nlohmann::adl_serializer> >::get_cbor_array<int>(int)::{lambda()#1}::operator()() const json/src/json.hpp:5377:20
  | #13 0x59ee2e in generate_n<std::__1::back_insert_iterator<std::__1::vector<nlohmann::basic_json<std::map, std::vector, std::__1::basic_string<char>, bool, long, unsigned long, double, std::allocator, adl_serializer>, std::__1::allocator<nlohmann::basic_json<std::map, std::vector, std::__1::basic_string<char>, bool, long, unsigned long, double, std::allocator, adl_serializer> > > >, int, (lambda at ../src/json.hpp:5375:73)> /usr/local/include/c++/v1/algorithm:2186:20
  | #14 0x59ee2e in nlohmann::basic_json<std::__1::map, std::__1::vector, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, bool, long, unsigned long, double, std::__1::allocator, nlohmann::adl_serializer> nlohmann::detail::binary_reader<nlohmann::basic_json<std::__1::map, std::__1::vector, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, bool, long, unsigned long, double, std::__1::allocator, nlohmann::adl_serializer> >::get_cbor_array<int>(int) json/src/json.hpp:5375
  | #15 0x59bd2e in nlohmann::detail::binary_reader<nlohmann::basic_json<std::__1::map, std::__1::vector, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, bool, long, unsigned long, double, std::__1::allocator, nlohmann::adl_serializer> >::parse_cbor_internal(bool) json/src/json.hpp:4717:24
  | #16 0x5a4bd0 in nlohmann::basic_json<std::__1::map, std::__1::vector, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, bool, long, unsigned long, double, std::__1::allocator, nlohmann::adl_serializer> nlohmann::detail::binary_reader<nlohmann::basic_json<std::__1::map, std::__1::vector, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, bool, long, unsigned long, double, std::__1::allocator, nlohmann::adl_serializer> >::get_cbor_array<int>(int)::{lambda()#1}::operator()() const json/src/json.hpp:5377:20
  | #17 0x59ee2e in generate_n<std::__1::back_insert_iterator<std::__1::vector<nlohmann::basic_json<std::map, std::vector, std::__1::basic_string<char>, bool, long, unsigned long, double, std::allocator, adl_serializer>, std::__1::allocator<nlohmann::basic_json<std::map, std::vector, std::__1::basic_string<char>, bool, long, unsigned long, double, std::allocator, adl_serializer> > > >, int, (lambda at ../src/json.hpp:5375:73)> /usr/local/include/c++/v1/algorithm:2186:20
  | #18 0x59ee2e in nlohmann::basic_json<std::__1::map, std::__1::vector, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, bool, long, unsigned long, double, std::__1::allocator, nlohmann::adl_serializer> nlohmann::detail::binary_reader<nlohmann::basic_json<std::__1::map, std::__1::vector, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, bool, long, unsigned long, double, std::__1::allocator, nlohmann::adl_serializer> >::get_cbor_array<int>(int) json/src/json.hpp:5375
  | #19 0x59bd2e in nlohmann::detail::binary_reader<nlohmann::basic_json<std::__1::map, std::__1::vector, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, bool, long, unsigned long, double, std::__1::allocator, nlohmann::adl_serializer> >::parse_cbor_internal(bool) json/src/json.hpp:4717:24

[nlohmann]

I analyzed the testcase. It more or less consists of a lot of nested arrays. On my machine, there is stack overflow at depth 1450. I am not sure how to cope with such issues. We have the same problem if we would parse a JSON file like

[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[ ...

At some point, the parser would explode. Any ideas on this?

[gregmarr]

The only ways to "fix" this are to impose an arbitrary limit on the level of nesting, or change whatever it is that is overflowing the stack from a recursive algorithm to an iterative algorithm.

[nlohmann]

Yes, and imposing such a limit would be OK with RFC 7159 (Sect. 9):

An implementation may set limits on the maximum depth of nesting.

But I would not like it...

The recursion is:

parse_cbor_internal -> get_cbor_array -> parse_cbor_internal -> get_cbor_array -> ...

[gregmarr]

According to the stack above, it's actually a 4 function recursion, which I think would be extremely difficult to break:

json nlohmann::detail::binary_reader<json>::get_cbor_array<int>(int)
nlohmann::detail::binary_reader<json>::parse_cbor_internal(bool)
json nlohmann::detail::binary_reader<json>::get_cbor_array<int>(int)::{lambda()}::operator()() const
generate_n<back_insert_iterator<vector<json>>, int, (lambda)>

[nlohmann]

Sorry, I simplified it a bit. And it all makes sense, when you try to parse a recursive structure with a recursive decent parser...

[nlohmann]

I think there is nothing we can do about this. JSON and CBOR base on recursion, and adding arbitrary limits is not a solution.

[nlohmann]

Strangely, I got this message yesterday:

Updates:
Labels: ClusterFuzz-Verified
Status: Verified

Comment #2 on issue 4234 by ClusterFuzz-External: json: Stack-overflow in nlohmann::basic_json<std::__1::map, std::__1::vector, std::__1::basic_string<cha
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=4234#c2

ClusterFuzz testcase 5854339613589504 is verified as fixed, so closing issue as verified.

If this is incorrect, please file a bug on https://github.com/google/oss-fuzz/issues/new