Skip to content

ARROW-2023: [C++] Fix ASAN failure on malformed / empty stream input, enable ASAN builds, add more dev docs - #1503

Closed
wesm wants to merge 4 commits into
apache:masterfrom
wesm:ARROW-2023
Closed

ARROW-2023: [C++] Fix ASAN failure on malformed / empty stream input, enable ASAN builds, add more dev docs#1503
wesm wants to merge 4 commits into
apache:masterfrom
wesm:ARROW-2023

Conversation

@wesm

@wesmwesm commented Jan 24, 2018

Copy link
Copy Markdown
Member

More to do here, see discussion in https://issues.apache.org/jira/browse/ARROW-1589

Change-Id: I6af65bb1fe46f8aa815895570849fcef5b624bcc
…s [skip ci]
Change-Id: I007e567af807d2af4d1801a11cd34279d80fd740
…ve Apache Kudu cruft from run-test.sh
Change-Id: Ib97dbdb32e3f4da4919584e29f17db55bfb964a5
…eturn more informative error message
Change-Id: I335736571e56af36f4de909e35f8d667693db7fe
@wesmwesm changed the title WIP ARROW-2023: [C++] Start stream reader tests for malformed inputsARROW-2023: [C++] Fix ASAN failure on malformed / empty stream input, enable ASAN builds, add more dev docsFeb 26, 2018
@wesm

wesm commented Feb 26, 2018

Copy link
Copy Markdown
MemberAuthor

@xhochy@crepererum this is ready to go. note this only fixes one of the fuzzer failures, now it returns a Flatbuffers error:

==19598==The signal is caused by a READ memory access.
#0 0x682c79 in int flatbuffers::ReadScalar<int>(void const*) /home/wesm/cpp-toolchain/include/flatbuffers/flatbuffers.h:198:23
#1 0x682b6f in flatbuffers::Table::GetVTable() const /home/wesm/cpp-toolchain/include/flatbuffers/flatbuffers.h:1583:20
#2 0x682a3a in flatbuffers::Table::GetOptionalFieldOffset(unsigned short) const /home/wesm/cpp-toolchain/include/flatbuffers/flatbuffers.h:1590:19
#3 0x6b0e8e in long flatbuffers::Table::GetField<long>(unsigned short, long) const /home/wesm/cpp-toolchain/include/flatbuffers/flatbuffers.h:1599:25
#4 0x783934 in org::apache::arrow::flatbuf::Message::bodyLength() const /home/wesm/code/arrow/cpp/build/src/arrow/ipc/Message_generated.h:294:12
#5 0x77ced7 in arrow::ipc::Message::ReadFrom(std::shared_ptr<arrow::Buffer> const&, arrow::io::InputStream*, std::unique_ptr<arrow::ipc::Message, std::default_delete<arrow::ipc::Message> >*) /home/wesm/code/arrow/cpp/build/../src/arrow/ipc/message.cc:140:37
#6 0x7824d9 in arrow::ipc::ReadMessage(arrow::io::InputStream*, std::unique_ptr<arrow::ipc::Message, std::default_delete<arrow::ipc::Message> >*) /home/wesm/code/arrow/cpp/build/../src/arrow/ipc/message.cc:236:10
#7 0x784036 in arrow::ipc::InputStreamMessageReader::ReadNextMessage(std::unique_ptr<arrow::ipc::Message, std::default_delete<arrow::ipc::Message> >*) /home/wesm/code/arrow/cpp/build/../src/arrow/ipc/message.cc:255:12
#8 0x67abb8 in arrow::ipc::ReadMessageAndValidate(arrow::ipc::MessageReader*, arrow::ipc::Message::Type, bool, std::unique_ptr<arrow::ipc::Message, std::default_delete<arrow::ipc::Message> >*) /home/wesm/code/arrow/cpp/build/../src/arrow/ipc/reader.cc:371:29
#9 0x6b2e82 in arrow::ipc::RecordBatchStreamReader::RecordBatchStreamReaderImpl::ReadSchema() /home/wesm/code/arrow/cpp/build/../src/arrow/ipc/reader.cc:426:23
#10 0x67d40f in arrow::ipc::RecordBatchStreamReader::RecordBatchStreamReaderImpl::Open(std::unique_ptr<arrow::ipc::MessageReader, std::default_delete<arrow::ipc::MessageReader> >) /home/wesm/code/arrow/cpp/build/../src/arrow/ipc/reader.cc:407:12
#11 0x6724d2 in arrow::ipc::RecordBatchStreamReader::Open(std::unique_ptr<arrow::ipc::MessageReader, std::default_delete<arrow::ipc::MessageReader> >, std::shared_ptr<arrow::RecordBatchReader>*) /home/wesm/code/arrow/cpp/build/../src/arrow/ipc/reader.cc:477:36
#12 0x672dc0 in arrow::ipc::RecordBatchStreamReader::Open(arrow::io::InputStream*, std::shared_ptr<arrow::RecordBatchReader>*) /home/wesm/code/arrow/cpp/build/../src/arrow/ipc/reader.cc:484:10
#13 0x651842 in LLVMFuzzerTestOneInput /home/wesm/code/arrow/cpp/build/../src/arrow/ipc/ipc-fuzzing-test.cc:31:12
#14 0x52a0ae in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) (/home/wesm/code/arrow/cpp/build/debug/ipc-fuzzing-test+0x52a0ae)
#15 0x529a94 in fuzzer::Fuzzer::RunOne(unsigned char const*, unsigned long, bool, fuzzer::InputInfo*) (/home/wesm/code/arrow/cpp/build/debug/ipc-fuzzing-test+0x529a94)
#16 0x52ad51 in fuzzer::Fuzzer::MutateAndTestOne() (/home/wesm/code/arrow/cpp/build/debug/ipc-fuzzing-test+0x52ad51)
#17 0x52af17 in fuzzer::Fuzzer::Loop() (/home/wesm/code/arrow/cpp/build/debug/ipc-fuzzing-test+0x52af17)
#18 0x523415 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) (/home/wesm/code/arrow/cpp/build/debug/ipc-fuzzing-test+0x523415)
#19 0x51e280 in main (/home/wesm/code/arrow/cpp/build/debug/ipc-fuzzing-test+0x51e280)
#20 0x7fdeb9619f44 in __libc_start_main /build/eglibc-ripdx6/eglibc-2.19/csu/libc-start.c:287
#21 0x51e185 in _start (/home/wesm/code/arrow/cpp/build/debug/ipc-fuzzing-test+0x51e185)

I created https://issues.apache.org/jira/browse/ARROW-2222 about adding an option to validate input messages

@wesm

wesm commented Feb 26, 2018

Copy link
Copy Markdown
MemberAuthor

@xhochyxhochy left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, LGTM

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@wesm@xhochy