Uh oh!
There was an error while loading. Please reload this page.
[Fix](arrow flight) Fix arrow::Status inline static empty msg core - #63191
Conversation
linrrzqqq
commented
May 12, 2026
run buildall |
hello-stephen
commented
May 13, 2026
BE UT Coverage ReportIncrement line coverage Increment coverage report
|
linrrzqqq
commented
May 13, 2026
run buildall |
linrrzqqq
commented
May 13, 2026
/review |
hello-stephen
commented
May 13, 2026
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
There was a problem hiding this comment.
Automated review completed for PR 63191.
Findings: no blocking issues found.
Critical checkpoint conclusions:
- Goal/test evidence: The PR replaces the narrower Arrow Flight finish-status workaround with a root-cause Arrow
Statuspatch that moves the empty message/detail sentinels out of inline header function-local statics and keeps error-status paths from touching the OK sentinel. This matches the crash described in the PR body. I did not find a PR-added test or build log proving the third-party patch was rebuilt, so the remaining verification risk is third-party rebuild coverage rather than code logic. - Scope/focus: The change is small and focused: one third-party patch application is swapped, the old workaround patch is removed, and the new Arrow 17 patch is added.
- Concurrency: The new function-local static pointer initialization remains C++ thread-safe; it avoids constructing a weak inline
std::stringobject in each including translation unit. No Doris shared-state locking changes are involved. - Lifecycle/static initialization: The patch intentionally leaks heap-allocated empty sentinel objects, avoiding teardown-order hazards and moving initialization to
status.cc; no cross-TU static dependency was found. - Configuration/compatibility: No Doris config, storage format, protocol, or FE/BE thrift compatibility change was found.
- Parallel paths: Both
message()anddetail()OK sentinels are handled consistently. Removing the Flight-specific workaround restores upstream rich status conversion while addressing the underlyingStatussentinel crash. - Conditional checks: The
state_ != NULLPTRchecks are equivalent to Arrow's OK/non-OK representation and are used only to select the correct sentinel/message path. - Test coverage/results: No new test artifacts are included. Given this is third-party patch infrastructure, the key missing evidence is an actual Arrow third-party rebuild or BE build using the patched source.
- Observability: No new observability is required for this patch-level crash fix.
- Transaction/data correctness: Not applicable; no data visibility, transaction, delete bitmap, or persistence path is changed.
- Performance: The non-OK path now returns
state_->msg/state_->detaildirectly and the OK path calls out to non-inline sentinels; no concerning hot-path overhead was found. - User focus: No additional user-provided review focus was present.
hello-stephen
commented
May 13, 2026
BE UT Coverage ReportIncrement line coverage Increment coverage report
|
hello-stephen
commented
May 13, 2026
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
hello-stephen
commented
May 13, 2026
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
PR approved by at least one committer and no changes requested. |
PR approved by anyone and no changes requested. |
Uh oh!
There was an error while loading. Please reload this page.
…pache#63191) Related PR: apache#62489 Problem Summary: ```cpp static const std::string no_message = ""; return ok() ? no_message : state_->msg; ``` In the clang-built Doris BE binary, this inline static `std::string` is emitted as a weak/COMDAT object and placed in `.data.rel.ro`. ```bash $ readelf -sW "$BIN" | c++filt | grep 'Status::message.*no_message' 105229: 000000007d6958e0 32 OBJECT WEAK DEFAULT 28 arrow::Status::message[abi:cxx11]() const::no_message[abi:cxx11] $ readelf -SW doris/output/be/lib/doris_be | grep '\[ *28\]' [28] .data.rel.ro PROGBITS 000000007d0f7720 7d0f6720 7e4208 00 WA 0 0 32 ``` After relocation, RELRO makes this section read-only. However, C++ function-local statics are lazily initialized on first execution, so the first call to `Status::message()` tries to construct `no_message` at runtime. The `std::string` constructor writes to the object storage in `.data.rel.ro`, which triggers `SIGSEGV invalid permissions for mapped object` like: *FromStatus(arrow::Status)* ```text *** Query id: 0-0 *** *** is nereids: 0 *** *** tablet id: 0 *** *** Aborted at 1778559900 (unix time) try "date -d @1778559900" if you are using GNU date *** *** Current BE git commitID: f02e9e680c8 *** *** SIGSEGV invalid permissions for mapped object (@0xaaaaf1ecbf28) received by PID 3634450 (TID 3636517 OR 0xfffa35fe97c0) from PID 18446744073473408808; stack trace: *** 0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /root/selectdb-core/be/src/common/signal_handler.h:421 1# PosixSignals::chained_handler(int, siginfo_t*, void*) [clone .part.0] in /opt/jdk/lib/server/libjvm.so 2# JVM_handle_linux_signal in /opt/jdk/lib/server/libjvm.so 3# 0x0000FFFFAE091830 in linux-vdso.so.1 4# std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string<std::allocator<char> >(char const*, std::allocator<char> const&) at /root/toolchain/ldb-toolchain-v0.21/bin/../lib/gcc/aarch64-linux-gnu/13/../../../../include/c++/13/bits/basic_string.h:632 5# arrow::flight::internal::TransportStatus::FromStatus(arrow::Status const&) in /opt/selectdb/4.1.3.2026042721/be/lib/doris_be 6# arrow::flight::transport::grpc::ToGrpcStatus(arrow::Status const&, grpc::ServerContext*) in /opt/selectdb/4.1.3.2026042721/be/lib/doris_be 7# arrow::flight::transport::grpc::(anonymous namespace)::GrpcServiceHandler::DoGet(grpc::ServerContext*, arrow::flight::protocol::Ticket const*, grpc::ServerWriter<arrow::flight::protocol::FlightData>*) in /opt/selectdb/4.1.3.2026042721/be/lib/doris_be 8# grpc::Status grpc::internal::CatchingFunctionHandler<grpc::internal::ServerStreamingHandler<arrow::flight::protocol::FlightService::Service, arrow::flight::protocol::Ticket, arrow::flight::protocol::FlightData>::RunHandler(grpc::internal::MethodHandler::HandlerParameter const&)::{lambda()#1}>(grpc::internal::ServerStreamingHandler<arrow::flight::protocol::FlightService::Service, arrow::flight::protocol::Ticket, arrow::flight::protocol::FlightData>::RunHandler(grpc::internal::MethodHandler::HandlerParameter const&)::{lambda()#1}&&) in /opt/selectdb/4.1.3.2026042721/be/lib/doris_be 9# grpc::internal::ServerStreamingHandler<arrow::flight::protocol::FlightService::Service, arrow::flight::protocol::Ticket, arrow::flight::protocol::FlightData>::RunHandler(grpc::internal::MethodHandler::HandlerParameter const&) in /opt/selectdb/4.1.3.2026042721/be/lib/doris_be 10# grpc::Server::SyncRequest::ContinueRunAfterInterception() in /opt/selectdb/4.1.3.2026042721/be/lib/doris_be 11# grpc::Server::SyncRequest::Run(std::shared_ptr<grpc::Server::GlobalCallbacks> const&, bool) in /opt/selectdb/4.1.3.2026042721/be/lib/doris_be 12# grpc::ThreadManager::MainWorkLoop() in /opt/selectdb/4.1.3.2026042721/be/lib/doris_be 13# grpc::ThreadManager::WorkerThread::WorkerThread(grpc::ThreadManager*)::$_0::__invoke(void*) in /opt/selectdb/4.1.3.2026042721/be/lib/doris_be 14# grpc_core::(anonymous namespace)::ThreadInternalsPosix::ThreadInternalsPosix(char const*, void (*)(void*), void*, bool*, grpc_core::Thread::Options const&)::{lambda(void*)#1}::__invoke(void*) in /opt/selectdb/4.1.3.2026042721/be/lib/doris_be 15# start_thread in /lib64/libc.so.6 16# thread_start in /lib64/libc.so.6 ``` *ToStatus()* ```text *** SIGSEGV invalid permissions for mapped object (@0x56448ff1ef38) received by PID 66637 (TID 67372 OR 0x7f2687017640) from PID 18446744071829581624; stack trace: *** 0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at ../src/common/signal_handler.h:417 1# PosixSignals::chained_handler(int, siginfo*, void*) [clone .part.0] in /usr/lib/jvm/java-17-openjdk-amd64/lib/server/libjvm.so 2# JVM_handle_linux_signal in /usr/lib/jvm/java-17-openjdk-amd64/lib/server/libjvm.so 3# 0x00007F2836CE7520 in /lib/x86_64-linux-gnu/libc.so.6 4# std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string<std::allocator<char> >(char const*, std::allocator<char> const&) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/basic_string.h:707 5# arrow::Status::WithDetail(std::shared_ptr<arrow::StatusDetail>) const in /mnt/disk1/PERFORMANCE_ENV/be/lib/doris_be 6# arrow::flight::internal::TransportStatus::ToStatus() const in /mnt/disk1/PERFORMANCE_ENV/be/lib/doris_be 7# arrow::flight::transport::grpc::FromGrpcStatus(grpc::Status const&, grpc::ClientContext*) in /mnt/disk1/PERFORMANCE_ENV/be/lib/doris_be 8# arrow::flight::transport::grpc::(anonymous namespace)::GrpcClientInterceptorAdapter::Intercept(grpc::experimental::InterceptorBatchMethods*) in /mnt/disk1/PERFORMANCE_ENV/be/lib/doris_be 9# grpc::internal::InterceptorBatchMethodsImpl::RunInterceptors() in /mnt/disk1/PERFORMANCE_ENV/be/lib/doris_be 10# grpc::internal::CallOpSet<grpc::internal::CallOpRecvInitialMetadata, grpc::internal::CallOpClientRecvStatus, grpc::internal::CallNoOp<3>, grpc::internal::CallNoOp<4>, grpc::internal::CallNoOp<5>, grpc::internal::CallNoOp<6> >::FinalizeResult(void**, bool*) in /mnt/disk1/PERFORMANCE_ENV/be/lib/doris_be 11# grpc::ClientReaderWriter<arrow::flight::protocol::FlightData, arrow::flight::protocol::FlightData>::Finish() in /mnt/disk1/PERFORMANCE_ENV/be/lib/doris_be 12# arrow::flight::transport::grpc::(anonymous namespace)::FinishableDataStream<grpc::ClientReaderWriter<arrow::flight::protocol::FlightData, arrow::flight::protocol::FlightData>, arrow::flight::internal::FlightData>::DoFinish() in /mnt/disk1/PERFORMANCE_ENV/be/lib/doris_be 13# arrow::flight::transport::grpc::(anonymous namespace)::WritableDataStream<grpc::ClientReaderWriter<arrow::flight::protocol::FlightData, arrow::flight::protocol::FlightData>, arrow::flight::internal::FlightData>::DoFinish() in /mnt/disk1/PERFORMANCE_ENV/be/lib/doris_be 14# arrow::flight::internal::ClientDataStream::Finish(arrow::Status) in /mnt/disk1/PERFORMANCE_ENV/be/lib/doris_be 15# arrow::flight::ClientStreamReader::Next() in /mnt/disk1/PERFORMANCE_ENV/be/lib/doris_be 16# doris::PythonClient::read_batch(std::shared_ptr<arrow::RecordBatch>*) at ./be/build_RELEASE/../src/udf/python/python_client.cpp:136 17# doris::PythonUDFClient::evaluate(arrow::RecordBatch const&, std::shared_ptr<arrow::RecordBatch>*) at ./be/build_RELEASE/../src/udf/python/python_udf_client.cpp:36 ``` ### fix Move the empty message/detail sentinels out of the header inline path and make non-OK statuses return `state_->msg` directly. This avoids touching the empty OK-message sentinel on error paths and prevents the inline weak/COMDAT `std::string` object from being lazily constructed from a read-only mapping.
Related PR: #63191 Problem Summary: Arrow 17 defaults to `C++17` when CMAKE_CXX_STANDARD is not specified, while Doris BE is built with `C++20`. This can make header-defined inline/template code from Arrow Flight and its dependencies be compiled under different C++ standard modes in the same final binary. In particular, Arrow Status-related inline paths may generate different implementations across C++17 and C++20, such as different initialization strategies for function-local static std::string objects: code: ```cpp const std::string& get_empty_string() { static const std::string s = ""; return s; } ``` cpp17 lazy initialization: ```asm get_empty_string[abi:cxx11](): push rbp mov rbp, rsp sub rsp, 64 cmp byte ptr [rip + guard variable for get_empty_string[abi:cxx11]()::s[abi:cxx11]], 0 jne .LBB0_4 lea rdi, [rip + guard variable for get_empty_string[abi:cxx11]()::s[abi:cxx11]] call __cxa_guard_acquire@PLT cmp eax, 0 je .LBB0_4 lea rdx, [rbp - 33] mov qword ptr [rbp - 32], rdx mov rax, qword ptr [rbp - 32] mov qword ptr [rbp - 8], rax lea rdi, [rip + get_empty_string[abi:cxx11]()::s[abi:cxx11]] lea rsi, [rip + .L.str] call std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>::basic_string<std::allocator<char>>(char const*, std::allocator<char> const&) jmp .LBB0_3 .LBB0_3: lea rax, [rbp - 33] mov qword ptr [rbp - 24], rax mov rdi, qword ptr [rbp - 24] call std::__new_allocator<char>::~__new_allocator() [base object destructor] lea rdi, [rip + std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>::~basic_string() [base object destructor]] lea rsi, [rip + get_empty_string[abi:cxx11]()::s[abi:cxx11]] lea rdx, [rip + __dso_handle] call __cxa_atexit@PLT lea rdi, [rip + guard variable for get_empty_string[abi:cxx11]()::s[abi:cxx11]] call __cxa_guard_release@PLT ``` cpp20 constant initialization: ```asm get_empty_string[abi:cxx11](): push rbp mov rbp, rsp lea rax, [rip + get_empty_string[abi:cxx11]()::s[abi:cxx11]] pop rbp ret get_empty_string[abi:cxx11]()::s[abi:cxx11]: .quad get_empty_string[abi:cxx11]()::s[abi:cxx11]+16 .quad 0 .zero 16 ``` Mixing those definitions through `weak/COMDAT` symbols is not a supported build model and can surface as runtime crashes in Flight error/status handling paths.
Related PR: #63191 Problem Summary: Arrow 17 defaults to `C++17` when CMAKE_CXX_STANDARD is not specified, while Doris BE is built with `C++20`. This can make header-defined inline/template code from Arrow Flight and its dependencies be compiled under different C++ standard modes in the same final binary. In particular, Arrow Status-related inline paths may generate different implementations across C++17 and C++20, such as different initialization strategies for function-local static std::string objects: code: ```cpp const std::string& get_empty_string() { static const std::string s = ""; return s; } ``` cpp17 lazy initialization: ```asm get_empty_string[abi:cxx11](): push rbp mov rbp, rsp sub rsp, 64 cmp byte ptr [rip + guard variable for get_empty_string[abi:cxx11]()::s[abi:cxx11]], 0 jne .LBB0_4 lea rdi, [rip + guard variable for get_empty_string[abi:cxx11]()::s[abi:cxx11]] call __cxa_guard_acquire@PLT cmp eax, 0 je .LBB0_4 lea rdx, [rbp - 33] mov qword ptr [rbp - 32], rdx mov rax, qword ptr [rbp - 32] mov qword ptr [rbp - 8], rax lea rdi, [rip + get_empty_string[abi:cxx11]()::s[abi:cxx11]] lea rsi, [rip + .L.str] call std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>::basic_string<std::allocator<char>>(char const*, std::allocator<char> const&) jmp .LBB0_3 .LBB0_3: lea rax, [rbp - 33] mov qword ptr [rbp - 24], rax mov rdi, qword ptr [rbp - 24] call std::__new_allocator<char>::~__new_allocator() [base object destructor] lea rdi, [rip + std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>::~basic_string() [base object destructor]] lea rsi, [rip + get_empty_string[abi:cxx11]()::s[abi:cxx11]] lea rdx, [rip + __dso_handle] call __cxa_atexit@PLT lea rdi, [rip + guard variable for get_empty_string[abi:cxx11]()::s[abi:cxx11]] call __cxa_guard_release@PLT ``` cpp20 constant initialization: ```asm get_empty_string[abi:cxx11](): push rbp mov rbp, rsp lea rax, [rip + get_empty_string[abi:cxx11]()::s[abi:cxx11]] pop rbp ret get_empty_string[abi:cxx11]()::s[abi:cxx11]: .quad get_empty_string[abi:cxx11]()::s[abi:cxx11]+16 .quad 0 .zero 16 ``` Mixing those definitions through `weak/COMDAT` symbols is not a supported build model and can surface as runtime crashes in Flight error/status handling paths.
Related PR: #62489
Problem Summary:
In the clang-built Doris BE binary, this inline static
std::stringis emitted as a weak/COMDAT object and placed in.data.rel.ro.After relocation, RELRO makes this section read-only. However, C++ function-local statics are lazily initialized on first execution, so the first call to
Status::message()tries to constructno_messageat runtime. Thestd::stringconstructor writes to the object storage in.data.rel.ro, which triggersSIGSEGV invalid permissions for mapped objectlike:FromStatus(arrow::Status)
ToStatus()
fix
Move the empty message/detail sentinels out of the header inline path and make non-OK statuses return
state_->msgdirectly. This avoids touching the empty OK-message sentinel on error paths and prevents the inline weak/COMDATstd::stringobject from being lazily constructed from a read-only mapping.