Describe the bug, including details regarding any error messages, version, and platform.
I have a case where my C++ application that uses arrow hangs on shutdown after failing to construct a thread for the IOThreadPool.
Expected behavior: clean shutdown after catching the exception or some kind of fatal condition
Actual behavior: hang on shutdown after catching the exception
Platform: Linux x86_64
Arrow Version: 19.0.1 (but based on reading code I believe this to still be an issue on main)
It is pretty easy to trigger std::thread::thread() to throw by setting RLIMIT_NPROC to something extreme like 1.
If std::thread::thread() throws in ThreadPool::LaunchWorkersUnlocked() there is a state_->workers entry even though the thread was never spawned. It is the thread's responsibility to remove the list entry when it exits so ThreadPool::Shutdown() never reaches its state_->workers_.empty() condition.
This is usually not noticeable because an abnormal shutdown will not call the IOThreadPool's destructor. In my case, I have a C++ binary that catches the exception then exits "normally" with an error message and exit code. This does call the ThreadPool destructor leading to the hang.
Hopefully this is clear. If it is not, I could put together a repro example.
Component(s)
C++
Describe the bug, including details regarding any error messages, version, and platform.
I have a case where my C++ application that uses arrow hangs on shutdown after failing to construct a thread for the IOThreadPool.
Expected behavior: clean shutdown after catching the exception or some kind of fatal condition
Actual behavior: hang on shutdown after catching the exception
Platform: Linux x86_64
Arrow Version: 19.0.1 (but based on reading code I believe this to still be an issue on main)
It is pretty easy to trigger
std::thread::thread()to throw by settingRLIMIT_NPROCto something extreme like1.If
std::thread::thread()throws inThreadPool::LaunchWorkersUnlocked()there is astate_->workersentry even though the thread was never spawned. It is the thread's responsibility to remove the list entry when it exits soThreadPool::Shutdown()never reaches itsstate_->workers_.empty()condition.This is usually not noticeable because an abnormal shutdown will not call the IOThreadPool's destructor. In my case, I have a C++ binary that catches the exception then exits "normally" with an error message and exit code. This does call the ThreadPool destructor leading to the hang.
Hopefully this is clear. If it is not, I could put together a repro example.
Component(s)
C++