GH-48137: [C++] Restore ThreadPool state when a worker fails to start

[advitrocks9]

Rationale for this change

LaunchWorkersUnlocked appends an entry to state_->workers_ before constructing the thread
that owns it, and only the worker itself erases that entry. If the std::thread constructor
fails, the entry stays behind with nothing left to remove it, so Shutdown waits forever on
workers_.empty(). The destructor takes the same path. The failure also escaped SpawnReal
after tasks_queued_or_running_ had been incremented, so WaitForIdle never returned either,
and once stale entries filled workers_ to capacity the pool stopped launching workers while
Spawn still returned OK for tasks nothing would run.

What changes are included in this PR?

LaunchWorkersUnlocked returns a Status. A failed thread construction erases the entry it had
reserved and returns an error, which SpawnReal and SetCapacity propagate. The task counter is
incremented after the launch rather than before, so a failed launch cannot leak a count.

Are these changes tested?

TestThreadPool.FailedWorkerLaunch lowers RLIMIT_NPROC to 1, spawns a task, restores the soft
limit, and then checks the pool reports no workers and no tasks and still shuts down. It skips on
macOS, where RLIMIT_NPROC counts processes rather than threads, and skips anywhere else the
lowered limit does not stop thread creation, such as under root.

Are there any user-facing changes?

Yes. Spawn, Submit and SetCapacity used to let a std::system_error escape when the OS
refused a new thread. They return an error Status now. ThreadPool::Make is unaffected, since
worker threads are only started on demand and a new pool starts none.

• GitHub Issue: [C++] Hang in ThreadPool::Shutdown() after failure to spawn a worker thread #48137

[github-actions]

⚠️ GitHub issue #48137has been automatically assigned in GitHub to PR creator.

[Copilot]

Pull request overview

Fixes exception-safety holes in Arrow’s C++ ThreadPool worker-launch path so that a failed std::thread construction cannot leave the pool in a wedged state (stale workers_ entries and inflated tasks_queued_or_running_) that would hang Shutdown() / WaitForIdle() and eventually prevent new workers from being started.

Changes:

• Make LaunchWorkersUnlocked() erase the just-appended workers_ entry if std::thread construction throws, then rethrow.
• Move tasks_queued_or_running_ increment in SpawnReal() to after a successful worker launch attempt (keeping the launch heuristic equivalent).
• Add a fork-based regression test that forces thread creation failure and asserts the pool returns to a clean state and can still shut down.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File	Description
cpp/src/arrow/util/thread_pool.h	Adds FRIEND_TEST access for the new fork-safety regression test.
cpp/src/arrow/util/thread_pool.cc	Restores ThreadPool internal invariants when worker thread creation throws; avoids counter leaks on launch failure.
cpp/src/arrow/util/thread_pool_test.cc	Adds FailedWorkerLaunch regression test that validates state restoration after forced thread creation failure.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[advitrocks9]

All three done.

[Copilot]

🟡 Changes recommended

The implementation and test expectations currently conflict with the PR description’s stated exception-propagation behavior, and the new test alters a process-wide rlimit in-process rather than isolating it in a forked child (risking flakiness).

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

• Files reviewed: 3/3 changed files
• Comments generated: 2
• Review effort level: Lite

[github-actions]

⚠️ GitHub issue #48137has been automatically assigned in GitHub to PR creator.

[pitrou]

@github-actions crossbow submit -g cpp

[github-actions]

Revision: 0fa803a

Submitted crossbow builds: ursacomputing/crossbow @ actions-b40897411d

Task	Status
example-cpp-minimal-build-static
example-cpp-minimal-build-static-system-dependency
example-cpp-tutorial
test-build-cpp-fuzz
test-conda-cpp
test-conda-cpp-valgrind
test-debian-13-cpp-amd64
test-debian-13-cpp-i386
test-debian-experimental-cpp-gcc-15
test-fedora-42-cpp
test-ubuntu-22.04-cpp
test-ubuntu-22.04-cpp-bundled
test-ubuntu-22.04-cpp-emscripten
test-ubuntu-22.04-cpp-no-threading
test-ubuntu-24.04-cpp
test-ubuntu-24.04-cpp-bundled-offline
test-ubuntu-24.04-cpp-gcc-13-bundled
test-ubuntu-24.04-cpp-gcc-14
test-ubuntu-24.04-cpp-minimal-with-formats
test-ubuntu-24.04-cpp-thread-sanitizer

[pitrou]

Thanks for the update @advitrocks9 ! LGTM now.

[pitrou]

(note: CI failures are unrelated)