Parallel sort algorithm performance on Threadripper

[ohhmm]

Hi,

I tried to use std::sort(std::execution::par parallel implementation using custom predicate on gigabytes of data.
Unfortunately its result gave no much benefit comparably to single-thread std::sort.
Simple custom implementation of merge-sort gave much better benefit:

		auto TotalThreads = std::thread::hardware_concurrency();

		template<class Iter, class Pred>
		void quick_merge_sort(Iter first, Iter last, Pred pred)
		{
			auto length = last - first;
			auto chunkSize = length / TotalThreads;
			auto chunks = length / chunkSize;
			if (chunkSize > 1)
			{
				auto tail = length - (chunkSize * chunks);
				auto hasTail = tail > 0;
				auto tailStart = first + chunks * chunkSize;
				{
					std::deque<std::future<void>> tasks;

					for (auto thread = 0; thread < chunks; ++thread) {
						auto chunkStart = first + thread * chunkSize;
						auto chunkEnd = chunkStart + chunkSize;
						tasks.emplace_back(std::async(std::launch::async, &std::sort<Iter, Pred>, chunkStart, chunkEnd, pred));
					}

					if (hasTail)
						std::sort(tailStart, last, pred);

					while (chunkSize < length)
					{
						auto bigChunkStart = first;
						auto bigChunkMiddle = bigChunkStart;
						auto bigChunkEnd = bigChunkStart;
						for (auto thread = 0; thread < chunks; ++thread) {
							tasks.pop_front();
							bigChunkEnd = bigChunkMiddle + chunkSize;
							if (thread % 2) {
								tasks.emplace_back(std::async(std::launch::async, &std::inplace_merge<Iter, Pred>, bigChunkStart, bigChunkMiddle, bigChunkEnd, pred));
								bigChunkStart = bigChunkEnd;
							}
							bigChunkMiddle += chunkSize;
						}

						chunkSize *= 2;
						chunks = length / chunkSize;
					}
				} // ensure tasks completion

				if (hasTail) {
					std::inplace_merge(first, tailStart, last, pred);
				}
			}
			else if (length > 1) {
				std::sort(first, last, pred);
			}

			assert(std::is_sorted(first, last, pred));
		}
	}

This code still has range-way to improve CPU threads utilization, but it gave much more benefit of parallelization.
How much threads are used to sort with parallel executor?

[cbezault]

Miya Natsuhara (@mnatsuhara)

[fsb4000]

Simple custom implementation of merge-sort gave much better benefit:

Did you try std::stable_sort with execution::par?

Implementation of std::sort with parallel executor:

STL/stl/inc/execution

Line 2749 in 392fb6d

template <class _ExPo, class _RanIt, class _Pr, _Enable_if_execution_policy_t<_ExPo> /* = 0 */>

Function that determines the number of threads:

STL/stl/src/parallel_algorithms.cpp

Line 26 in 392fb6d

_NODISCARD unsigned int __stdcall __std_parallel_algorithms_hw_threads() noexcept {

[StephanTLavavej]

Can you provide a complete test case demonstrating the performance issue? (With something that generates the data, of course; we don't care about the data or even about what the predicate does, but we need to have an example of both of them to understand how they influence the implementation.)

[ohhmm]

Hi Stephan T. Lavavej (@StephanTLavavej),

That would be much extra effort.
Do you really need it?
I was hoping you can see the difference using own tests.

Thanks,
Serg

[ohhmm]

I could launch some tests on my local environment for gathering some statistics if it helps?

[MikeGitb]

Can't speak for STL, but fixing a performance problem without having some representative test case along with compilation flags and stuff can be extremely difficult when there isn't something really obvious going on.

[StephanTLavavej]

I agree with what MikeGitb (@MikeGitb) said. We ran performance tests when we first implemented these algorithms, to tune their implementations and to verify that they were worth parallelizing at all - if we had observed what you're observing, we would have addressed the issue. So yes, we do need a test case.

[fsb4000]

I could launch some tests on my local environment for gathering some statistics if it helps?

Could you run the code with your build options:

#include <iostream>
#include <algorithm>
#include <execution>

int main()
{
  std::cout << __std_parallel_algorithms_hw_threads() << '\n';
}

and could you run your code with std::stable_sort with execution::par instead of std::sort?

[ohhmm]

Ok, I can do that. I'll let you know.

[ohhmm]

std::deque of 1513002 272-byte records
__std_parallel_algorithms_hw_threads: 64
std::stable_sort
3131872300 ns
quick_merge_sort
726697900 ns
std::sort
3355967900 ns
std::stable_sort(std::execution::par, ...
598296600 ns
std::sort(std::execution::par, ...
277424400 ns

[ohhmm]

Wait, wrong data. That was half-minute. I'll find that sample. The measurement is for release configuration.