The CPU suite never runs anything in parallel — turn it on, under TSan, and fix the deadlock that finds

[balbasty]

Answers "is the CPU atomic path ever executed?". No — and worse than expected in one direction, better in another. The investigation also turned up a reproducible hang in the shipped thread pool.

Merged with main at f63c7d8 (picks up #87, #90, #91, #95). One conflict, in parallel.h, where #91's FF_NAMESPACE_BEGIN(FF_NS) rename lands on the line this branch inserts above. Every number below was re-measured on the merge result — see §5.

1. The CPU suite has never executed a line of concurrent code

parallel_for reaches the thread pool only when a loop covers strictly more thanGRAIN_SIZE (32768) elements. Every workload in tests/lib-cpu/ is below that. Measured with strace -e trace=clone,clone3 on the 13 test binaries:

test_distance clones=0 test_reg_flow clones=0
test_distance_mesh clones=0 test_reg_op clones=0
test_distance_spline clones=0 test_resize clones=0
test_posdef clones=0 test_restrict clones=0
test_pushpull clones=0 test_solve_field clones=0
test_pushpull_backward clones=0 test_splinc clones=0
test_reg_field clones=0
TOTAL = 0

Zero. The pool is never even constructed (get_global_pool() is lazy). The threshold is exact — a probe at n = 32768 makes 0 clones, at n = 32769 it makes 2, identically on main and on this branch. So internal::invoke_parallel, all of threadpool.h, and every "several threads accumulate into one output" path in pushpull/restrict are compiled, shipped, and reported green by 59,886 passing checks without ever having run multi-threaded.

2. The CAS path is not merely unexecuted — it is uninstantiable

AtomicAdd<true> (the std::atomic compare-exchange loop) is selected by has_atomic_add<T>. That predicate answers false for every T, in every standard:

has_atomic_add<int> = 0 has_atomic_add<float> = 0
has_atomic_add<int64_t> = 0 has_atomic_add<double> = 0
(identical under -std=c++11, -std=c++17 and -std=c++20)

The reason is not "fetch_add for floats is C++20". has_fetch_add probes decltype(&C::fetch_add), and std::atomic<T>::fetch_add is an overload set (with and without a memory_order), so taking its address is ambiguous, SFINAE rejects it, and the detector answers "no" — for integers too, in C++20 too.

And if it ever did answer yes, it would not compile: AtomicAdd<true>::atomicAdd returns void while anyAtomicAdd returns T. Shown by patching only the detector:

atomic.h:82: error: cannot initialize return object of type 'int' with an rvalue of type 'void'
return AtomicAdd<has_atomic_add<T>::value>::atomicAdd(address, val);
note: in instantiation of function template specialization 'ff::anyAtomicAdd<int>' requested here

Not hypothetical rot: git log --follow shows AtomicAdd<true> previously stored a pointer into an uninitialisedstd::atomic and returned from a void function (f6aa841, "was nonsense"), and shipped that way for months. Nothing noticed, because nothing instantiates it.

So the CPU is safe by construction, not by atomicity.has_atomic_add also gates the parallelisation strategy at six sites — impl/cpu/pushpull.h:125,206,428,688 (push, count, pull_backward, grad_backward) and impl/cpu/restrict.h:55,204:

if (has_atomic_add<scalar_t>::value) { /* parallel over ALL elements, atomics resolve collisions */ }
else { /* parallel over the BATCH dim only, spatial loop serial */ }

Since the predicate is always false, the scatter ops always take the disjoint-slice branch and the non-atomic += is never contended. Correct — but one "fix" to the detector away from silent lost updates. Hence the new tests/kernels/atomic/test.cpp, which pins it rather than leaving it as a comment.

3. Turning parallelism on found a real deadlock

-DFF_GRAIN_SIZE=1 + TSan, first run: test_pushpullhung. gdb: main thread blocked in future::get() inside invoke_parallel(0, 9, grain=1), both workers asleep in pthread_cond_wait at threadpool.h:183.

Two defects, both in threadpool.h:

• Data race.Worker::mExit is a plain bool (line 160), written by the pool destructor on the main thread (line 148) and read by the worker's own thread (line 164).
• Lost wakeup → hang.wake() (line 132) calls mCv.notify_one() without ever touching mCvMut, and threadFunc() (line 183) waits on it with no predicate. The worker can check "no work", the pusher can enqueue and notify, and the worker then sleeps on work already in its queue. Because parallel_for is always called from the main thread, pushWork() never matches a worker id and every task lands on mWorkers.front() — so one missed wakeup on that single worker stalls the entire call.

Both reproduce on main at f63c7d8 with a 12-line driver (parallel_for(0, 9, grain=1) in a loop). The two defects fail differently, and it is worth being precise about which:

	main @ f63c7d8	this PR
data race, under TSan, FF_NUM_THREADS=4	1 report, every run (threadpool.h:148 write / :164 read) — deterministic	0 reports
lost-wakeup hang, no sanitizer, 20 trials each at FF_NUM_THREADS 8/16/32/64	8 hangs / 320 trials — stochastic, rate varies with load	0 / 320

FF_NUM_THREADS / OMP_NUM_THREADS / MKL_NUM_THREADS are all honoured, and the default is hardware_concurrency()/2 — 32 workers on a 64-core node. Any production parallel_for over more than 32768 elements is exposed.

4. What this PR changes

• impl/kernels/parallel.h — GRAIN_SIZE overridable via -DFF_GRAIN_SIZE. A threshold, never a correctness switch; 32768 stays the shipping value. Satisfies refactor: prefix every remaining public macro with FF_ #91's prefix rule.
• .github/workflows/ci.yml — a test-cpu (tsan, grain=1) job: the same 13 suites, same 59,886 checks, rebuilt with -DFF_GRAIN_SIZE=1 under ThreadSanitizer with FF_NUM_THREADS=4 and halt_on_error=1. Stronger than adding one big-volume test case, and the gate's numbers do not move. Separate job because TSan and ASan cannot share a binary.
• impl/kernels/threadpool.h — mExit becomes std::atomic<bool>; wake() takes and releases mCvMut before notifying; the wait is predicated on (exiting || queue non-empty). (Incidentally silences a pre-existing -Wreorder-ctor on Worker's init list.)
• tests/kernels/atomic/test.cpp + make test-atomics — pins §2 in-tree. Deliberately not part of make test.

5. Verification, all re-run on the merge result

	suites	checks	failures
tools/test-baseline.sh --legs default,lib	13 (+2 lib)	59,886 (+14)	0
make test-lib-cpu -DFF_GRAIN_SIZE=1	13	59,886	0
same + -fsanitize=thread, FF_NUM_THREADS=4, halt_on_error=1	13	59,886	0, and zero TSan reports

The first row is byte-identical to tools/test-baseline.expected for every default and lib row (diff clean), so the gate is verified in the project's own diff-able format rather than by hand-counting.

• python3 tools/rename-macros.py --check → 0 file(s) would change / include/ is clean.
• codespell --config .codespellrc → clean.
• Clone syscalls: 0 across all 13 binaries at the shipping grain size, 2 per binary at FF_GRAIN_SIZE=1.
• TSan run phase ≈ 2.5 min for all 13 suites (the compile dominates), so the 45-minute timeout matches the existing sanitize leg.

Known residual, deliberately not fixed here

A worker whose own queue is empty but which could steal still waits until someone wakes it, and requestSteal() mutates mStealIter unsynchronised — harmless while only the main thread submits, a race the moment a nested parallel_for appears. Fixing either is a redesign of the pool, not a patch.

Does not touch atomic.h: the CUDA side of that file, the missing half/bf16 CAS fallbacks, the integer-CAS retyping bugs, the BSD-3 attribution gap, and the -gencode matrix are reported separately.