Uh oh!
There was an error while loading. Please reload this page.
refactor: de-duplicate the dispatch helpers into shared headers - #90
Conversation
Every src/lib-cpu/<module>.cpp and src/lib-cuda/<module>.cpp opened with the
same private prologue: pointer marshalling macros, a 32-bit-index predicate,
and a family of argument checks. There were 19 copies of VOIDPTR, 19 of
CANUSE32BITS, 19 of CHECK_NO_LANES, 17 of CHECK_SAME and 17 of
CHECK_SAME_DTYPE, plus 9 copies each of the hub's IS_CPU / IS_CUDA, three of
as_weights() and four of _reg_stream().
They had already drifted. Three of them are not a formatting difference:
CVOIDPTR posdef.cpp alone guards against a null `data`, because an
absent optional operand arrives as a descriptor with
data == nullptr and offsetting a null pointer is UB.
CHECK_SAME_BATCH distance.cpp and posdef.cpp alone also reject ndim < D,
because they derive D from an operand's own ndim and would
otherwise read past the end of shape[].
CHECK_SAME_SHAPE was TWO DIFFERENT MACROS SHARING ONE NAME -- a 2-argument
whole-shape check in distance.cpp, a 3-argument leading-D
check in the regularisers and solve_field.
That last one is the argument for doing this at all: with 17 private copies,
one name silently meant two things, and nothing could have told you.
WHAT MOVED WHERE
include/fastfields/core/dispatch.h (new)
FF_VOIDPTR, FF_CVOIDPTR, FF_CVOIDPTR_OR_NULL, FF_CANUSE32BITS, the
FF_CHECK_* family, and as_weights().
In core/ because src/lib-cuda is compiled by nvcc while src/lib-cpu and
src/lib use the host compiler: anything all three share has to be
backend-agnostic, and core/ is the only directory that is so by
contract. api/checks.h is host-only and was the wrong home.
include/fastfields/api/cuda/stream.h (new)
_reg_stream(). Names cudaStream_t, so it is CUDA-only by construction
and belongs under api/cuda/ -- which also keeps CI's path filter honest.
include/fastfields/api/checks.h (extended)
is_cpu() / is_cuda() inline functions, replacing IS_CPU / IS_CUDA. Used
only by the host-compiled hub. A function in ff:: is collision-safe with
no prefix at all, so these two are deleted rather than renamed.
All three preserved divergences keep their own name -- FF_CVOIDPTR_OR_NULL,
FF_CHECK_SAME_BATCH_ND, FF_CHECK_SAME_SHAPE_N -- so the difference is visible
at the call site instead of hiding in one file's prologue.
NAMING
A macro that lives in a .cpp cannot collide with anything; a macro in an
installed header is a name taken from every downstream translation unit. So
the moment one of these was hoisted it had to be prefixed, which is why
de-duplicating and prefixing are one change and not two. (VOIDPTR and friends
were in fact already on the public surface via api/{cpu,cuda}/
pushpull_dispatch.h, which the audit of "17 copies in .cpp files" had missed;
the true counts are 19 and 17.)
EVIDENCE
Not asserted, proved. tools/macro-equivalence.py preprocesses every
(file, macro) pair on both sides of the change and compares token streams:
121 macro expansions compared across 23 files:
119 exact, 2 balanced-paren-only, 0 MISMATCHED
The two are FF_CVOIDPTR_OR_NULL, which composes FF_CVOIDPTR and so carries one
extra balanced parenthesis pair; the tool checks the token streams are equal
once parentheses are removed. A whole-translation-unit token diff was tried
first and is the wrong instrument -- core/dispatch.h pulls in <vector> and
makes as_weights() visible everywhere, which swamps the signal with additive,
behaviour-free noise.
The rewrite itself is tools/dedup-dispatch-helpers.py, committed so that the
~1100 renamed call sites can be re-derived and diffed rather than read.balbasty
commented
Aug 19, 2026
Gate: unmoved
Every row matches
Layered evidence
The |
balbasty
commented
Aug 19, 2026
Local clean-worktree run finished, confirming the CI numbers on a second machine and toolchain invocation: Per-suite, identical to Also verified: Generated by Claude Code |
balbasty
commented
Aug 19, 2026
Validation status at hand-off
All four boundary/spline compile policies agree with the reference oracle, ASan
Generated by Claude Code |
Keeps both additions to CLAUDE.md: main's corrected CUDA memory note (the measured 12.98 GB reg_flow peak, replacing the stale ~3.8 GB figure) and this branch's FF_-prefix rule for macros in installed headers.
Uh oh!
There was an error while loading. Please reload this page.
Brings in #87, #90, #91 and #95. One conflict, in include/fastfields/impl/kernels/parallel.h: #91 renamed FF_NAMESPACE_BEGIN(FF) to FF_NAMESPACE_BEGIN(FF_NS) on the line this branch inserts the FF_GRAIN_SIZE block above. Resolved by keeping both -- the new block, then main's FF_NS spelling. Everything else auto-merged. #91's renames do not touch anything this branch depends on: has_atomic_add / anyAtomicAdd keep their names, FF_NS still expands to ff, and the CUDEV -> FF_CUDEV rename is confined to the CUDA half of atomic.h. FF_GRAIN_SIZE, the one macro this branch adds to an installed header, already satisfies #91's FF_-prefix rule -- `tools/rename-macros.py --check` reports "0 file(s) would change" and "include/ is clean". Re-verified on the merge result: * tools/test-baseline.sh --legs default,lib -> byte-identical to tools/test-baseline.expected. 13 suites, 59,886 checks, 0 failures. * -DFF_GRAIN_SIZE=1 -> 59,886 / 13 / 0. * -DFF_GRAIN_SIZE=1 + TSan, FF_NUM_THREADS=4, halt_on_error=1 -> 59,886 / 13 / 0, zero reports. * clone syscalls: 0 across all 13 binaries at the shipping grain size, 2 per binary at FF_GRAIN_SIZE=1. The threshold is unchanged by the merge (0 clones at n=32768, 2 at n=32769, on main and on this branch alike). * The thread-pool defects still reproduce on main at f63c7d8: the data race is deterministic under TSan (threadpool.h:148 write / :164 read) and the lost-wakeup deadlock is stochastic (8/320 trials over FF_NUM_THREADS 8/16/32/64). Both are gone on this branch: 0/320 hangs, 0 TSan reports.
De-duplicates the copy-pasted helper macros and functions in the two
dtype-dispatch layers and the hub into shared headers, giving them
collision-safe
FF_names as they move.This is PR 1 of 2. The follow-up
(
refactor/prefix-public-macros) prefixes the remaining unprefixed publicmacros,
FFitself included. De-dup first, because it is the smaller and moresubstantive change, and separating them makes a regression trivially
bisectable.
What was duplicated
VOIDPTR,CANUSE32BITS,CHECK_NO_LANESsrc/lib-cpu(8–10) +src/lib-cuda(9) + bothapi/*/pushpull_dispatch.hCHECK_SAME,CHECK_SAME_DTYPECVOIDPTRCHECK_SAME_BATCHCHECK_SAME_SHAPEIS_CPU,IS_CUDAsrc/lib(hub)as_weights(),_reg_stream()reg_field*/reg_flow*Note the counts are 19/17, not the 17/15 in the original audit:
VOIDPTRandfriends were already on the installed public surface via
include/fastfields/api/{cpu,cuda}/pushpull_dispatch.h. They were never purelyTU-private, which strengthens rather than weakens the case for the rename.
Three of them had genuinely diverged
Not formatting — behaviour:
CVOIDPTRposdef.cppalone guards a nulldata; an absent optional operand arrives as a descriptor withdata == nullptr, and offsetting a null pointer is UBFF_CVOIDPTR_OR_NULLCHECK_SAME_BATCHdistance.cpp/posdef.cppalone also rejectndim < D— they deriveDfrom an operand's ownndimand would otherwise read past the end ofshape[]FF_CHECK_SAME_BATCH_NDCHECK_SAME_SHAPEdistance.cpp, a 3-arg leading-Dcheck in the regularisers andsolve_fieldFF_CHECK_SAME_SHAPE/FF_CHECK_SAME_SHAPE_NThe last one is the argument for doing this at all. With 17 private copies, one
name silently meant two things, and nothing in the tree could have told you.
Each divergence keeps its own name so the difference is visible at the call
site rather than hiding in one file's prologue.
(
CHECK_NO_LANESandCHECK_SAME_DTYPEalso had two spellings each — thosereally are line-wrapping only, and the proof below confirms it.)
Where things went, and why
Placement is by audience, and the audience question that bites is which
compiler sees the code.
include/fastfields/core/dispatch.h(new)FF_VOIDPTR,FF_CVOIDPTR,FF_CVOIDPTR_OR_NULL,FF_CANUSE32BITS, theFF_CHECK_*family,as_weights()src/lib-cudais compiled by nvcc whilesrc/lib-cpuandsrc/libuse the host compiler. Anything all three share must be backend-agnostic, andcore/is the only directory that is so by contract.api/checks.his host-only and was the wrong home.include/fastfields/api/cuda/stream.h(new)_reg_stream()cudaStream_t, so CUDA-only by construction. Also keeps CI's path filter honest — incore/it would have triggered the CPU legs for nothing.include/fastfields/api/checks.h(extended)is_cpu()/is_cuda()inline functionsff::is collision-safe with no prefix at all, so these two macros are deleted rather than renamed.core/dispatch.hdeliberately does not wrap the checks indo { … } while (0), and says so: the 17 copies did not, and wrapping themwould silently change which statements a brace-less
if (cond) FF_CHECK_…(…);guards. Tightening that is a behaviour change andbelongs in its own commit, measured against the CPU suite.
The naming rule applied
Macros private to a single
.cppare exempt: they never leave the translationunit. But the moment one is hoisted into a header it stops being private —
which is exactly why de-duplicating and prefixing are one change and not two.
Where an
inlinefunction will do, prefer it and delete the macro outright(
IS_CPU/IS_CUDAhere;uchar_tin PR 2).Evidence: proved, not asserted
tools/macro-equivalence.pypreprocesses every (file, macro) pair on bothsides of the change and compares token streams:
The two non-exact ones are
FF_CVOIDPTR_OR_NULL, which composesFF_CVOIDPTRand so carries one extra balanced parenthesis pair; the tool only accepts
that after checking the token streams are identical once parentheses are
removed. Everything else matches exactly.
A whole-translation-unit token diff was tried first and is the wrong
instrument —
core/dispatch.hpulls in<vector>and makesas_weights()visible everywhere, which swamps the signal with additive, behaviour-free
noise. Comparing the expansions in isolation asks the question that matters.
Reviewing this
Do not read 1100 renamed call sites. Re-derive them:
Both scripts are committed and idempotent for exactly this reason.
Validation
make test CXX=clang++— the CPU suite, 59,886 checks across 13 suites,0 failures, unchanged. (Results are in the review comment below once the
clean-worktree run finishes.)
codespellclean.build-cudain CI isthe bar.
core/dispatch.hcorrectly triggers the full matrix.Note on the
clang-formatcheckIt will report against this PR, and its report will be empty. The job is
broken, independently of this change:
git-clang-format-18 --diffexits 1when the diff is non-empty, and the step runs it under
set -euo pipefailin acommand substitution, so the script dies at the assignment before reaching
the
casethat would print the diff. That is why #85 (docs-only) shows a bareexit 1with no output. It iscontinue-on-error: true, so it does not block.Reproduced locally; details in the task report. Not fixed here — the workflow
file is already being edited by #87.
For the record, the tree is not clang-format-clean on
maineither(
src/lib-cpu/splinc.cppalone differs by 154 lines), which is why that job isadvisory in the first place.
Generated by Claude Code