Found while measuring dispatch shapes for #158; it is not something either that
PR or #159 touches, and I don't want it to get lost in a review thread.
The measurement
BOUNDFLAGS / SPLINEFLAGS exist to bound instantiation count — per
src/lib-cuda/Makefile, the boundary policy "is what brought this build back
from a ~16 GB ptxas OOM". Compiling each module under the two extreme policies
and counting defined text/weak symbols (clang++ -std=c++11 -fPIC,
-DFF_STATIC_BOUNDS=1 -DFF_STATIC_SPLINES=1 vs =0 =0):
| module | all-static | all-dynamic | effect |
|---|
pushpull (-O0) | 73,604 | 2,084 | −97.2% |
resize (-O0) | 15,015 | 15,015 | 0 |
resize (-O1) | 4,555 | 4,555 | 0 |
restrict (-O1) | 4,778 | 4,778 | 0 |
splinc (-O1) | 391 | 391 | 0 |
Not "less effective" — exactly zero, to the symbol. The knob does nothing
for those three.
Why
pushpull dispatches through the FF_BOUND_<NAME> / FF_SPLINE_<NAME>
selectors, which collapse onto the shared Dynamic instantiation when the
policy says so:
// api/cpu/pushpull_dispatch.hcasebound_t::Zero: FF_PP_DTYPE(D, I, FF_BOUND_ZERO, FN, args); break;
resize, restrict and splinc name the enumerator directly, so there is
nothing for the policy to redirect:
// src/lib-cpu/resize.cppcasebound_t::Zero: RS_DTYPE(D, I, bound_t::Zero, args); break;
// src/lib-cpu/splinc.cppcasebound_t::Zero: return _splinc<NP, bound_t::Zero, S, O>(args);
Same on the CUDA side (src/lib-cuda/{resize,restrict,splinc}.cpp). A census
of the whole tree: FF_BOUND_* appears in the six regulariser TUs and the two
pushpull_dispatch.h headers, and nowhere else.
Why it may matter
Those three are not free on the CUDA side. From the measured table above
MODULES in src/lib-cuda/Makefile:
resize 2.00 GB 188 s <-- 4th heaviest module
restrict 1.30 GB 113 s
splinc 0.42 GB 67 s
3.72 GB of a 16 GB runner, in a build the same file describes as having no
headroom — "-j2 is luck, not headroom", with two of the listed overlap pairs
summing to 15.59 GB and 21.90 GB. Whatever slack the bound/spline policy could
buy, it currently cannot buy any of it here.
There is also a second-order sign that this is unintentional: resize and
restrict still carry FF_TEST_SPARSE — the hand-written covering subset
that throws on most (order, bound) combinations to keep test builds cheap.
pushpull had the same hack and it was removed, with the reasoning recorded
in api/cpu/pushpull_dispatch.h:
There used to be a second, hand-duplicated FF_PP_ORDER (behind
-DFF_TEST_SPARSE) … That is now redundant with, and weaker than, the
FF_BOUND_<NAME>/FF_SPLINE_<NAME> Dynamic-routing policy …
So resize/restrict are still using the older, weaker workaround for exactly
the problem the policy was introduced to solve — which is what you would expect
if they were simply missed when the selectors were introduced, rather than
deliberately excluded.
What I am not claiming
- Not that this is a bug. There may be a reason — a kernel that cannot take
bound::utils<Dynamic>, or a deliberate call that these are hot enough to
want every bound statically specialised. pushpull proves the machinery
works for a bound::utils-based module, but that is an argument, not a proof
for these three. - Not that switching them is free. Routing an axis through
Dynamic moves
the condition from a template parameter to a runtime data member. That is a
per-voxel cost in the inner loop, and there is no GPU in CI, so nobody here
can measure what it would cost resize. Same unbenchmarked-knob caveat the
index axis carries. - Not that it should change now. Anything that shifts the relative phase of
two heavy CUDA compiles reshuffles the -j2 overlap, which the Makefile is
explicit about.
The question
Is the asymmetry deliberate? If it is, the BOUNDFLAGS documentation in
CLAUDE.md and both Makefiles should say which modules the policy actually
governs, because today they read as if it were tree-wide. If it is not, the
change is mechanical — swap the literals for the FF_BOUND_* / FF_SPLINE_*
selectors, which would also retire the last two FF_TEST_SPARSE users — and it
is worth a measured build-cuda run to see what it buys before anyone commits
to it.
Reproduce with:
for pol in "-DFF_STATIC_BOUNDS=1 -DFF_STATIC_SPLINES=1" "-DFF_STATIC_BOUNDS=0 -DFF_STATIC_SPLINES=0"; do
clang++ -std=c++11 -O1 -fPIC $pol -Iinclude -c src/lib-cpu/resize.cpp -o /tmp/r.o
nm -C /tmp/r.o | grep -cE ' [TWt] '
done
Found while measuring dispatch shapes for #158; it is not something either that
PR or #159 touches, and I don't want it to get lost in a review thread.
The measurement
BOUNDFLAGS/SPLINEFLAGSexist to bound instantiation count — persrc/lib-cuda/Makefile, the boundary policy "is what brought this build backfrom a ~16 GB ptxas OOM". Compiling each module under the two extreme policies
and counting defined text/weak symbols (
clang++ -std=c++11 -fPIC,-DFF_STATIC_BOUNDS=1 -DFF_STATIC_SPLINES=1vs=0 =0):pushpull(-O0)resize(-O0)resize(-O1)restrict(-O1)splinc(-O1)Not "less effective" — exactly zero, to the symbol. The knob does nothing
for those three.
Why
pushpulldispatches through theFF_BOUND_<NAME>/FF_SPLINE_<NAME>selectors, which collapse onto the shared
Dynamicinstantiation when thepolicy says so:
resize,restrictandsplincname the enumerator directly, so there isnothing for the policy to redirect:
Same on the CUDA side (
src/lib-cuda/{resize,restrict,splinc}.cpp). A censusof the whole tree:
FF_BOUND_*appears in the six regulariser TUs and the twopushpull_dispatch.hheaders, and nowhere else.Why it may matter
Those three are not free on the CUDA side. From the measured table above
MODULESinsrc/lib-cuda/Makefile:3.72 GB of a 16 GB runner, in a build the same file describes as having no
headroom — "
-j2is luck, not headroom", with two of the listed overlap pairssumming to 15.59 GB and 21.90 GB. Whatever slack the bound/spline policy could
buy, it currently cannot buy any of it here.
There is also a second-order sign that this is unintentional:
resizeandrestrictstill carryFF_TEST_SPARSE— the hand-written covering subsetthat throws on most (order, bound) combinations to keep test builds cheap.
pushpullhad the same hack and it was removed, with the reasoning recordedin
api/cpu/pushpull_dispatch.h:So
resize/restrictare still using the older, weaker workaround for exactlythe problem the policy was introduced to solve — which is what you would expect
if they were simply missed when the selectors were introduced, rather than
deliberately excluded.
What I am not claiming
bound::utils<Dynamic>, or a deliberate call that these are hot enough towant every bound statically specialised.
pushpullproves the machineryworks for a
bound::utils-based module, but that is an argument, not a prooffor these three.
Dynamicmovesthe condition from a template parameter to a runtime data member. That is a
per-voxel cost in the inner loop, and there is no GPU in CI, so nobody here
can measure what it would cost
resize. Same unbenchmarked-knob caveat theindex axis carries.
two heavy CUDA compiles reshuffles the
-j2overlap, which the Makefile isexplicit about.
The question
Is the asymmetry deliberate? If it is, the
BOUNDFLAGSdocumentation inCLAUDE.mdand both Makefiles should say which modules the policy actuallygoverns, because today they read as if it were tree-wide. If it is not, the
change is mechanical — swap the literals for the
FF_BOUND_*/FF_SPLINE_*selectors, which would also retire the last two
FF_TEST_SPARSEusers — and itis worth a measured
build-cudarun to see what it buys before anyone commitsto it.
Reproduce with: