Skip to content

build: make the 32-bit index axis a per-backend option (FF_INDEX32), default unchanged - #143

Merged
balbasty merged 1 commit into
mainfrom
build/index32-per-backend-option
Aug 20, 2026
Merged

build: make the 32-bit index axis a per-backend option (FF_INDEX32), default unchanged#143
balbasty merged 1 commit into
mainfrom
build/index32-per-backend-option

Conversation

@balbasty

@balbastybalbasty commented Aug 20, 2026

Copy link
Copy Markdown
Collaborator

Makes the 32-bit index axis a build option, per backend, defaulting to
today's behaviour on both
. Nothing about the shipping libraries changes; what
changes is that the axis can be turned off for one backend without touching the
other, and that both positions are now built by CI.

Follow-up to the measurement in #94 (draft, not merged). It takes that PR's
FF_INDEX32 mechanism only — not the dispatch-driver experiment, and not the
struct-wrapping that re-indents whole files.

The axis, and what it costs

Every templated kernel below the dispatch layer is templated on offset_t,
which has exactly two values chosen per call by canUse32BitIndexMath:
int32_t when every operand's largest element offset fits in 32 bits,
int64_t otherwise. So the narrow path is exactly ×2 instantiations of
everything underneath
— on CUDA, ×2 device code, ×2 SASS, ×2 ptxas memory.
core/autocast.h exists solely to feed that second instantiation.

The spelling, and why

FF_INDEX32one macro in core/dispatch.h — plus INDEXFLAGS, one make
variable defaulted separately in src/lib-cpu/Makefile and
src/lib-cuda/Makefile.

That is the BOUNDFLAGS/SPLINEFLAGS pattern verbatim, and the per-library
default is the per-backend mechanism: FF_STATIC_BOUNDS is also one macro,
and only differs between backends because the two Makefiles say so. An
FF_INDEX32_CPU / FF_INDEX32_CUDA pair would instead force
core/dispatch.h to branch on __CUDACC__ — putting the policy in the source,
in the one directory that is backend-agnostic by contract — and any third
consumer would have to know both names. INDEXFLAGS also stays outside
CXXFLAGS, for the reason the other two do: CUDA CI overrides CXXFLAGS to
force -O1 and must not silently drop a policy while doing it.

The type the narrow arm names is a typedef, ff::off32_t, not a macro —
CLAUDE.md's own guidance ("prefer an inline function to a macro where one
will do; a name in ff:: is collision-safe without any prefix"). It is also
exactly as wide as int32_t, so the 140-line source change is a pure token
swap and not one line-continuation backslash moves.

Default unmoved — object-level proof

All ten src/lib-cpu modules compiled from main (85fdac7) and from this
branch at the default produce byte-identical objects:

distance IDENTICAL posdef IDENTICAL resize IDENTICAL restrict IDENTICAL
splinc IDENTICAL pushpull IDENTICAL pushpull_backward IDENTICAL
reg_field IDENTICAL reg_flow IDENTICAL solve_field IDENTICAL

(clang++ -std=c++11 -O1 -fPIC -DFF_STATIC_BOUNDS=0 -DFF_STATIC_SPLINES=0,
cmp on the .o.)

Gate, locally, make test-lib-cpu: 13 suites, 59,886 checks, 0 failures, in
both positions. In CI the four pre-existing test-cpu legs, sanitize and
tsan are all green, and test-cpu (clang-index64)'s per-suite counts sum
row-for-row to tools/test-baseline.expected (2352 + 4622 + 704 + 4012 + 308 +
6381 + 19250 + 16347 + 186 + 630 + 65 + 452 + 4577 = 59,886).

tools/rename-macros.py --check0 file(s) would change, include/ is clean. codespell clean.

Both positions build and pass

CI gains the two legs that make this an option rather than a claim:

  • test-cpu (clang-index64) — the whole suite with FF_INDEX32=0. Green.
  • build-cuda becomes a two-leg matrix, index32 (shipping, the gate) and
    index64. Both green, both through the full compile + hub link +
    ldd -r check. The off leg is the cheaper of the two (~19 min vs ~38), and
    it turns the FFMEM table into a direct measurement of the axis on the
    backend where it costs the most.

Numbers

CPUclang++ -O3 -fPIC, BOUNDFLAGS/SPLINEFLAGS at the CUDA shipping
policy. "instantiations" = defined weak symbols in the object.

moduleinstantiationsobjectpeak RSS
reg_flowmain10,21918,952 KB1,120 MB
reg_flowFF_INDEX32=05,123 (−49.9%)9,658 KB (−49.0%)624 MB (−44.3%)
reg_fieldmain9,65519,075 KB1,044 MB
reg_fieldFF_INDEX32=04,857 (−49.7%)9,570 KB (−49.8%)602 MB (−42.3%)
pushpull_backwardmain5,24811,213 KB958 MB
pushpull_backwardFF_INDEX32=02,656 (−49.4%)5,479 KB (−51.1%)562 MB (−41.3%)
posdefmain420671 KB167 MB
posdefFF_INDEX32=0252 (−40.0%)383 KB (−42.9%)150 MB (−10.2%)

Reproduces #94's table (it measured 18,952 KB / 1,126 MB for reg_flow on
main and 9,661 KB / 624 MB with the axis off).

Whole libfastfields-cpu.so, all-Dynamic policy:

exported symbolssize
FF_INDEX32=127,49130.5 MiB
FF_INDEX32=014,051 (−48.9%)15.6 MiB (−49.0%)

The off-position symbol set is a strict subset: 13,440 symbols disappear,
0 appear, and every one that disappears is an offset_t = int32_t template
instantiation. No public entry point changes — the exported ABI is the same
library with half the template bodies.

CUDA — both legs of one build-cuda run, nvcc -O1 -j2, shipping
bound/spline policy. Peak nvcc RSS (GiB) / wall per module:

moduleFF_INDEX32=1FF_INDEX32=0Δ RSS
reg_flow12.97 / 1023 s5.69 / 497 s−56.1%
reg_field8.09 / 791 s3.76 / 343 s−53.5%
reg_field_rls7.07 / 701 s3.42 / 353 s−51.7%
resize2.00 / 193 s1.08 / 113 s−46.2%
reg_flow_rls1.90 / 155 s1.01 / 89 s−46.9%
pushpull_backward1.51 / 167 s0.80 / 102 s−46.9%
pushpull1.48 / 158 s0.79 / 97 s−46.6%
restrict1.30 / 120 s0.68 / 72 s−47.4%
distance0.80 / 118 s0.54 / 61 s−33.1%
splinc0.42 / 68 s0.27 / 37 s−35.5%
posdef0.37 / 40 s0.35 / 28 s−6.9%
total nvcc CPU3532 s1791 s−49.3%
job wall clock~38 min~19 min

CUDA is worse than the host-compiler extrapolation implied, not better:
reg_flow sheds 56.1% rather than the 44.6% clang showed. This table is now
recorded above MODULES in src/lib-cuda/Makefile.

Two things worth flagging from the left-hand column, since the existing note
there claims the heavy modules reproduce to better than 0.1%: reg_flow does
(12.97 vs the recorded 12.98 GiB), reg_field does not (8.09 vs a recorded
8.93). Recorded rather than silently overwritten.

What the stated preference costs — plainly

The decision on #94 is to keep 32-bit dispatch, with a per-backend option
because "it's probably more important on CUDA than on CPU". The
register-pressure instinct is very likely right. The consequence is worth
seeing in one place, without argument:

  • CUDA is the backend whose build budget is exhausted.reg_flow peaks at
    12.97 GiB of a 16 GB runner; build-cuda gates at FF_MEM_BUDGET_KB = 14 GiB
    (ci(cuda): gate nvcc peak memory on a budget, and record why -j2 is not headroom #95); under -j2reg_field + reg_flow peaks sum to over the limit and
    the build is green only because the two spikes have not yet coincided.
  • CPU is the backend with room to spare, 64-bit integer units, and no
    register file to protect.
  • So keeping the axis on CUDA and dropping it on CPU relieves the backend that
    is not constrained and leaves the constrained one exactly where it is.
    After this PR the CUDA build still sits at ~13 GiB against a 16 GB runner,
    and anything that makes a regulariser heavier — -arch/-gencode flags,
    half/bf16, a new energy term — spends from that same ~1 GiB of margin.

That is not an argument against the decision. It is the reason the knob is
worth having: the position is now reversible by one make variable, and the
numbers for the other position are on the record instead of being guessed at.

The benchmark that would settle it — scoped, not run

The narrow path is an ATen inheritance and, as far as anyone can tell, has
never been benchmarked in this repository
. There is no GPU in CI and no nvcc
on the machine this was prepared on, so it cannot be settled here. The spec is
#144: which ops, which shapes, which ncu metrics, and a decision rule
fixed in advance so the result is not re-litigated afterwards. An afternoon's
work once hardware exists.

One related cost, noted and not chased: on CUDA the narrow path is paid for
with a cudaMallocHost + cudaFreeHost per array per call
(core/autocast.h), justified in that header as enabling an async H2D copy —
but 363 of the 391 upload sites under include/fastfields/impl/cuda/ call
the synchronouscopyToDevice (only distance_{euclidean,l1,mesh}.h use
copyToDeviceAsync), so for almost every op there is no async copy for the
pinning to enable. If that cost is real it comes off whatever the register win
is worth, worst exactly where the kernel is cheapest. #144 says to measure it
alongside and report it separately.

Cost of this PR

The build-cuda matrix roughly doubles that job's runner-minutes on
CUDA-triggering changes — but the second leg is the cheaper one (~19 min of the
~57 total), the two run in parallel so wall clock is unchanged, and it is the
only thing that makes the off position a supported configuration rather than an
untested claim.

lint (clang-format, changed lines) is red; it is continue-on-error and
cannot report its own findings (#89), and the source diff here is a
same-width token swap that moves no formatting.

Every templated kernel below the dispatch layer is templated on `offset_t`,
which has exactly two values chosen per call by `canUse32BitIndexMath`:
`int32_t` when every operand's largest element offset fits in 32 bits,
`int64_t` otherwise. That axis costs exactly x2 instantiations of everything
underneath -- on CUDA, x2 device code, x2 SASS and x2 ptxas memory. Measured
on reg_flow, the module that peaks at 12.98 GB of a 16 GB runner: dropping it
is -50.3% instantiations and -44.6% peak compiler RSS (fastfields-lib#94).
Nobody has ever benchmarked what the narrow path buys here -- it is an ATen
inheritance and the argument for it (register pressure) is a GPU argument,
while there is no GPU in CI. So this does not change the default. It makes the
axis a build-time policy, per backend, so the position is reversible once
somebody with hardware can answer the question.
FF_INDEX32=1 (default, both backends) today's behaviour, unchanged
FF_INDEX32=0 the narrow arm names int64_t too, so
both arms are one instantiation, the
axis collapses, and the
canUse32BitIndexMath probe folds away
Spelling follows BOUNDFLAGS/SPLINEFLAGS exactly: ONE macro, with the default
chosen separately by each library's own Makefile via a new INDEXFLAGS variable.
That per-library default IS the per-backend mechanism -- FF_STATIC_BOUNDS is
one macro too, and only differs between backends because the two Makefiles say
so. An FF_INDEX32_CPU/FF_INDEX32_CUDA pair would instead force core/dispatch.h
to branch on __CUDACC__, putting the policy in the source in the one directory
that is backend-agnostic by contract, and every third consumer would have to
know both names. INDEXFLAGS stays outside CXXFLAGS for the same reason the
other two do: CUDA CI overrides CXXFLAGS to force -O1 and must not silently
drop a policy while doing it.
The type the narrow arm names is a typedef, `ff::off32_t`, not a macro --
CLAUDE.md's own guidance (a name in ff:: is collision-safe with no prefix at
all), and it keeps the 140-line source change a pure token swap of the same
width, so not one line-continuation backslash moves.
CI gains the two legs that make this an option rather than a claim: a
`test-cpu (clang-index64)` leg that runs the whole suite with the axis off, and
`build-cuda` as a two-leg matrix over the two positions -- the off leg is the
cheaper of the two and turns the per-module FFMEM table into a direct
measurement of what the axis costs on the backend where it costs the most.
@balbasty
balbasty merged commit de288a9 into mainAug 20, 2026
13 of 14 checks passed
@balbasty
balbasty deleted the build/index32-per-backend-option branch August 20, 2026 11:57
balbasty pushed a commit that referenced this pull request Aug 20, 2026
Resolved by taking main's tree wholesale for include/, src/, tests/ and
CLAUDE.md, then re-running the sweep script over the merged base -- the
procedure the committed script exists for. #146 (the <fastfields/...>
delimiter sweep) and #143 (the per-backend FF_INDEX32 option) both landed
underneath this branch and both touch the header preambles this sweep
rewrites.
balbasty pushed a commit that referenced this pull request Aug 20, 2026
#146 (angle-bracket includes) and #143 (FF_INDEX32) landed on main while
this proposal was being written, which turned the script's central claim
from an assertion into something demonstrable. The rebase was done the way
the docstring prescribes -- revert, merge main, re-run unedited -- and the
same script emitted <fastfields/...> on the new base with no edit and no
hand-merging. Both runs are now recorded in Appendix A.
Also corrects the vox migration cost: 26 FF_NAMESPACE_BEGIN(<module>) sites
across 25 files, not 28 across 26, with the per-module breakdown. The
earlier figure came from a scan that counted non-module namespaces
(bound, spline, meta, internal, _splinc) alongside the module ones.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016AjQcY78NgbagPSbPJRr6Z
balbasty pushed a commit that referenced this pull request Aug 20, 2026
… on the new base
Regenerated by re-running `tools/move-core-headers.py` unedited on
de288a9 (main after #146 and #143), which is the whole point: the earlier
prototype commit was reverted rather than hand-merged, and this is what the
script produced on its own.
10 headers moved, 4,262 lines
81 files rewritten, now emitting <fastfields/...> instead of "fastfields/..."
`--check`: clean, idempotent, no dependency leak
Drop this commit and the design in the preceding ones still stands.
The one-line tools/test-baseline.sh probe update rides along here because
the script does not touch that file: the gate tool probes for
impl/kernels/bounds.h to validate tree layout and dies before running a
single test if it is absent. tools/consolidate.sh and the six-repo probe in
the same script name the same paths and are deliberately left alone -- they
describe the frozen pre-consolidation layout.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016AjQcY78NgbagPSbPJRr6Z
balbasty pushed a commit that referenced this pull request Aug 20, 2026
Brings in #143 (per-backend FF_INDEX32), #145 (#pragma once), #146
(<fastfields/...> includes) and #151 (the measured index-axis table).
One conflict, in src/lib-cuda/reg_flow.cpp: this branch splits the file and
moves the dtype x offset dispatch macros into reg_flow_slice.inl, while #143
rewrote the narrow arm of those same macros from int32_t to off32_t and #146
rewrote the include lines. Resolved by taking this branch's structure with both
of main's changes applied in their new home:
* all 20 narrow dispatch arms in reg_flow_slice.inl now name off32_t, which
is exactly the count main's reg_flow.cpp carries;
* the new files already spelled public includes <fastfields/...> and kept
the same-directory sibling quoted, so #146 needed nothing;
* reg_flow_slice.h switched from an #ifndef guard to #pragma once -- #145
landed the opposite way round from its original proposal, and the new
files followed the proposal rather than the merge.
tools/normalise-header-guards.py --check, tools/normalise-include-delimiters.py
--check and tools/rename-macros.py --check are all clean on the result.
Re-verified after the merge:
* tools/test-baseline.sh --legs default,lib -> row-for-row identical to
tools/test-baseline.expected across all 15 rows. 59,886 checks / 13 suites
for the default leg, plus the 2 hub suites, 0 failures.
* every one of the 30 slice functions is still defined exactly once across
the twelve slice TUs (preprocessor census: 13 occurrences each -- twelve
declarations from the shared header plus one definition).
* the front TU still instantiates nothing: 205,400 kB, 1.8 s.
* the heaviest slice still compiles: 2,037,652 kB, matching the 2,039,660 kB
build-cuda reported for it before the rebase (0.1%).
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@balbasty