No CUDA launch error checking anywhere: 39 launches, zero cudaGetLastError — failures are silently discarded

[balbasty]

Found by: claude-jitfields-to-fastfields, during the core/-boundary and naming design work.

There are 39 kernel launches across include/fastfields/impl/cuda/. cudaGetLastError and cudaPeekAtLastError appear zero times in the entire repository:

$ grep -rc 'cudaGetLastError' include/ src/ → 0 files
$ grep -rc 'cudaPeekAtLastError' include/ src/ → 0 files
$ grep -rc '<<<' include/fastfields/impl/cuda/ → 39 launches

A CUDA kernel launch is asynchronous and does not throw. It reports failure by setting the error state, which the next cudaGetLastError / cudaPeekAtLastError (or the next synchronising call's return value) would surface. Nothing here inspects either. So a launch that never runs is indistinguishable from one that ran correctly: the function returns normally and the caller reads whatever was already in the output buffer — stale data, zeros, or uninitialised memory.

This is not hypothetical, and one trigger is already present

include/fastfields/impl/cuda/utils.h sets

static constexpr int CUDA_NUM_THREADS = 1024;

and every launch site uses it. 1024 is the architectural maximum, not a safe default. A kernel whose register usage is high enough has cudaFuncGetAttributes(...).maxThreadsPerBlock < 1024, and launching it at 1024 fails immediately with cudaErrorLaunchOutOfResources ("too many resources requested for launch").

These kernels are heavily templated over dtype × ndim × spline order × boundary condition, and some are large. Whether any current instantiation exceeds the register budget is unknown — but if one does, today it fails silently.

Why nothing has caught it

Same structural blind spot as #86 (unzeroed accumulation buffer → wrong sign), #88 (stream ordering), and #100 (2-byte CAS alignment): there is no GPU in CI, so the accepted CUDA bar is compile+link. A launch configuration error is invisible to a compiler — it is a runtime property of register allocation. The CPU suite cannot help, because launch configuration has no CPU counterpart.

Suggested fix, in order of value

1. Check the error after every launch. A small macro — launch, then cudaGetLastError(), and throw on failure — applied at all 39 sites. This alone converts silent wrong answers into loud failures and is worth doing regardless of everything below.
2. Stop hard-coding 1024. Prefer cudaOccupancyMaxPotentialBlockSize per kernel, which picks a block size that both fits the register budget and maximises occupancy. Failing that, clamp to cudaFuncGetAttributes(...).maxThreadsPerBlock.
3. Note that GET_BLOCKS(..., max_threads_per_block = CUDA_NUM_THREADS) already takes the block size as a runtime parameter that is simply always defaulted — so the plumbing for (2) partly exists.

Related

• Half/bf16 CUDA atomics will need CAS fallbacks — and a 4-byte alignment precondition on user DLPack pointers #100 — half/bf16 atomics; same file, same "no GPU in CI" blind spot.
• CUDA launchers rely on cudaFree's implicit device-wide sync for stream ordering #88, cuda(mesh): add the sdt_naive host launcher; zero the vertex normals #86 — the other two silent-wrong-answer classes in the CUDA launchers.
• Complete/fix the CUDA mesh signed-distance host launcher (sdt) #5 — CUDA mesh sdt; real-hardware validation is its acceptance bar too.

Worth stating plainly: of the CUDA defects found in this repository so far, none were catchable by the validation that exists. This one is the cheapest to close, because error checking is a mechanical change that does not need hardware to implement — only to exercise.