Skip to content

fix: make libfastfields.so link depend on cpu/cuda sub-libraries - #64

Merged
balbasty merged 1 commit into
mainfrom
fix-56-makefile-race
Aug 5, 2026
Merged

fix: make libfastfields.so link depend on cpu/cuda sub-libraries#64
balbasty merged 1 commit into
mainfrom
fix-56-makefile-race

Conversation

@balbasty

Copy link
Copy Markdown
Collaborator

Summary

  • libfastfields.so's link rule listed only $(OBJECTS) as prerequisites. Its dependency on build/lib/libfastfields-cpu.so (and, when USE_CUDA=1, build/lib/libfastfields-cuda.so) was expressed only by listing libcpu/$(CUDA_DEP) as siblings of the link target under the recipe-less lib: goal — not as a real prerequisite of the link rule itself.
  • Under make -j, prerequisites of lib: have no ordering guarantee and can run concurrently, so the link step could start (and finish) before make -C cpu install had produced build/lib/libfastfields-cpu.so, failing with ld: cannot find -lfastfields-cpu.
  • Fix: add $(BUILDDIR)/lib/libfastfields-cpu.$(SOSUF) and $(CUDA_DEP) as real prerequisites of the $(BUILDDIR)/libfastfields.$(SOSUF) rule, so make -j cannot schedule the link before they exist. The recipe now references $(OBJECTS) explicitly instead of $^, since $^ would otherwise also pull the newly-added .so prerequisites into the link command's positional arguments (redundant with the existing -lfastfields-cpu/$(CUDA_LDFLAGS) flags).
  • Checked cuda's submake for the analogous issue per the issue's suggestion: same pattern ($(CUDA_DEP) was only a sibling, not a real prerequisite), fixed in the same rule. Both cpu-lib's and cuda-lib's own internal install/libcpu/libcuda rules already declare correct real (non-sibling) prerequisites, so no further changes were needed there.

Closes#56

Verification

Ran from a clean build/ each time (rm -rf build), CPU path only (USE_CUDA defaults to 0, matching how CI builds this repo today).

Before the fix, a clean make -C . all CXX=clang++ -j4 reproduces the exact failure from the issue:

clang++ -std=c++11 -O3 -ferror-limit=1 -ftemplate-backtrace-limit=0 -shared -fPIC -Wl,-soname,libfastfields.so -Wl,-rpath,'$ORIGIN'/../lib \
-L./build/lib -lfastfields-cpu \
-o build/libfastfields.so build/distance.o build/posdef.o build/resize.o build/restrict.o build/splinc.o build/pushpull.o build/reg_field.o build/reg_flow.o
/usr/bin/ld: cannot find -lfastfields-cpu: No such file or directory
clang++: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [Makefile:187: build/libfastfields.so] Error 1

(the top-level link ran before make -C cpu install had finished copying libfastfields-cpu.so into build/lib/).

After the fix:

  • make -C . all CXX=clang++ -j4 from a clean build/2 consecutive clean runs, both succeed, both produce build/libfastfields.so and build/lib/libfastfields-cpu.so, no cannot find -lfastfields-cpu / no Error markers in either log.
  • Plain non-parallel make -C . all CXX=clang++ from a clean build/ — also succeeds, same artifacts produced, no errors.
$ tail -3 after-j4-run1.log
-L./build/lib -lfastfields-cpu \
-o build/libfastfields.so ./build/distance.o ./build/posdef.o ./build/resize.o ./build/restrict.o ./build/splinc.o ./build/pushpull.o ./build/reg_field.o ./build/reg_flow.o
make: Leaving directory '/home/user/fastfields-lib'
$ tail -3 after-j4-run2.log
-L./build/lib -lfastfields-cpu \
-o build/libfastfields.so ./build/distance.o ./build/posdef.o ./build/resize.o ./build/restrict.o ./build/splinc.o ./build/pushpull.o ./build/reg_field.o ./build/reg_flow.o
make: Leaving directory '/home/user/fastfields-lib'
$ ls -la build/libfastfields.so build/lib/libfastfields-cpu.so # after both -j4 runs and the non-parallel run
-rwxr-xr-x 1 root root 55448 build/libfastfields.so
-rwxr-xr-x 1 root root 163008960 build/lib/libfastfields-cpu.so
  • make -n dry-run sanity check that the link target now lists the cpu .so as a prerequisite
  • Clean make -C . all CXX=clang++ -j4 — before: reproducibly fails; after: 2/2 clean runs pass
  • Clean non-parallel make -C . all CXX=clang++ — passes
  • No standalone tests at this level (per CLAUDE.md); correctness remains gated by fastfields-cpu-lib's test suite, unaffected by this change

Generated by Claude Code

Under `make -j`, the top-level `libfastfields.so` link rule only listed
$(OBJECTS) as prerequisites -- its dependency on the cpu (and cuda, when
USE_CUDA=1) sub-library was expressed only as a sibling under the
recipe-less `lib:` target, not as a real prerequisite of the link rule
itself. Make is free to run independent prerequisites of `lib:` in any
order/concurrently under -j, so the link step could run before
`make -C cpu install` finished producing build/lib/libfastfields-cpu.so,
failing with `ld: cannot find -lfastfields-cpu`.
Add the cpu .so (and $(CUDA_DEP)) as real prerequisites of the
$(BUILDDIR)/libfastfields.$(SOSUF) rule so make -j can't schedule the
link before they exist. The recipe now lists $(OBJECTS) explicitly
instead of $^, since $^ would otherwise pull the .so prerequisites into
the link command's positional arguments.
Closes#56
@github-actions

Copy link
Copy Markdown

Submodule staleness (non-blocking -- fastfields-lib#15)

submodulepinned committracksstatusbehind by
cpubf57a02fastfields/fastfields-cpu-lib@mainup to date0
cudab8e5e9cfastfields/fastfields-cuda-lib@mainup to date0

@balbasty
balbasty merged commit 06e664a into mainAug 5, 2026
4 checks passed
@balbasty
balbasty deleted the fix-56-makefile-race branch August 5, 2026 12:42
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Makefile: make -j races and fails to link libfastfields.so (missing -lfastfields-cpu)

2 participants

@balbasty@claude