Unable to build the project on Raspberry Pi 4

[ping-ee]

As title, I can not build the project on Raspberry Pi 4 with Raspberry Pi OS (Debian 12 (bookworm)) and gcc 12.

command: python setup_env.py -md models/BitNet-b1.58-2B-4T -q i2_s

[ 1%] Building CXX object 3rdparty/llama.cpp/ggml/src/CMakeFiles/ggml.dir/__/__/__/__/src/ggml-bitnet-mad.cpp.o
In file included from /home/a91249061/BitNet/3rdparty/llama.cpp/ggml/src/./ggml-quants.h:4,
 from /home/a91249061/BitNet/src/ggml-bitnet-mad.cpp:5:
/home/a91249061/BitNet/3rdparty/llama.cpp/ggml/src/./ggml-common.h:154:16: warning: ISO C++ prohibits anonymous structs [-Wpedantic]
 154 | struct {
 | ^
/home/a91249061/BitNet/3rdparty/llama.cpp/ggml/src/./ggml-common.h:175:16: warning: ISO C++ prohibits anonymous structs [-Wpedantic]
 175 | struct {
 | ^
/home/a91249061/BitNet/3rdparty/llama.cpp/ggml/src/./ggml-common.h:196:16: warning: ISO C++ prohibits anonymous structs [-Wpedantic]
 196 | struct {
 | ^
/home/a91249061/BitNet/3rdparty/llama.cpp/ggml/src/./ggml-common.h:261:16: warning: ISO C++ prohibits anonymous structs [-Wpedantic]
 261 | struct {
 | ^
/home/a91249061/BitNet/3rdparty/llama.cpp/ggml/src/./ggml-common.h:294:16: warning: ISO C++ prohibits anonymous structs [-Wpedantic]
 294 | struct {
 | ^
/home/a91249061/BitNet/3rdparty/llama.cpp/ggml/src/./ggml-common.h:311:16: warning: ISO C++ prohibits anonymous structs [-Wpedantic]
 311 | struct {
 | ^
/home/a91249061/BitNet/src/ggml-bitnet-mad.cpp: In function ‘size_t quantize_i2_s(const float*, void*, int64_t, int64_t, const float*)’:
/home/a91249061/BitNet/src/ggml-bitnet-mad.cpp:46:100: warning: unused parameter ‘quant_weights’ [-Wunused-parameter]
 46 | size_t quantize_i2_s(const float * src, void * dst, int64_t nrow, int64_t n_per_row, const float * quant_weights) {
 | ~~~~~~~~~~~~~~^~~~~~~~~~~~~
/home/a91249061/BitNet/src/ggml-bitnet-mad.cpp: In function ‘void ggml_vec_dot_i2_i8_s(int, float*, size_t, const void*, size_t, const void*, size_t, int)’:
/home/a91249061/BitNet/src/ggml-bitnet-mad.cpp:95:28: warning: cast from type ‘const void*’ to type ‘uint8_t*’ {aka ‘unsigned char*’} casts away qualifiers [-Wcast-qual]
 95 | const uint8_t * x = (uint8_t *)vx;
 | ^~~~~~~~~~~~~
/home/a91249061/BitNet/src/ggml-bitnet-mad.cpp:96:28: warning: cast from type ‘const void*’ to type ‘int8_t*’ {aka ‘signed char*’} casts away qualifiers [-Wcast-qual]
 96 | const int8_t * y = (int8_t *)vy;
 | ^~~~~~~~~~~~
/home/a91249061/BitNet/src/ggml-bitnet-mad.cpp:94:52: warning: unused parameter ‘bs’ [-Wunused-parameter]
 94 | void ggml_vec_dot_i2_i8_s(int n, float * s, size_t bs, const void * vx, size_t bx, const void * vy, size_t by, int nrc) {
 | ~~~~~~~^~
/home/a91249061/BitNet/src/ggml-bitnet-mad.cpp:94:80: warning: unused parameter ‘bx’ [-Wunused-parameter]
 94 | void ggml_vec_dot_i2_i8_s(int n, float * s, size_t bs, const void * vx, size_t bx, const void * vy, size_t by, int nrc) {
 | ~~~~~~~^~
/home/a91249061/BitNet/src/ggml-bitnet-mad.cpp:94:108: warning: unused parameter ‘by’ [-Wunused-parameter]
 94 | void ggml_vec_dot_i2_i8_s(int n, float * s, size_t bs, const void * vx, size_t bx, const void * vy, size_t by, int nrc) {
 | ~~~~~~~^~
/home/a91249061/BitNet/src/ggml-bitnet-mad.cpp:94:116: warning: unused parameter ‘nrc’ [-Wunused-parameter]
 94 | void ggml_vec_dot_i2_i8_s(int n, float * s, size_t bs, const void * vx, size_t bx, const void * vy, size_t by, int nrc) {
 | ~~~~^~~
during GIMPLE pass: vect
/home/a91249061/BitNet/src/ggml-bitnet-mad.cpp: In function ‘size_t quantize_i2_s(const float*, void*, int64_t, int64_t, const float*)’:
/home/a91249061/BitNet/src/ggml-bitnet-mad.cpp:46:8: internal compiler error: in vect_transform_reduction, at tree-vect-loop.cc:7457
 46 | size_t quantize_i2_s(const float * src, void * dst, int64_t nrow, int64_t n_per_row, const float * quant_weights) {
 | ^~~~~~~~~~~~~
0x7f90e8773f __libc_start_call_main
	../sysdeps/nptl/libc_start_call_main.h:58
0x7f90e87817 __libc_start_main_impl
	../csu/libc-start.c:360
Please submit a full bug report, with preprocessed source (by using -freport-bug).
Please include the complete backtrace with any bug report.
See <file:///usr/share/doc/gcc-12/README.Bugs> for instructions.
gmake[2]: *** [3rdparty/llama.cpp/ggml/src/CMakeFiles/ggml.dir/build.make:132: 3rdparty/llama.cpp/ggml/src/CMakeFiles/ggml.dir/__/__/__/__/src/ggml-bitnet-mad.cpp.o] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:759: 3rdparty/llama.cpp/ggml/src/CMakeFiles/ggml.dir/all] Error 2
gmake: *** [Makefile:136: all] Error 2

[Benjamin-Wegener]

try
cmake .. -DCMAKE_CXX_FLAGS="-O2"
This avoids the crash by reducing optimization level and skipping problematic vectorization.

[ping-ee]

try cmake .. -DCMAKE_CXX_FLAGS="-O2" This avoids the crash by reducing optimization level and skipping problematic vectorization.

Neither the compile flag -DCMAKE_CXX_FLAGS=-O2 nor -DCMAKE_CXX_FLAGS_RELEASE="-O2 -DNDEBUG" works for me.

[ping-ee]

I followed steps 5 and 6 mentioned on this page instead of running setup_env.py, and it works.

[worldbiomusic]

I followed steps 5 and 6 mentioned on this page instead of running setup_env.py, and it works.

It works. Thank you💖💖

[Zodijackyl98]

I followed everything I found on the internet and tried to solve the issue with the help of multiple AI agents but still I end up getting the same error. I don't think there is a hardware wall that I can't overcome since there are people ran BitNet on Pi 4
like the one I have. So far I've tried these;

• Using different compilers; Cling 18, 20 and Cmake.
• Following official Github repo instructions and others I found online including the ones that run successfully on Pi 4 Here.
• Editing ~/BitNet/3rdparty/llama.cpp/ggml/src/ggml-quants.c
• Running cmake command with different flags such as

cmake .. -DCMAKE_BUILD_TYPE=Release \
 -DCMAKE_C_COMPILER=/usr/bin/clang-18 \
 -DCMAKE_CXX_COMPILER=/usr/bin/clang++-18 \
 -DGGML_NATIVE=OFF \
 -DBITNET_ARM_TL1=OFF \
 -DCMAKE_C_FLAGS="-march=armv8-a" \
 -DCMAKE_CXX_FLAGS="-march=armv8-a"

• Building the project with and without setup_env.py file.
  Here is the error;

warning: not compiled with GPU offload support, --gpu-layers option will be ignored
warning: see main README.md for information on enabling GPU BLAS support
build: 3962 (1f86f058) with Debian clang version 18.1.8 (++20240731024826+3b5b5c1ec4a3-1~exp1~20240731144843.145) for aarch64-unknown-linux-gnu
main: llama backend init
Error occurred while running command: Command '['build/bin/llama-cli', '-m', 'models/BitNet-b1.58-2B-4T/ggml-model-i2_s.gguf', '-n', '128', '-t', '2', '-p', 'You are a helpful assistant', '-ngl', '0', '-c', '2048', '--temp', '0.8', '-b', '1', '-cnv']' died with <Signals.SIGILL: 4>.

• AI focuses on this one line specifically -- Performing Test COMPILER_SUPPORTS_ARMV82_DOTPROD - Success since the CPU does not support dotprod and i8mm it behaves as if it supports.

Here is the ouput after compiling;

 -- The C compiler identification is Clang 18.1.8

-- The CXX compiler identification is Clang 18.1.8

-- Detecting C compiler ABI info

-- Detecting C compiler ABI info - done

-- Check for working C compiler: /usr/bin/clang-18 - skipped

-- Detecting C compile features

-- Detecting C compile features - done

-- Detecting CXX compiler ABI info

-- Detecting CXX compiler ABI info - done

-- Check for working CXX compiler: /usr/bin/clang++-18 - skipped

-- Detecting CXX compile features

-- Detecting CXX compile features - done

-- Performing Test CMAKE_HAVE_LIBC_PTHREAD

-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success

-- Found Threads: TRUE 

-- Found Git: /usr/bin/git (found version "2.39.5") 

-- Could NOT find OpenMP_C (missing: OpenMP_C_FLAGS OpenMP_C_LIB_NAMES) 

-- Could NOT find OpenMP_CXX (missing: OpenMP_CXX_FLAGS OpenMP_CXX_LIB_NAMES) 

-- Could NOT find OpenMP (missing: OpenMP_C_FOUND OpenMP_CXX_FOUND) 

CMake Warning at 3rdparty/llama.cpp/ggml/src/CMakeLists.txt:170 (message):

 OpenMP not found



-- Using llamafile

-- ccache found, compilation results will be cached. Disable with GGML_CCACHE=OFF.

-- CMAKE_SYSTEM_PROCESSOR: aarch64

-- ARM detected

-- Performing Test COMPILER_SUPPORTS_FP16_FORMAT_I3E

-- Performing Test COMPILER_SUPPORTS_FP16_FORMAT_I3E - Failed

-- Performing Test GGML_COMPILER_SUPPORT_DOTPROD

-- Performing Test GGML_COMPILER_SUPPORT_DOTPROD - Failed

-- Performing Test COMPILER_SUPPORTS_ARMV82_DOTPROD

-- Performing Test COMPILER_SUPPORTS_ARMV82_DOTPROD - Success

-- Configuring done

-- Generating done

-- Build files have been written to: /home/user/BitNet/build

[lepus2589]

I encountered the SIGILL signal (illegal instruction), too. I tried to build the latest main (8fd3412) on the latest Raspberry Pi OS from December 2025 based on Debian 13 (Trixie).

This gives you Clang/LLVM v19 and Python v3.13. Which unfortunately meant, that I had to build Python v3.12 from source, as the PyTorch version required by the BitNet project only works for Python<=3.12.

But after I solved that problem, I followed the steps from Bijan Bowen's (Bijan Bowen (@OminousIndustries)) BitNet b1.58 on Raspberry Pi 4B blog post. After setting up the Python virtual environment, I did:

1. Generating the header and config files with the command from setup_env.py#L180:

   $ python utils/codegen_tl1.py \
    --model bitnet_b1_58-3B \
    --BM 160,320,320 \
    --BK 64,128,64 \
    --bm 32,64,32

2. Building with CMake:

   $ export CC=clang-19
   $ export CXX=clang++-19
   $ cmake -B ./build -DCMAKE_BUILD_TYPE=Release -DBITNET_ARM_TL1=OFF .
   $ cmake --build ./build --verbose -j4

Running python run_inference.py ... caused the SIGILL error.

Because some SO answers pointed to a problem with the armv8-a +lse extension being assumed but unavailable on the Pi 4B, I even configured the complete arch and cpu info in CMake like this:

$ cmake -B ./build \ 
 -DCMAKE_BUILD_TYPE=Release \
 -DBITNET_ARM_TL1=OFF \
 -DCMAKE_C_FLAGS="-march=armv8-a+nolse -mcpu=cortex-a72 -mtune=cortex-a72 -mno-outline-atomics" \
 -DCMAKE_CXX_FLAGS="-march=armv8-a+nolse -mcpu=cortex-a72 -mtune=cortex-a72 -mno-outline-atomics" \
 .

But that didn't help.

Next, I tried to checkout a revision from the approximate time of Bijan Bowen's blog post. I decided for c17d1c5, as it already had the inference server and the model architecture name fix in it. I deleted the ./build directory, regenerated the header files and rebuilt with CMake and, voilà, it worked!

I decided to do a git bisect and it turns out, that the commit introducing this regression is c17d1c5. The last commit working on the RPi 4B is 404980e.

So, for anyone trying this in 2026, do this after cloning the repository:

$ git switch --detach 404980eecae38affa4871c3e419eae3f44536a95
$ git submodule update --recursive

Mert Yasin Şafak (@Zodijackyl98) Your AIs might be onto something there. The GGML_COMPILER_SUPPORT_DOTPROD try_compile() done by the 3rdparty/llama.cpp/ggml/src/CMakeLists.txt first tests, if the dotprod extension is supported by the target system. If you are on a system with an armv8.2-a+dotprod arch, this should succeed and simply add the respective pre-processor define. If it fails (as it does on RPi 4B), we follow up with the COMPILER_SUPPORTS_ARMV82_DOTPROD test to see, if the compiler supports the -march=armv8.2-a+dotprod flag (so basically, if you could cross-compile for such a system). If that succeeds, we explicitly set the -march=armv8.2-a+dotprod flag (and thus are now basically cross-compiling for such a CPU). If you are on a armv8-a (like the RPi 4B) and your compiler supports cross-compiling for armv8.2-a+dotprod (which clang-19 apparently does), you are now cross-compiling for such a system instead of compiling for your host system. Coincidentally, this try_compile() block was introduced in the "bad" commit.

Now, that I was explicitly looking for it, I actually saw the -march=armv8.2-a+dotprod given to the compiler. No wonder, it produces invalid instructions.

I think, the complete COMPILER_SUPPORTS_ARMV82_DOTPROD test needs to go. It only tests the cross-compile capability of the compiler, which doesn't help at all determining the capabilities of the target system:

diff --git a/ggml/src/CMakeLists.txt b/ggml/src/CMakeLists.txt
index 9006bdb7..197c4f4a 100644
--- a/ggml/src/CMakeLists.txt
+++ b/ggml/src/CMakeLists.txt
@@ -1155,12 +1155,12 @@ if (CMAKE_OSX_ARCHITECTURES STREQUAL "arm64" OR
 check_cxx_source_compiles("#include <arm_neon.h>\nint main(){ int8x16_t a{}, b{}; int32x4_t s = vdupq_n_s32(0); s = vdotq_s32(s,a,b); (void)s; return 0; }" GGML_COMPILER_SUPPORT_DOTPROD)
 if (GGML_COMPILER_SUPPORT_DOTPROD)
 add_compile_definitions(__ARM_FEATURE_DOTPROD)
- else()
- check_cxx_compiler_flag("-march=armv8.2-a+dotprod" COMPILER_SUPPORTS_ARMV82_DOTPROD)
- if (COMPILER_SUPPORTS_ARMV82_DOTPROD)
- list(APPEND ARCH_FLAGS -march=armv8.2-a+dotprod)
- add_compile_definitions(__ARM_FEATURE_DOTPROD)
- endif()
 endif()
 endif()
 elseif (CMAKE_OSX_ARCHITECTURES STREQUAL "x86_64" OR CMAKE_GENERATOR_PLATFORM_LWR MATCHES "^(x86_64|i686|amd64|x64|win32)$" OR

I tried this on main and it compiles and runs, but now it produces gibberish when running inference. This looks like what is described in #411.

Until these bugs are fixed, the "good" commit above is the only thing that works.