Feature request: Compile time argument for models folder in cli

[Timh1970]

I am building a package and installing in a common location to make it easier to use as a global library and am currently patching cli.cpp thus:

src/cli/cli.cpp

--- a/src/cli/cli.cpp	2026-02-06 12:13:30.426359449 +0000
+++ b/src/cli/cli.cpp	2026-02-06 13:56:47.601175698 +0000
@@ -354,8 +354,14 @@
 
 void run_sam(cli_args const& args) {
 backend_device backend = backend_init(args);
+
 auto [file, weights] = load_model_weights(
+#ifdef MODEL_PATH
+ args, backend, MODEL_PATH "/MobileSAM-F16.gguf", 0, backend.preferred_layout());
+#else
 args, backend, "models/MobileSAM-F16.gguf", 0, backend.preferred_layout());
+#endif
+
 sam_params params{};
 
 require_inputs(args.inputs, 1, "<image>");
@@ -408,8 +414,14 @@
 
 void run_birefnet(cli_args const& args) {
 backend_device backend = backend_init(args);
+
+#ifdef MODEL_PATH
 auto [file, weights] = load_model_weights(
- args, backend, "models/BiRefNet-F16.gguf", 0, backend.preferred_layout());
+ args, backend, MODEL_PATH "/BiRefNet-lite-F16.gguf", 0, backend.preferred_layout());
+#else
+ auto [file, weights] = load_model_weights(
+ args, backend, "models/BiRefNet-lite-F16.gguf", 0, backend.preferred_layout());
+#endif
 
 require_inputs(args.inputs, 1, "<image>");
 image_data image = image_load(args.inputs[0]);
@@ -452,8 +464,14 @@
 
 void run_depth_anything(cli_args const& args) {
 backend_device backend = backend_init(args);
+
+#ifdef MODEL_PATH
+ auto [file, weights] = load_model_weights(
+ args, backend, MODEL_PATH "/DepthAnythingV2-Small-F32.gguf", 0, backend.preferred_layout());
+#else
 auto [file, weights] = load_model_weights(
 args, backend, "models/DepthAnythingV2-Small-F32.gguf", 0, backend.preferred_layout());
+#endif
 
 require_inputs(args.inputs, 1, "<image>");
 image_data image = image_load(args.inputs[0]);
@@ -488,8 +506,15 @@
 
 void run_migan(cli_args const& args) {
 backend_device backend = backend_init(args);
+
+#ifdef MODEL_PATH
+ auto [file, weights] = load_model_weights(
+ args, backend, MODEL_PATH "/MIGAN-512-places2-F16.gguf", backend.preferred_layout());
+#else
 auto [file, weights] = load_model_weights(
 args, backend, "models/MIGAN-512-places2-F16.gguf", backend.preferred_layout());
+#endif
+
 migan_params params = migan_detect_params(file);
 params.invert_mask = true; // -> inpaint opaque areas
 
@@ -526,8 +551,15 @@
 
 void run_esrgan(cli_args const& args) {
 backend_device backend = backend_init(args);
+
+#ifdef MODEL_PATH
+ auto [file, weights] = load_model_weights(
+ args, backend, MODEL_PATH "/RealESRGAN-x4.gguf", 0, backend.preferred_layout());
+#else
 auto [file, weights] = load_model_weights(
 args, backend, "models/RealESRGAN-x4.gguf", 0, backend.preferred_layout());
+#endif
+
 esrgan_params params = esrgan_detect_params(file);
 printf("- scale: %dx\n", params.scale);
 printf("- block count: %d\n", params.n_blocks);

src/cli/CMakeLists.txt

--- a/src/cli/CMakeLists.txt	2026-02-05 23:47:15.000000000 +0000
+++ b/src/cli/CMakeLists.txt	2026-02-06 13:16:26.118842969 +0000
@@ -4,4 +4,5 @@
 target_compile_definitions(vision-cli PRIVATE ${VISP_ASSERT} ${VISP_DEFINITIONS})
 target_compile_options(vision-cli PRIVATE ${VISP_WARNINGS} ${VISP_COMP_OPTIONS})
 target_link_options(vision-cli PRIVATE ${VISP_LINK_OPTIONS})
-target_link_libraries(vision-cli PRIVATE ggml visioncpp ${VISP_FMT_LINK})
\ No newline at end of file
+target_link_libraries(vision-cli PRIVATE ggml visioncpp ${VISP_FMT_LINK})
+target_compile_definitions(vision-cli PRIVATE "MODEL_PATH=\"${MODEL_PATH}\"")

Note Also changed "BiRefNet-F16.gguf" to "BiRefNet-lite-F16.gguf"

If it would be possible to integrate this, or something similar it would be much appreciated.

[Acly]

Would a search path make more sense? Something like

• ./models
• $XDG_DATA_HOME/visioncpp/models
• $HOME/.local/share/visioncpp/models
• ${SYSTEM_MODEL_PATH} (which might be /usr/local/share/visioncpp/models but a cmake variable to override is fine)

By the way how do you intend to handle the ggml libraries when packaging? They currently aren't compatible with upstream ggml (which may be provided as separate package). I try and upstream most changes, but some of it is very vision-specific, and I'm not confident it will be included in near future.

[Timh1970]

Running cli globally would fail to find the default models if they were in a $HOME folder for users logged in with accounts other than that of the installer so I try to keep dependencies of global applications in global locations; that is why I was minded to put the default models in /usr/share/vision.cpp/models when packaging.

I have to admit to paying little attention to where the upstream ggml was coming from so that is a whole new issue, but not insurmountable.

Option 1 (ideal)

Static linking

Option 2

I am thinking install the shared objects in /usr/lib/vision.cpp then when building against vision.cpp it will be a case of

• -L/usr/share/vision.cpp
  and
• -Wl,-rpath=/usr/lib/vision.cpp

I can then bundle a pkg-config file with something like

• Libs: -L${libdir} -Wl,-rpath=${libdir} -lggml

and then building against vision.cpp should be just a case of using

• pkg-config --libs vision.cpp

There is always the possibility of conflicts with the upstream library when linking but paths in PKG_CONFIG_PATH are searched first, so theoretically this version takes precedence.

It was kinda lucky I didn't have libggml alreayd on my test boxes or I would have broken something.

[Timh1970]

I have tested building libvisioncpp while statically linking llama and everything works as expected in principle; the downside is the resultant libvisioncpp.so is 50mb but I still think it is a better option than option 2

Tests show no effect on processing speed so I think static linking is the best solution.

While I appreciate that I will need to track this and manually update I have changed

root CMakeLists.txt:

--- CMakeLists.txt	2026-02-02 17:18:42.662809881 +0000
+++ CMakeLists.patch.txt	2026-02-09 22:53:04.064581097 +0000
@@ -85,9 +85,12 @@
 endif()
 
 # Dependencies
+set(CMAKE_POSITION_INDEPENDENT_CODE ON)
+
 
 add_subdirectory(depend/stb)
 
+
 if(VISP_FMT_LIB)
 add_subdirectory(depend/fmt)
 set(VISP_FMT_LINK fmt::fmt)
@@ -109,7 +112,6 @@
 endforeach()
 endif()
 endif()
-add_subdirectory(depend/llama/ggml)
 
 # Vision.cpp libraries and executables
 
@@ -124,6 +126,17 @@
 add_subdirectory(models)
 endif()
 
+
+# Save and force static build for llama
+set(BUILD_SHARED_LIBS_SAVED ${BUILD_SHARED_LIBS})
+set(BUILD_SHARED_LIBS OFF)
+set(LLAMA_STATIC ON CACHE BOOL "Build llama as static" FORCE)
+set(GGML_STATIC ON CACHE BOOL "Build ggml as static" FORCE)
+set(GGML_BACKEND_DL OFF CACHE BOOL "Disable dynamic backend" FORCE)
+add_subdirectory(depend/llama)
+# Restore shared build for vision.cpp
+set(BUILD_SHARED_LIBS ${BUILD_SHARED_LIBS_SAVED})
+
 # Installation
 
 set(VISP_INCLUDE_INSTALL_DIR ${CMAKE_INSTALL_INCLUDEDIR} CACHE PATH "Location of header files")

and visp/CMakeLists.txt

--- vcpp-CMakeLists.txt	2026-02-02 17:18:42.666143249 +0000
+++ vcpp-CMakeLists.patch.txt	2026-02-09 22:26:29.711088653 +0000
@@ -25,8 +25,7 @@
 )
 target_link_options(visioncpp PRIVATE ${VISP_LINK_OPTIONS})
 target_link_libraries(visioncpp
- PUBLIC ggml
- PRIVATE stb ${VISP_FMT_LINK}
+ PRIVATE llama stb ${VISP_FMT_LINK}
 )
 # set_target_properties(visioncpp PROPERTIES
 # VERSION ${PROJECT_VERSION}

I still need to make a package, install it and test but don't see any major challenges. I may maintain a shared version and a static version but set set conficts(llama) in the shared version.

[Acly]

Static build for ggml/ggml-base makes sense, I have to try it still, but wouldn't mind making it the default.

For the backends (GGML_BACKEND_DL) I think it would still make sense to keep dynamic loading. These are not linked at compile time, so they don't need to be on the linker path. Rather ggml tries to dynamically load them at runtime, so it just needs to know where to find them and they can be anywhere.

The advantage to keeping the backends dynamic is that people without Vulkan installed (maybe no GPU in the system) can still load and use the library, which would not be possible if the Vulkan backend is statically linked into libvisioncpp. Same for CPU if the minimum instruction set (currently AVX for x86) is not supported.

Running cli globally would fail to find the default models if they were in a $HOME folder for users logged in with accounts other than that of the installer so I try to keep dependencies of global applications in global locations; that is why I was minded to put the default models in /usr/share/vision.cpp/models when packaging.

Definitely install models that ship with the package into the system location. I was thinking of users downloading alternative models eg. there are more precise high-res versions of BiRefNet, but they're more heavy-weight. Having a search path with fallbacks would allow these to co-exist.

[Timh1970]

It will probably be easier to maintain packages with without Vulkan separately; I am still very much experimenting and seeing what works for now.

My rig is not well suited to running on cpu (it's ancient). It does work, but in tests it took on a large file (which I use to try and punish it) it took 9 minuted to process and the cooling fans went for it. Same file with Vulkan took 20 seconds on an old AMD RX 580; on a more sensible file I am getting around 390ms. Interestingly on a nvidia RTX4060 the large file makes it fall over; but that is an nvidia issue as it doesn't manage memory as well (sensible file works okay and runs inference in around 70ms); both tested with BiRefNet-dynamic-F16

Definitely install models that ship with the package into the system location. I was thinking of users downloading alternative models eg. there are more precise high-res versions of BiRefNet, but they're more heavy-weight. Having a search path with fallbacks would allow these to co-exist.

Yes, I use BiRefNet dynamic ordinarily but the cli is a great little convenience for quick testing and to give a rough idea of what each model does.

I will have a go at compiling with GGML_BACKEND_DL dynamic and then do some tests to see what happens.

[Timh1970]

Is there any chance you can do a 0.3.1 tag so I can play with my patches? I did a dumb thing and both packaged and made patches without a tag.

[Acly]

Not really sure why it's important, but okay: https://github.com/Acly/vision.cpp/tree/v0.3.1

don't think it'll confuse anybody at this point

[Timh1970]

Normally I would just download a release tarball but couldn't do that because sub modules aren't in the tarball so I have to clone it instead. If a line of code changes in any of the files that I have patched then I need to redo the patches. I can't use the normal source() directive in PKGBUILD because --recursive is not supported (I always build on Arch first because it has the one of the easiest way to build in a chroot environments, then move on to the more challenging distros later). Instead I need to git clone --recursive --branch v0.3.1 [https://](https://github.com/Acly/vision.cpp Building against a branch will mean that the patches will always work regardless of how many changes are made.

Once everything tests out okay you can decide what (if anything) you wish to incorporate and I can merge the changes with any remaining patches on the next tag.

I messed up because I should have included the --branch v0.3.0 but a few changes have been made since then so when I remembered it was too late (I either re did the patches or got a new tag).

I have amended the patches for the CMakeLists as I messed up a bit and it only showed up when I built in a chroot.

src/visp/CMakeLists.txt

--- a/vision.cpp/src/visp/CMakeLists.txt	2026-02-10 22:21:27.011962744 +0000
+++ b/vision.cpp/src/visp/CMakeLists.txt	2026-02-10 23:32:54.765315012 +0000
@@ -25,8 +25,8 @@
 )
 target_link_options(visioncpp PRIVATE ${VISP_LINK_OPTIONS})
 target_link_libraries(visioncpp
- PUBLIC ggml
- PRIVATE stb ${VISP_FMT_LINK}
+ PUBLIC ggml::ggml
+ PRIVATE llama stb ${VISP_FMT_LINK}
 )
 # set_target_properties(visioncpp PROPERTIES
 # VERSION ${PROJECT_VERSION}

root/CMakeLists.txt

--- a/vision.cpp/CMakeLists.txt	2026-02-02 17:18:42.662809881 +0000
+++ b/vision.cpp/CmakeLists.txt	2026-02-10 22:57:27.524971901 +0000
@@ -85,9 +85,12 @@
 endif()
 
 # Dependencies
+set(CMAKE_POSITION_INDEPENDENT_CODE ON)
+
 
 add_subdirectory(depend/stb)
 
+
 if(VISP_FMT_LIB)
 add_subdirectory(depend/fmt)
 set(VISP_FMT_LINK fmt::fmt)
@@ -109,15 +112,22 @@
 endforeach()
 endif()
 endif()
-add_subdirectory(depend/llama/ggml)
 
-# Vision.cpp libraries and executables
+# Save and force static build for llama
+set(BUILD_SHARED_LIBS_SAVED ${BUILD_SHARED_LIBS})
+set(BUILD_SHARED_LIBS OFF)
+set(LLAMA_STATIC ON CACHE BOOL "Build llama as static" FORCE)
+set(GGML_STATIC ON CACHE BOOL "Build ggml as static" FORCE)
+set(GGML_BACKEND_DL OFF CACHE BOOL "Disable dynamic backend" FORCE)
+add_subdirectory(depend/llama)
+# Restore shared build for vision.cpp
+set(BUILD_SHARED_LIBS ${BUILD_SHARED_LIBS_SAVED})
 
+# Vision.cpp libraries and executables
 add_subdirectory(src/visp)
 add_subdirectory(src/cli)
 
 # Tests
-
 if(VISP_TESTS)
 enable_testing()
 add_subdirectory(tests)
@@ -125,7 +135,6 @@
 endif()
 
 # Installation
-
 set(VISP_INCLUDE_INSTALL_DIR ${CMAKE_INSTALL_INCLUDEDIR} CACHE PATH "Location of header files")
 set(VISP_LIB_INSTALL_DIR ${CMAKE_INSTALL_LIBDIR} CACHE PATH "Location of library files")
 set(VISP_BIN_INSTALL_DIR ${CMAKE_INSTALL_BINDIR} CACHE PATH "Location of binary files")

The PKGBUILD lets CMake do the installing and then strips out the archives and unnecessary headers afterwards for the static build to avoid conflicts.

pkgname=vision.cpp-vulkan-static-git
pkgver=0.3.1
pkgrel=13
pkgdesc="Computer Vision ML inference in C++"
arch=(x86_64)
license=('MIT')
depends=('shaderc' 'vulkan-headers' 'vulkan-icd-loader' 'vulkan-validation-layers' 'spirv-headers' 'spirv-tools' 'vulkan-tools' 'vulkan-utility-libraries' 'vulkan-validation-layers' 'vulkan-extra-tools' 'vulkan-extra-layers')
makedepends=('git' 'cmake') 
options=('staticlibs')
source=(
 "vision.cpp-cli-CmakeLists.txt.patch"
 "vision.cpp-cli.cpp.patch"
 "vision.cpp-root-CMakeLists.txt.patch"
 "vision.cpp-visp-CMakeLists.txt.patch"
)
noextract=()
sha256sums=('c8449fba4f4211e22ba1c75a02ba732f33ae5eb6a57c6c831c8d58906f1920d9'
 'ba70b02cda1b8467d27d8fdba4e4910c29c336a1b2f23ea8cebc9af74e02f7a3'
 'db718372ef83fc873bbe4818c1d05712822f79a9df4e37feb4db907267a9d675'
 'c181c07701e6257e68cc284a15bed146227e28bed1243f929bc8cca83549093f')

prepare() {
 cd "$srcdir"
 git clone --recursive --branch v0.3.1 https://github.com/Acly/vision.cpp
}

build() {

 patch -p1 < "$srcdir/vision.cpp-cli-CmakeLists.txt.patch"
 patch -p1 < "$srcdir/vision.cpp-cli.cpp.patch"
 patch -p1 < "$srcdir/vision.cpp-root-CMakeLists.txt.patch"
 patch -p1 < "$srcdir/vision.cpp-visp-CMakeLists.txt.patch"


 cd "$srcdir/vision.cpp"

 cmake -B build \
 -DCMAKE_INSTALL_PREFIX=/usr \
 -DCMAKE_BUILD_TYPE=Release \
 -DVISP_VULKAN=ON \
 -DMODEL_PATH="/usr/share/vision.cpp/models"

 cmake --build build
}

package() {
 cd "$srcdir/vision.cpp"
 DESTDIR="$pkgdir" cmake --install build
 models=("${srcdir}/vision.cpp/models/"*.gguf)
 install -Dm644 -t "$pkgdir/usr/share/vision.cpp/models" "${models[@]}"
 rm "${pkgdir}/usr/include/"*.h
 rm -R "${pkgdir}/usr/lib/cmake/llama"
 rm -R "${pkgdir}/usr/lib/cmake/ggml"
 rm "${pkgdir}/usr/lib/"*.a
}

[Acly]

made some changes: #20

not sure I got it all, but this should allow you to do

 cmake -B build \
 -DCMAKE_INSTALL_PREFIX=/usr \
 -DCMAKE_BUILD_TYPE=Release \
 -DVISP_VULKAN=ON \
 -DVISP_INSTALL_MODELS=ON \
 -DVISP_STATIC_GGML=ON \
 -DVISP_TESTS=OFF

and it will install models to /usr/share/visioncpp (as part of cmake --install) and cli will find them there

I've only done a bit of testing on windows so far

NB: this will not install any ESRGAN model, the one downloaded for tests is not really suitable. but now we could define a more generalist upscaler instead that comes with VISP_INSTALL_MODELS but not VISP_TESTS.

[Timh1970]

Ooooh so close it isn't even worth writing a patch for. Building on the mode-install-path branch gives me

/usr/bin/ld: ../../lib/libggml-vulkan.a(ggml-vulkan.cpp.o): warning: relocation against `log_f32_rte_data' in read-only section `.text'
/usr/bin/ld: ../../lib/libggml.a(ggml-backend-reg.cpp.o): relocation R_X86_64_PC32 against symbol `_ZN21ggml_backend_registryD1Ev' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: bad value
collect2: error: ld returned 1 exit status
make[2]: *** [src/visp/CMakeFiles/visioncpp.dir/build.make:301: lib/libvisioncpp.so] Error 1
make[1]: *** [CMakeFiles/Makefile2:470: src/visp/CMakeFiles/visioncpp.dir/all] Error 2
make: *** [Makefile:156: all] Error 2

Which is kind of self explanatory.

Line 109 insert 'set(CMAKE_POSITION_INDEPENDENT_CODE ON)` and it compiles.

I will have to do a bit of deeper testing in a couple of days but on the face of it that looks great.

PS. I am afraid that the best I can manage is to fumble around when it comes to CMake but that looks like a much cleaner solution more elegant solution than mine.

I don't have time to do a full test right now but at a glance I think you might need to put a leading / in VISP_MODEL_INSTALL_DIR as I have a feeling without it it will resolve to /usrshare.

[Acly]

Ah I was thinking "I will put PIC in a separate commit" and then forgot.

I don't have time to do a full test right now but at a glance I think you might need to put a leading / in VISP_MODEL_INSTALL_DIR as I have a feeling without it it will resolve to /usrshare.

I think for installing it should be fine, but there's an issue when passing it to CLI since VISP_MODEL_INSTALL_DIR is typically a path relative to CMAKE_INSTALL_PREFIX (ie. just "share/visioncpp"). The install command concatenates them and makes "/usr/share/visioncpp", but need to do the same manually for the CLI definition.

Should be fixed now on the branch.

[Timh1970]

So far all seems good (good enough to continue at least). I haven't done exhaustive tests as my workload just got silly, but in principle all is good.

I am thinking now that this works well enough for my use case, but it makes sense now we have gone this far to do it properly if you like.

Next thing would be to make it ABI friendly (and I have no idea how to do that in Cmake, but it can't be that hard) Were it an ordinary makefile then you would:

MAJOR := $(major)
MINOR := $(minor)
PATCH := $(patch)
VERSION := $(MAJOR).$(MINOR).$(PATCH)
APPLICATION=xxxxxxxxxxxxx.so
FULLNAME = $(APPLICATION).$(VERSION)
SONAME = $(APPLICATION).$(MAJOR)

SOURCES = $(wildcard *.cpp)
HEADERS = $(wildcard *.h)
OBJECTS = $(SOURCES:.cpp=.o)

CXX = g++
USER_CXXFLAGS ?=
CXXFLAGS += ....


$(FULLNAME): $(OBJECTS)
	$(CXX) -shared -Wl,-soname,$(SONAME) -o $@ $^
	ln -sfT $(FULLNAME) $(SONAME)
	ln -sfT $(SONAME) $(APPLICATION)

%.o: %.cpp $(HEADERS)
	$(CXX) $(CXXFLAGS) -c $< -o $@

.PHONY: install
install:

	install -Dm755 -t $(DESTDIR)/usr/lib/ $(FULLNAME)
	ln -sfT $(FULLNAME) $(DESTDIR)/usr/lib/$(SONAME)
	ln -sfT $(SONAME) $(DESTDIR)/usr/lib/$(APPLICATION)

...

This gives you a libvisioncpp.so.0.3.0 file and a couple of symlinks to libvisioncpp.so.0 and libvisioncpp.so; which is enough to package it for Debian based distros (ish).

[Timh1970]

It turns out making the ABI compliant library is easier than it looks

--- CMakeLists.txt.orig
+++ CMakeLists.txt
@@ -107,6 +107,14 @@
 # Vision.cpp libraries and executables
 
 add_subdirectory(src/visp)
+
+# Set ABI versioning for the shared library
+if(TARGET visioncpp)
+ set_target_properties(visioncpp PROPERTIES
+ VERSION ${PROJECT_VERSION}
+ SOVERSION ${PROJECT_VERSION_MAJOR}
+ )
+endif()
+
 add_subdirectory(src/cli)

While you are at it, if you could create a visioncpp.pc.in file in the project root

prefix=@CMAKE_INSTALL_PREFIX@
exec_prefix=${prefix}
libdir=${prefix}/@CMAKE_INSTALL_LIBDIR@
includedir=${prefix}/@CMAKE_INSTALL_INCLUDEDIR@

Name: visioncpp
Description: Computer Vision ML inference in C++
Version: @PROJECT_VERSION@
Libs: -L${libdir} -lvisioncpp
Cflags: -I${includedir}

and add this to the bottom of the CMakeLists.txt file so that it is installed in the right place, that would be fantastic.

# --- ABI Versioning and pkg-config Standardisation ---

if(TARGET visioncpp)
 # Sets libvisioncpp.so.0 and symlinks for ABI compliance
 set_target_properties(visioncpp PROPERTIES
 VERSION ${PROJECT_VERSION}
 SOVERSION ${PROJECT_VERSION_MAJOR}
 )

 # Generate and Install pkg-config file
 configure_file(
 "${CMAKE_CURRENT_SOURCE_DIR}/visioncpp.pc.in"
 "${CMAKE_CURRENT_BINARY_DIR}/visioncpp.pc"
 @ONLY
 )

 install(
 FILES "${CMAKE_CURRENT_BINARY_DIR}/visioncpp.pc"
 DESTINATION "${CMAKE_INSTALL_LIBDIR}/pkgconfig"
 )
endif()

[Acly]

I commented out SO version at some point, it was causing issues in some case, have to recheck. But it should be solvable somehow.

[Acly]

Yea seems to work fine.

Something else I noticed:

• The static ggml libraries currently get installed into the package too. It's not so easy to avoid, so probably you just want to filter them out.
• Likewise lib/cmake/ggml gets installed, but should probably not be packaged
• But if consumers of the library can't link ggml, can they use libvisioncpp?

The library was never meant to be a sealed layer around ggml, but rather something that can be used alongside. Parts of it can be used as building blocks for bigger ggml graphs. I'll try to do some testing with an example project that consumes the package with statically built ggml without linking it. Maybe with little effort the "high-level API" can work like that. And the rest is not so important for now (IMO people who are interested in that would include the lib as part of their source-tree anyway, not link a system package).