Add EXQLITE_EXTRA_SRC: statically compile SQLite extensions into the NIF

[AlanMcCann]

Why

Run-time loadable extensions don't cover every target: iOS forbids dynamic code loading outright, and some hardened server deployments disable load_extension. Shipping one pre-compiled .so per architecture is also a real maintenance burden. For those cases the natural answer is compiling the extension into the NIF next to the bundled sqlite3.c — SQLite supports this officially via SQLITE_EXTRA_INIT — but the Makefile's source list is currently fixed, so today this requires forking exqlite.

We hit this migrating a production Elixir app (SQLite + sqlite-vec for vector search, including an iOS target) and carried exactly this patch as a vendored fork; upstreaming the generic hook so nobody else has to.

What

EXQLITE_EXTRA_SRC — a space-separated list of C files compiled and linked into the NIF. Combined with the existingEXQLITE_SYSTEM_CFLAGS, an extension registers on every connection with no load_extension anywhere:

config:exqlite,force_build: true,make_env: %{"EXQLITE_EXTRA_SRC"=>"/path/to/sqlite-vec.c /path/to/vec_init_shim.c","EXQLITE_SYSTEM_CFLAGS"=>"-DSQLITE_CORE=1 -DSQLITE_EXTRA_INIT=exqlite_vec_init"}

with an 8-line shim calling sqlite3_auto_extension(sqlite3_vec_init). README section and CHANGELOG entry included.

Design notes

• No-op when unset — OBJ and the existing pattern rules are byte-identical without the variable.
• Ignored under EXQLITE_USE_SYSTEM (linking is the system library's business there).
• Extra objects are namespaced extra_<basename>.o with sources resolved via vpath, so out-of-tree paths (deps, vendored dirs) work. Basename collisions are documented.
• POSIX make only; Makefile.win untouched (happy to add if you want it).

Verified by execution

Against this branch, consumed as a plain path dep through make_env only:

UPSTREAM EXTRA_SRC PROOF: PASS - sqlite=3.53.4 vec=v0.1.9 cos_orthogonal=1.0 cos_45deg=0.2928932309150696
NO-HOOK REGRESSION CHECK: PASS - sqlite=3.53.4

(orthogonal vectors → cosine distance 1.0; 45° → 1 − 1/√2 — known-answer checks, not just "it loaded"). The same static-link approach (same flags + shim) was additionally verified on iOS: device-SDK static archive builds clean and the query executes correctly in the simulator, which is the platform where this hook is the only option.

[warmwaffles]

At some point I am going to package https://sqlite.org/vec1/doc/trunk/doc/vec1.md with this project once it becomes "offical" and stable.

[warmwaffles]

This doesn't seem too insane to me. Tools for people who know the risks they are taking.

I was contemplating bundling sqlite-vec with this project, but I don't know if I can get it to cross compile for all supported precompiled architectures.

[warmwaffles]

Question:

Run-time loadable extensions don't cover every target: iOS forbids
dynamic code loading outright, and some hardened server deployments
disable load_extension.

How is this possible? This is a precompiled binary or compiled straight from source, how is the "hardened" server deployments disabling load extension @AlanMcCann?

Awaiting response from author.