Hacking things on llama.cpp [ Interested in my works? Buy me a ☕ or Sponsor me 🩷 ]
Some of my works
- Support Gemma 4 12B "unified": ggml-org/llama.cpp#24077
- Add prompt token counting API to
llama-server: ggml-org/llama.cpp#23913 - Refactor
llama_modelinterface: ggml-org/llama.cpp#22004 - Add
/v1/audio/transcriptionsAPI tollama-server: ggml-org/llama.cpp#21863 - Support Gemma 4: ggml-org/llama.cpp#21309
- Support Mistral Small 4: ggml-org/llama.cpp#20649
- Support GLM-OCR: ggml-org/llama.cpp#19677
- Support GLM-5: ggml-org/llama.cpp#19460
- Optimize Qwen3-Next (~20% prefill speedup): ggml-org/llama.cpp#18683
- New jinja template engine: ggml-org/llama.cpp#18462
- Adding MiMo-V2-Flash support: ggml-org/llama.cpp#18328
- Adding GLM-4V support: ggml-org/llama.cpp#18042
- The brand new
llama-cli: ggml-org/llama.cpp#17824 - Serving multiple models in parallel via
llama-server: ggml-org/llama.cpp#17470 - Shipping Ministral 3, Devstral 2 (partnership with Mistral): ggml-org/llama.cpp#17644
- Shipping GPT-OSS (partnership with OpenAI and GGML team): ggml-org/llama.cpp#15091
- Bringing vision support to
llama-server: ggml-org/llama.cpp#12898 (on hackernews) - (Big) refactoring vision support in llama.cpp, introducing
libmtmd: ggml-org/llama.cpp#12849 - Support various vision models: Pixtral, SmolVLM, etc (check out this viral demo)
- Gemma 3 Vision support (partnership with Google): ggml-org/llama.cpp#12343
- WASM speed improvement: ggml-org/llama.cpp#11453 (also checkout wllama and hackernews post)
- Refactor argument parser, for a better CLI UX: ggml-org/llama.cpp#9308
- Revamp llama.cpp Web UI (currently deprecated, but was the precursor for the modern version): ggml-org/llama.cpp#10175
- llama.cpp <> Hugging Face inference endpoint integration: read docs
- Hot-swapping LoRA adapters: ggml-org/llama.cpp#8332
- Control vector generator: ggml-org/llama.cpp#7514
- Initial chat template support: ggml-org/llama.cpp#5538