MaKLlama

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

LLM inference in C/C++

Docker Model Runner

MAD (Model Automation and Dashboarding)

Manage GPU clusters for running LLMs

Ramalama is an open-source developer tool that simplifies the local serving of AI models from any source and facilitates their use for inference in production, all through the familiar language of containers.

Next-generation AI Agent Optimization Platform: Cozeloop addresses challenges in AI agent development by providing full-lifecycle management capabilities from development, debugging, and evaluation to monitoring.

OctoTools: An agentic framework with extensible tools for complex reasoning

LLM inference server implementation based on llama.cpp.

Stable Diffusion and Flux in pure C/C++