Pinned Loading

1. ModelCloud/GPTQModelModelCloud/GPTQModelPublic

   LLM model quantization (compression) toolkit with HW acceleration support for Nvidia, AMD, Intel GPU and Intel/AMD/Apple CPU via HF, vLLM, and SGLang.

   Python 1.2k 203

2. ModelCloud/TokenicerModelCloud/TokenicerPublic

   A (nicer) tokenizer you want to use for model inference and training: with all known peventable gotchas normalized or auto-fixed.

   Python 12 4

3. ModelCloud/Device-SMIModelCloud/Device-SMIPublic

   Self-contained Python lib with zero-dependencies that give you a unified device properties for gpu, cpu, and npu. No more calling separate tools such as nvidia-smi or /proc/cpuinfo and parsing it y…

   Python 16 1

4. sgl-project/sglangsgl-project/sglangPublic

   SGLang is a high-performance serving framework for large language models and multimodal models.

   Python 32.6k 8.3k

5. vllm-project/vllmvllm-project/vllmPublic

   A high-throughput and memory-efficient inference and serving engine for LLMs

   Python 90.3k 21.3k

6. ModelCloud/LogBarModelCloud/LogBarPublic

   A unified Logger and ProgressBar util with zero dependencies.

   Python 10