Forge smaller, run faster, measure honestly.
BitForge is an Inference Foundry project for exploring LLM quantization—compressing models to lower bit widths and measuring the trade-offs in quality, speed, and memory.
git clone https://github.com/Inference-Foundry/BitForge.git
cd BitForge
pip install -e .
bitforge --help# Quantize
bitforge quantize --model-id meta-llama/Llama-2-7b-hf --bits 4 --method gptq --output-path ./out/model
# Evaluate
bitforge evaluate --model-id ./out/model --dataset wikitext2 --device cudaRequirements: Python ≥ 3.8, PyTorch ≥ 2.1, GPU recommended for quantization.
| Path | Purpose |
|---|---|
bitforge/ | Python library — quantizers (GPTQ, AWQ, PTQ), calibration, metrics, CLI |
experiments/ | Reproducible benchmark scripts (perplexity, latency, QLoRA merge, outliers) |
site/ | Interactive quantization dashboard — open site/index.html in a browser |
Full documentation lives in the GitHub Wiki.
| Wiki page | Topics |
|---|---|
| Home | Overview and research questions |
| Getting Started | Install, hardware notes, first commands |
| Architecture | Package layout and module map |
| Quantization Theory | Methods, granularity, calibration |
| Experiments | Running benchmark scripts |
| CLI Reference | quantize and evaluate |
| Runtime Integration | llama.cpp, vLLM, Super-Ollama |
| Interactive Lab | Web dashboard guide |
| Contributing | How to help |
Early-stage (v0.1.0). Core interfaces and experiment scaffolding are in place; some pipelines return placeholder data until full calibration loops land. See the wiki Architecture page for details.
- Super-Ollama — serves quantized GGUF models at inference time
- Inference Foundry — org hub
Contributions welcome. See the wiki Contributing page and open an issue before large changes.