An implementation of Speculative RAG exploring latency-quality trade-offs in multi-draft retrieval. Features batched parallel drafting via vLLM and log-probability verifier selection for fast, high-quality QA on a single A100 GPU.
nlpbenchmarkinggpulatencyhigh-performance-computingthroughputquantizationfaissragvector-searchhpmla100llmvllmretrieval-augmented-generationspeculative-rag
-
Updated
Apr 28, 2026 - Python