Infini-AI-Lab

Popular repositories Loading

1. Sequoia SequoiaPublic

   scalable and robust tree-based speculative decoding algorithm

   Python 377 37

2. TriForce TriForcePublic

   [COLM 2024] TriForce: Lossless Acceleration of Long Sequence Generation with Hierarchical Speculative Decoding

   Python 281 20

3. MagicPIG MagicPIGPublic

   [ICLR2025 Spotlight] MagicPIG: LSH Sampling for Efficient LLM Generation

   Python 255 21

4. MagicDec MagicDecPublic

   [ICLR2025] Breaking Throughput-Latency Trade-off for Long Sequences with Speculative Decoding

   Python 156 12

5. MonarchRT MonarchRTPublic

   Python 141 13

6. UMbreLLa UMbreLLaPublic

   LLM Inference on consumer devices

   Python 131 15