draft-model

On-policy distillation for speculative decoding: train the draft model on the target's own rollouts, no offline feature cache. A fork of deepseek-ai/DeepSpec with DeepSeek-V4-Flash (DSpark) support.

Speculative decoding runtime with rejection sampling, adaptive gamma strategy, and provable correctness guarantees. Achieves 1.41x speedup on CPU with Qwen2-0.5B/1.5B pair. Draft model generates candidates, target model verifies in single forward pass. 31/31 tests passing.

Simulates speculative decoding to find the optimal speculation length K across 576 configurations (3 draft models x 8 K values x 6 acceptance rates x 4 cost ratios). Key findings: 6.06x max speedup, breakeven at cost_ratio=0.25, optimal K grows from 1-3 at 50% acceptance to 7-15 at 95% acceptance.

Measures real speculative decoding speedup using the official HuggingFace assistant_model API across 4 model pairs and output lengths up to 512 tokens. Best result: distilgpt2->gpt2-medium achieves 1.747x speedup at 512 output tokens. Validates that cost_ratio and output_length are the key parameters.

Tree-based speculative decoding benchmarked against linear under equal verification budget — branch factor, depth, and draft quality sweep with fair node-level comparison.

Analytical benchmark for speculative decoding at batch sizes 1-64. Finds the gamma crossover where speculative decoding becomes slower than greedy, and shows that batch size does not degrade performance when continuous refill is active.

Calibrated simulation benchmark for real-time LLM request routing, comparing complexity signals, output-length awareness, cost savings, and quality-risk trade-offs.

CLI for building and testing DFlash-style speculative decoding draft models.