ParCIS Lab, BUPT

Spec-K: Adaptive Resource Allocation via Speculative Preview for Efficient MoE Inference

The code of UltraGNN

FlashSparse significantly reduces the computation redundancy for unstructured sparsity (for SpMM and SDDMM) on Tensor Cores through a Swap-and-Transpose mapping strategy. FlashSparse is accepted by PPoPP 2025.

Chimera: bidirectional pipeline parallelism for efficiently training large-scale models.

C++/MPI proxies for distributed training of deep neural networks.

Ok-Topk is a scheme for distributed training with sparse gradients. Ok-Topk integrates a novel sparse allreduce algorithm (less than 6k communication volume which is asymptotically optimal) with the decentralized parallel Stochastic Gradient Descent (SGD) optimizer, and its convergence is proved theoretically and empirically.

Magicube is a high-performance library for quantized sparse matrix operations (SpMM and SDDMM) of deep learning on Tensor Cores.