Hidet is an open-source DNN inference framework based on compilation. It supports end-to-end compilation of DNN models from PyTorch and ONNX to efficient cuda kernels. A series of graph-level and operator-level optimizations are applied to optimize the performance.
pip install hidetSee here for building from source.
Optimize a PyTorch model through hidet (require PyTorch 2.0):
importtorchimporthidet# Register hidet backends for pytorch dynamo, can be omitted if you import torch before hidethidet.torch.register_dynamo_backends() # Define pytorch modelmodel=torch.hub.load('pytorch/vision:v0.6.0', 'resnet18', pretrained=True).cuda().eval()
x=torch.rand(1, 3, 224, 224).cuda()
# Compile the model through Hidetmodel_opt=torch.compile(model, backend='hidet') # Run the optimized modely=model_opt(x)See the following tutorials to learn other usgae:
Hidet originates from the following research work. If you used Hidet in your research, welcome to cite our paper.
- Hidet: Task-Mapping Programming Paradigm for Deep Learning Tensor Programs.
Yaoyao Ding, Cody Hao Yu, Bojian Zheng, Yizhi Liu, Yida Wang, and Gennady Pekhimenko.
Hidet is currently under active development by a team at CentML Inc.
We welcome contributions from the community. Please see contribution guide for more details.
Hidet is released under the Apache 2.0 license.