Tiny configuration for Triton Inference Server
-
Updated
Jan 10, 2025 - Python
Tiny configuration for Triton Inference Server
Triton backend for https://github.com/OpenNMT/CTranslate2
A complete containerized setup for Triton inference server and its python client using a realistic pre-trained XGBoost classifier model.
Triton backend is difficult for a client to use whether it's sending by rest-api or grpc. If the client wants to customize the request body then this repository would like to offer a sidecar along with rest-api and triton client on Kubernetes.
High-throughput Triton client that handles all shared memory complexity for you.
A Node.js client for the Triton Inference Server.
QuickStart for Deploying a Basic Model on the Triton Inference Server
To associate your repository with the tritonclient topic, visit your repo's landing page and select "manage topics."