Skip to content
BranchesTags

Repository files navigation

LicenseDocumentationRoadmap

TensorRT Open Source Software

This repository contains the Open Source Software (OSS) components of NVIDIA TensorRT. It includes the sources for TensorRT plugins and ONNX parser, as well as sample applications demonstrating usage and capabilities of the TensorRT platform. These open source software components are a subset of the TensorRT General Availability (GA) release with some extensions and bug-fixes.

Need enterprise support? NVIDIA global support is available for TensorRT with the NVIDIA AI Enterprise software suite. Check out NVIDIA LaunchPad for free access to a set of hands-on labs with TensorRT hosted on NVIDIA infrastructure.

Join the TensorRT and Triton community and stay current on the latest product updates, bug fixes, content, best practices, and more.

Prebuilt TensorRT Python Package

We provide the TensorRT Python package for an easy installation.
To install:

pip install tensorrt

You can skip the Build section to enjoy TensorRT with Python.

Build

Prerequisites

To build the TensorRT-OSS components, you will first need the following software packages.

TensorRT GA build

  • TensorRT v10.14.1.48
    • Available from direct download links listed below

System Packages

Optional Packages

Downloading TensorRT Build

  1. Download TensorRT OSS

    git clone -b main https://github.com/nvidia/TensorRT TensorRT
cd TensorRT
git submodule update --init --recursive

  2. (Optional - if not using TensorRT container) Specify the TensorRT GA release build path

    If using the TensorRT OSS build container, TensorRT libraries are preinstalled under /usr/lib/x86_64-linux-gnu and you may skip this step.

    Else download and extract the TensorRT GA build from NVIDIA Developer Zone with the direct links below:

    Example: Ubuntu 22.04 on x86-64 with cuda-13.0

    cd~/Downloads
tar -xvzf TensorRT-10.14.1.48.Linux.x86_64-gnu.cuda-13.0.tar.gz
export TRT_LIBPATH=`pwd`/TensorRT-10.14.1.48

    Example: Windows on x86-64 with cuda-12.9

    Expand-Archive-Path TensorRT-10.14.1.48.Windows.win10.cuda-12.9.zip
$env:TRT_LIBPATH="$pwd\TensorRT-10.14.1.48\lib"

Setting Up The Build Environment

For Linux platforms, we recommend that you generate a docker container for building TensorRT OSS as described below. For native builds, please install the prerequisiteSystem Packages.

  1. Generate the TensorRT-OSS build container.

    Example: Ubuntu 24.04 on x86-64 with cuda-13.0 (default)

    ./docker/build.sh --file docker/ubuntu-24.04.Dockerfile --tag tensorrt-ubuntu24.04-cuda13.0

    Example: Rockylinux8 on x86-64 with cuda-13.0

    ./docker/build.sh --file docker/rockylinux8.Dockerfile --tag tensorrt-rockylinux8-cuda13.0

    Example: Ubuntu 24.04 cross-compile for Jetson (aarch64) with cuda-13.0 (JetPack SDK)

    ./docker/build.sh --file docker/ubuntu-cross-aarch64.Dockerfile --tag tensorrt-jetpack-cuda13.0

    Example: Ubuntu 24.04 on aarch64 with cuda-13.0

    ./docker/build.sh --file docker/ubuntu-24.04-aarch64.Dockerfile --tag tensorrt-aarch64-ubuntu24.04-cuda13.0

  2. Launch the TensorRT-OSS build container.

    Example: Ubuntu 24.04 build container

    ./docker/launch.sh --tag tensorrt-ubuntu24.04-cuda13.0 --gpus all

    NOTE:
    1. Use the --tag corresponding to build container generated in Step 1.
    2. NVIDIA Container Toolkit is required for GPU access (running TensorRT applications) inside the build container.
    3. sudo password for Ubuntu build containers is 'nvidia'.
    4. Specify port number using --jupyter <port> for launching Jupyter notebooks.
    5. Write permission to this folder is required as this folder will be mounted inside the docker container for uid:gid of 1000:1000.

Building TensorRT-OSS

  • Generate Makefiles and build

    Example: Linux (x86-64) build with default cuda-13.0

    cd$TRT_OSSPATH
mkdir -p build &&cd build
cmake .. -DTRT_LIB_DIR=$TRT_LIBPATH -DTRT_OUT_DIR=`pwd`/out
make -j$(nproc)

    Example: Linux (aarch64) build with default cuda-13.0

    cd$TRT_OSSPATH
mkdir -p build &&cd build
cmake .. -DTRT_LIB_DIR=$TRT_LIBPATH -DTRT_OUT_DIR=`pwd`/out -DCMAKE_TOOLCHAIN_FILE=$TRT_OSSPATH/cmake/toolchains/cmake_aarch64-native.toolchain
make -j$(nproc)

    Example: Native build on Jetson Thor (aarch64) with cuda-13.0

    cd$TRT_OSSPATH
mkdir -p build &&cd build
cmake .. -DTRT_LIB_DIR=$TRT_LIBPATH -DTRT_OUT_DIR=`pwd`/out -DTRT_PLATFORM_ID=aarch64
CC=/usr/bin/gcc make -j$(nproc)

    NOTE: C compiler must be explicitly specified via CC= for native aarch64 builds of protobuf.

    Example: Ubuntu 24.04 Cross-Compile for Jetson Thor (aarch64) with cuda-13.0 (JetPack)

    cd$TRT_OSSPATH
mkdir -p build &&cd build
cmake .. -DTRT_LIB_DIR=$TRT_LIBPATH -DCMAKE_TOOLCHAIN_FILE=$TRT_OSSPATH/cmake/toolchains/cmake_aarch64_cross.toolchain
make -j$(nproc)

    Example: Ubuntu 24.04 Cross-Compile for DriveOS (aarch64) with cuda-13.0

    cd$TRT_OSSPATH
mkdir -p build &&cd build
cmake .. -DTRT_LIB_DIR=$TRT_LIBPATH -DCMAKE_TOOLCHAIN_FILE=$TRT_OSSPATH/cmake/toolchains/cmake_aarch64_dos_cross.toolchain
make -j$(nproc)

    Example: Native builds on Windows (x86) with cuda-13.0

    cd$TRT_OSSPATH
mkdir -p build
cd -p build
cmake .. -DTRT_LIB_DIR="$env:TRT_LIBPATH" -DTRT_OUT_DIR="$pwd\\out"
msbuild TensorRT.sln /property:Configuration=Release -m:$env:NUMBER_OF_PROCESSORS

    NOTE: The default CUDA version used by CMake is 13.0. To override this, for example to 12.9, append -DCUDA_VERSION=12.9 to the cmake command.

  • Required CMake build arguments are:

    • TRT_LIB_DIR: Path to the TensorRT installation directory containing libraries.
    • TRT_OUT_DIR: Output directory where generated build artifacts will be copied.

  • Optional CMake build arguments:

    • CMAKE_BUILD_TYPE: Specify if binaries generated are for release or debug (contain debug symbols). Values consists of [Release] | Debug
    • CUDA_VERSION: The version of CUDA to target, for example [12.9.9].
    • CUDNN_VERSION: The version of cuDNN to target, for example [8.9].
    • PROTOBUF_VERSION: The version of Protobuf to use, for example [3.20.1]. Note: Changing this will not configure CMake to use a system version of Protobuf, it will configure CMake to download and try building that version.
    • CMAKE_TOOLCHAIN_FILE: The path to a toolchain file for cross compilation.
    • BUILD_PARSERS: Specify if the parsers should be built, for example [ON] | OFF. If turned OFF, CMake will try to find precompiled versions of the parser libraries to use in compiling samples. First in ${TRT_LIB_DIR}, then on the system. If the build type is Debug, then it will prefer debug builds of the libraries before release versions if available.
    • BUILD_PLUGINS: Specify if the plugins should be built, for example [ON] | OFF. If turned OFF, CMake will try to find a precompiled version of the plugin library to use in compiling samples. First in ${TRT_LIB_DIR}, then on the system. If the build type is Debug, then it will prefer debug builds of the libraries before release versions if available.
    • BUILD_SAMPLES: Specify if the samples should be built, for example [ON] | OFF.
    • GPU_ARCHS: GPU (SM) architectures to target. By default we generate CUDA code for all major SMs. Specific SM versions can be specified here as a quoted space-separated list to reduce compilation time and binary size. Table of compute capabilities of NVIDIA GPUs can be found here. Examples: - NVidia A100: -DGPU_ARCHS="80" - RTX 50 series: -DGPU_ARCHS="120" - Multiple SMs: -DGPU_ARCHS="80 120"
    • TRT_PLATFORM_ID: Bare-metal build (unlike containerized cross-compilation). Currently supported options: x86_64 (default).

References

TensorRT Resources

Known Issues

About

NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT.

developer.nvidia.com/tensorrt

Resources

Readme
License

Contributing

Contributing
Activity
Custom properties

Stars

1 star

Watchers

0 watching

Forks

0 forks
Report repository

Releases

Packages

Used by

Contributors

Languages

, 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all 
 blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
GitHub - BlueRiverTechnology/TensorRT: NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT. · GitHub
Skip to content
BranchesTags
Repository files navigation
LicenseDocumentationRoadmap
TensorRT Open Source Software
This repository contains the Open Source Software (OSS) components of NVIDIA TensorRT. It includes the sources for TensorRT plugins and ONNX parser, as well as sample applications demonstrating usage and capabilities of the TensorRT platform. These open source software components are a subset of the TensorRT General Availability (GA) release with some extensions and bug-fixes.
Need enterprise support? NVIDIA global support is available for TensorRT with the NVIDIA AI Enterprise software suite. Check out NVIDIA LaunchPad for free access to a set of hands-on labs with TensorRT hosted on NVIDIA infrastructure.
Join the TensorRT and Triton community and stay current on the latest product updates, bug fixes, content, best practices, and more.
Prebuilt TensorRT Python Package
We provide the TensorRT Python package for an easy installation. 

To install:
pip install tensorrt
You can skip the Build section to enjoy TensorRT with Python.
Build
Prerequisites
To build the TensorRT-OSS components, you will first need the following software packages.
TensorRT GA build
  • TensorRT v10.14.1.48

    • Available from direct download links listed below
System Packages
Optional Packages
Downloading TensorRT Build
  1. Download TensorRT OSS
    git clone -b main https://github.com/nvidia/TensorRT TensorRT
cd TensorRT
git submodule update --init --recursive
  2. (Optional - if not using TensorRT container) Specify the TensorRT GA release build path
    If using the TensorRT OSS build container, TensorRT libraries are preinstalled under /usr/lib/x86_64-linux-gnu and you may skip this step.
    Else download and extract the TensorRT GA build from NVIDIA Developer Zone with the direct links below:
    Example: Ubuntu 22.04 on x86-64 with cuda-13.0
    cd~/Downloads
tar -xvzf TensorRT-10.14.1.48.Linux.x86_64-gnu.cuda-13.0.tar.gz
export TRT_LIBPATH=`pwd`/TensorRT-10.14.1.48
    Example: Windows on x86-64 with cuda-12.9
    Expand-Archive-Path TensorRT-10.14.1.48.Windows.win10.cuda-12.9.zip
$env:TRT_LIBPATH="$pwd\TensorRT-10.14.1.48\lib"
Setting Up The Build Environment
For Linux platforms, we recommend that you generate a docker container for building TensorRT OSS as described below. For native builds, please install the prerequisiteSystem Packages.
  1. Generate the TensorRT-OSS build container.
    Example: Ubuntu 24.04 on x86-64 with cuda-13.0 (default)
    ./docker/build.sh --file docker/ubuntu-24.04.Dockerfile --tag tensorrt-ubuntu24.04-cuda13.0
    Example: Rockylinux8 on x86-64 with cuda-13.0
    ./docker/build.sh --file docker/rockylinux8.Dockerfile --tag tensorrt-rockylinux8-cuda13.0
    Example: Ubuntu 24.04 cross-compile for Jetson (aarch64) with cuda-13.0 (JetPack SDK)
    ./docker/build.sh --file docker/ubuntu-cross-aarch64.Dockerfile --tag tensorrt-jetpack-cuda13.0
    Example: Ubuntu 24.04 on aarch64 with cuda-13.0
    ./docker/build.sh --file docker/ubuntu-24.04-aarch64.Dockerfile --tag tensorrt-aarch64-ubuntu24.04-cuda13.0
  2. Launch the TensorRT-OSS build container.
    Example: Ubuntu 24.04 build container
    ./docker/launch.sh --tag tensorrt-ubuntu24.04-cuda13.0 --gpus all
    NOTE:

     1. Use the --tag corresponding to build container generated in Step 1.

     2. NVIDIA Container Toolkit is required for GPU access (running TensorRT applications) inside the build container.

     3. sudo password for Ubuntu build containers is 'nvidia'.

     4. Specify port number using --jupyter <port> for launching Jupyter notebooks.

     5. Write permission to this folder is required as this folder will be mounted inside the docker container for uid:gid of 1000:1000.
Building TensorRT-OSS
  • Generate Makefiles and build
    Example: Linux (x86-64) build with default cuda-13.0
    cd$TRT_OSSPATH
mkdir -p build &&cd build
cmake .. -DTRT_LIB_DIR=$TRT_LIBPATH -DTRT_OUT_DIR=`pwd`/out
make -j$(nproc)
    Example: Linux (aarch64) build with default cuda-13.0
    cd$TRT_OSSPATH
mkdir -p build &&cd build
cmake .. -DTRT_LIB_DIR=$TRT_LIBPATH -DTRT_OUT_DIR=`pwd`/out -DCMAKE_TOOLCHAIN_FILE=$TRT_OSSPATH/cmake/toolchains/cmake_aarch64-native.toolchain
make -j$(nproc)
    Example: Native build on Jetson Thor (aarch64) with cuda-13.0
    cd$TRT_OSSPATH
mkdir -p build &&cd build
cmake .. -DTRT_LIB_DIR=$TRT_LIBPATH -DTRT_OUT_DIR=`pwd`/out -DTRT_PLATFORM_ID=aarch64
CC=/usr/bin/gcc make -j$(nproc)
    NOTE: C compiler must be explicitly specified via CC= for native aarch64 builds of protobuf.
    Example: Ubuntu 24.04 Cross-Compile for Jetson Thor (aarch64) with cuda-13.0 (JetPack)
    cd$TRT_OSSPATH
mkdir -p build &&cd build
cmake .. -DTRT_LIB_DIR=$TRT_LIBPATH -DCMAKE_TOOLCHAIN_FILE=$TRT_OSSPATH/cmake/toolchains/cmake_aarch64_cross.toolchain
make -j$(nproc)
    Example: Ubuntu 24.04 Cross-Compile for DriveOS (aarch64) with cuda-13.0
    cd$TRT_OSSPATH
mkdir -p build &&cd build
cmake .. -DTRT_LIB_DIR=$TRT_LIBPATH -DCMAKE_TOOLCHAIN_FILE=$TRT_OSSPATH/cmake/toolchains/cmake_aarch64_dos_cross.toolchain
make -j$(nproc)
    Example: Native builds on Windows (x86) with cuda-13.0
    cd$TRT_OSSPATH
mkdir -p build
cd -p build
cmake .. -DTRT_LIB_DIR="$env:TRT_LIBPATH" -DTRT_OUT_DIR="$pwd\\out"
msbuild TensorRT.sln /property:Configuration=Release -m:$env:NUMBER_OF_PROCESSORS
    NOTE: The default CUDA version used by CMake is 13.0. To override this, for example to 12.9, append -DCUDA_VERSION=12.9 to the cmake command.
  • Required CMake build arguments are:
    • TRT_LIB_DIR: Path to the TensorRT installation directory containing libraries.
    • TRT_OUT_DIR: Output directory where generated build artifacts will be copied.
  • Optional CMake build arguments:
    • CMAKE_BUILD_TYPE: Specify if binaries generated are for release or debug (contain debug symbols). Values consists of [Release] | Debug
    • CUDA_VERSION: The version of CUDA to target, for example [12.9.9].
    • CUDNN_VERSION: The version of cuDNN to target, for example [8.9].
    • PROTOBUF_VERSION: The version of Protobuf to use, for example [3.20.1]. Note: Changing this will not configure CMake to use a system version of Protobuf, it will configure CMake to download and try building that version.
    • CMAKE_TOOLCHAIN_FILE: The path to a toolchain file for cross compilation.
    • BUILD_PARSERS: Specify if the parsers should be built, for example [ON] | OFF. If turned OFF, CMake will try to find precompiled versions of the parser libraries to use in compiling samples. First in ${TRT_LIB_DIR}, then on the system. If the build type is Debug, then it will prefer debug builds of the libraries before release versions if available.
    • BUILD_PLUGINS: Specify if the plugins should be built, for example [ON] | OFF. If turned OFF, CMake will try to find a precompiled version of the plugin library to use in compiling samples. First in ${TRT_LIB_DIR}, then on the system. If the build type is Debug, then it will prefer debug builds of the libraries before release versions if available.
    • BUILD_SAMPLES: Specify if the samples should be built, for example [ON] | OFF.
    • GPU_ARCHS: GPU (SM) architectures to target. By default we generate CUDA code for all major SMs. Specific SM versions can be specified here as a quoted space-separated list to reduce compilation time and binary size. Table of compute capabilities of NVIDIA GPUs can be found here. Examples: - NVidia A100: -DGPU_ARCHS="80" - RTX 50 series: -DGPU_ARCHS="120" - Multiple SMs: -DGPU_ARCHS="80 120"
    • TRT_PLATFORM_ID: Bare-metal build (unlike containerized cross-compilation). Currently supported options: x86_64 (default).
References
TensorRT Resources
Known Issues
About
NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT.
developer.nvidia.com/tensorrt
Resources
Readme
License
Contributing
Contributing
Activity
Custom properties
Stars
1 star
Watchers
0 watching
Forks
0 forks
Report repository
Releases
Packages
Used by
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Force GitHub README to respect dark mode
(function() {
var style = document.createElement('style');
style.textContent = '
.markdown-body {
color-scheme: dark light;
}
.markdown-body pre { background: #161b22 !important; }
.markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }
.markdown-body table th, .markdown-body table td { border-color: #30363d !important; }
.markdown-body img { background: #0d1117; }
.markdown-body blockquote { border-left-color: #8b949e; }
.markdown-body hr { border-color: #30363d; }
';
document.head.appendChild(style);
})();
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
GitHub - BlueRiverTechnology/TensorRT: NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT. · GitHub
Skip to content
BranchesTags
Repository files navigation
LicenseDocumentationRoadmap
TensorRT Open Source Software
This repository contains the Open Source Software (OSS) components of NVIDIA TensorRT. It includes the sources for TensorRT plugins and ONNX parser, as well as sample applications demonstrating usage and capabilities of the TensorRT platform. These open source software components are a subset of the TensorRT General Availability (GA) release with some extensions and bug-fixes.
Need enterprise support? NVIDIA global support is available for TensorRT with the NVIDIA AI Enterprise software suite. Check out NVIDIA LaunchPad for free access to a set of hands-on labs with TensorRT hosted on NVIDIA infrastructure.
Join the TensorRT and Triton community and stay current on the latest product updates, bug fixes, content, best practices, and more.
Prebuilt TensorRT Python Package
We provide the TensorRT Python package for an easy installation. 

To install:
pip install tensorrt
You can skip the Build section to enjoy TensorRT with Python.
Build
Prerequisites
To build the TensorRT-OSS components, you will first need the following software packages.
TensorRT GA build
  • TensorRT v10.14.1.48

    • Available from direct download links listed below
System Packages
Optional Packages
Downloading TensorRT Build
  1. Download TensorRT OSS
    git clone -b main https://github.com/nvidia/TensorRT TensorRT
cd TensorRT
git submodule update --init --recursive
  2. (Optional - if not using TensorRT container) Specify the TensorRT GA release build path
    If using the TensorRT OSS build container, TensorRT libraries are preinstalled under /usr/lib/x86_64-linux-gnu and you may skip this step.
    Else download and extract the TensorRT GA build from NVIDIA Developer Zone with the direct links below:
    Example: Ubuntu 22.04 on x86-64 with cuda-13.0
    cd~/Downloads
tar -xvzf TensorRT-10.14.1.48.Linux.x86_64-gnu.cuda-13.0.tar.gz
export TRT_LIBPATH=`pwd`/TensorRT-10.14.1.48
    Example: Windows on x86-64 with cuda-12.9
    Expand-Archive-Path TensorRT-10.14.1.48.Windows.win10.cuda-12.9.zip
$env:TRT_LIBPATH="$pwd\TensorRT-10.14.1.48\lib"
Setting Up The Build Environment
For Linux platforms, we recommend that you generate a docker container for building TensorRT OSS as described below. For native builds, please install the prerequisiteSystem Packages.
  1. Generate the TensorRT-OSS build container.
    Example: Ubuntu 24.04 on x86-64 with cuda-13.0 (default)
    ./docker/build.sh --file docker/ubuntu-24.04.Dockerfile --tag tensorrt-ubuntu24.04-cuda13.0
    Example: Rockylinux8 on x86-64 with cuda-13.0
    ./docker/build.sh --file docker/rockylinux8.Dockerfile --tag tensorrt-rockylinux8-cuda13.0
    Example: Ubuntu 24.04 cross-compile for Jetson (aarch64) with cuda-13.0 (JetPack SDK)
    ./docker/build.sh --file docker/ubuntu-cross-aarch64.Dockerfile --tag tensorrt-jetpack-cuda13.0
    Example: Ubuntu 24.04 on aarch64 with cuda-13.0
    ./docker/build.sh --file docker/ubuntu-24.04-aarch64.Dockerfile --tag tensorrt-aarch64-ubuntu24.04-cuda13.0
  2. Launch the TensorRT-OSS build container.
    Example: Ubuntu 24.04 build container
    ./docker/launch.sh --tag tensorrt-ubuntu24.04-cuda13.0 --gpus all
    NOTE:

     1. Use the --tag corresponding to build container generated in Step 1.

     2. NVIDIA Container Toolkit is required for GPU access (running TensorRT applications) inside the build container.

     3. sudo password for Ubuntu build containers is 'nvidia'.

     4. Specify port number using --jupyter <port> for launching Jupyter notebooks.

     5. Write permission to this folder is required as this folder will be mounted inside the docker container for uid:gid of 1000:1000.
Building TensorRT-OSS
  • Generate Makefiles and build
    Example: Linux (x86-64) build with default cuda-13.0
    cd$TRT_OSSPATH
mkdir -p build &&cd build
cmake .. -DTRT_LIB_DIR=$TRT_LIBPATH -DTRT_OUT_DIR=`pwd`/out
make -j$(nproc)
    Example: Linux (aarch64) build with default cuda-13.0
    cd$TRT_OSSPATH
mkdir -p build &&cd build
cmake .. -DTRT_LIB_DIR=$TRT_LIBPATH -DTRT_OUT_DIR=`pwd`/out -DCMAKE_TOOLCHAIN_FILE=$TRT_OSSPATH/cmake/toolchains/cmake_aarch64-native.toolchain
make -j$(nproc)
    Example: Native build on Jetson Thor (aarch64) with cuda-13.0
    cd$TRT_OSSPATH
mkdir -p build &&cd build
cmake .. -DTRT_LIB_DIR=$TRT_LIBPATH -DTRT_OUT_DIR=`pwd`/out -DTRT_PLATFORM_ID=aarch64
CC=/usr/bin/gcc make -j$(nproc)
    NOTE: C compiler must be explicitly specified via CC= for native aarch64 builds of protobuf.
    Example: Ubuntu 24.04 Cross-Compile for Jetson Thor (aarch64) with cuda-13.0 (JetPack)
    cd$TRT_OSSPATH
mkdir -p build &&cd build
cmake .. -DTRT_LIB_DIR=$TRT_LIBPATH -DCMAKE_TOOLCHAIN_FILE=$TRT_OSSPATH/cmake/toolchains/cmake_aarch64_cross.toolchain
make -j$(nproc)
    Example: Ubuntu 24.04 Cross-Compile for DriveOS (aarch64) with cuda-13.0
    cd$TRT_OSSPATH
mkdir -p build &&cd build
cmake .. -DTRT_LIB_DIR=$TRT_LIBPATH -DCMAKE_TOOLCHAIN_FILE=$TRT_OSSPATH/cmake/toolchains/cmake_aarch64_dos_cross.toolchain
make -j$(nproc)
    Example: Native builds on Windows (x86) with cuda-13.0
    cd$TRT_OSSPATH
mkdir -p build
cd -p build
cmake .. -DTRT_LIB_DIR="$env:TRT_LIBPATH" -DTRT_OUT_DIR="$pwd\\out"
msbuild TensorRT.sln /property:Configuration=Release -m:$env:NUMBER_OF_PROCESSORS
    NOTE: The default CUDA version used by CMake is 13.0. To override this, for example to 12.9, append -DCUDA_VERSION=12.9 to the cmake command.
  • Required CMake build arguments are:
    • TRT_LIB_DIR: Path to the TensorRT installation directory containing libraries.
    • TRT_OUT_DIR: Output directory where generated build artifacts will be copied.
  • Optional CMake build arguments:
    • CMAKE_BUILD_TYPE: Specify if binaries generated are for release or debug (contain debug symbols). Values consists of [Release] | Debug
    • CUDA_VERSION: The version of CUDA to target, for example [12.9.9].
    • CUDNN_VERSION: The version of cuDNN to target, for example [8.9].
    • PROTOBUF_VERSION: The version of Protobuf to use, for example [3.20.1]. Note: Changing this will not configure CMake to use a system version of Protobuf, it will configure CMake to download and try building that version.
    • CMAKE_TOOLCHAIN_FILE: The path to a toolchain file for cross compilation.
    • BUILD_PARSERS: Specify if the parsers should be built, for example [ON] | OFF. If turned OFF, CMake will try to find precompiled versions of the parser libraries to use in compiling samples. First in ${TRT_LIB_DIR}, then on the system. If the build type is Debug, then it will prefer debug builds of the libraries before release versions if available.
    • BUILD_PLUGINS: Specify if the plugins should be built, for example [ON] | OFF. If turned OFF, CMake will try to find a precompiled version of the plugin library to use in compiling samples. First in ${TRT_LIB_DIR}, then on the system. If the build type is Debug, then it will prefer debug builds of the libraries before release versions if available.
    • BUILD_SAMPLES: Specify if the samples should be built, for example [ON] | OFF.
    • GPU_ARCHS: GPU (SM) architectures to target. By default we generate CUDA code for all major SMs. Specific SM versions can be specified here as a quoted space-separated list to reduce compilation time and binary size. Table of compute capabilities of NVIDIA GPUs can be found here. Examples: - NVidia A100: -DGPU_ARCHS="80" - RTX 50 series: -DGPU_ARCHS="120" - Multiple SMs: -DGPU_ARCHS="80 120"
    • TRT_PLATFORM_ID: Bare-metal build (unlike containerized cross-compilation). Currently supported options: x86_64 (default).
References
TensorRT Resources
Known Issues
About
NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT.
developer.nvidia.com/tensorrt
Resources
Readme
License
Contributing
Contributing
Activity
Custom properties
Stars
1 star
Watchers
0 watching
Forks
0 forks
Report repository
Releases
Packages
Used by
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Highlight search terms from Google/DuckDuckGo/Bing referrer
(function() {
var ref = document.referrer;
var terms = [];
if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {
var url = new URL(ref);
var q = url.searchParams.get('q') || url.searchParams.get('p');
if (q) {
terms = q.split(/\s+/).filter(function(t) { return t.length > 2; });
}
}
if (terms.length === 0) return;
var style = document.createElement('style');
style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';
document.head.appendChild(style);
function highlight(node) {
if (node.nodeType === 3) { // text node
var text = node.textContent;
var found = false;
terms.forEach(function(term) {
var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi');
if (regex.test(text)) {
found = true;
var frag = document.createDocumentFragment();
var parts = text.split(regex);
parts.forEach(function(part, i) {
if (i % 2 === 0) {
frag.appendChild(document.createTextNode(part));
} else {
var span = document.createElement('span');
span.className = 'userscript-highlight';
span.textContent = part;
frag.appendChild(span);
}
});
node.parentNode.replaceChild(frag, node);
}
});
} else if (node.nodeType === 1 && node.childNodes) { // element
var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];
if (!skipTags.includes(node.tagName)) {
Array.from(node.childNodes).forEach(highlight);
}
}
}
highlight(document.body);
// Re-highlight on dynamic content
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1 || node.nodeType === 3) highlight(node);
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
GitHub - BlueRiverTechnology/TensorRT: NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT. · GitHub
Skip to content
BranchesTags
Repository files navigation
LicenseDocumentationRoadmap
TensorRT Open Source Software
This repository contains the Open Source Software (OSS) components of NVIDIA TensorRT. It includes the sources for TensorRT plugins and ONNX parser, as well as sample applications demonstrating usage and capabilities of the TensorRT platform. These open source software components are a subset of the TensorRT General Availability (GA) release with some extensions and bug-fixes.
Need enterprise support? NVIDIA global support is available for TensorRT with the NVIDIA AI Enterprise software suite. Check out NVIDIA LaunchPad for free access to a set of hands-on labs with TensorRT hosted on NVIDIA infrastructure.
Join the TensorRT and Triton community and stay current on the latest product updates, bug fixes, content, best practices, and more.
Prebuilt TensorRT Python Package
We provide the TensorRT Python package for an easy installation. 

To install:
pip install tensorrt
You can skip the Build section to enjoy TensorRT with Python.
Build
Prerequisites
To build the TensorRT-OSS components, you will first need the following software packages.
TensorRT GA build
  • TensorRT v10.14.1.48

    • Available from direct download links listed below
System Packages
Optional Packages
Downloading TensorRT Build
  1. Download TensorRT OSS
    git clone -b main https://github.com/nvidia/TensorRT TensorRT
cd TensorRT
git submodule update --init --recursive
  2. (Optional - if not using TensorRT container) Specify the TensorRT GA release build path
    If using the TensorRT OSS build container, TensorRT libraries are preinstalled under /usr/lib/x86_64-linux-gnu and you may skip this step.
    Else download and extract the TensorRT GA build from NVIDIA Developer Zone with the direct links below:
    Example: Ubuntu 22.04 on x86-64 with cuda-13.0
    cd~/Downloads
tar -xvzf TensorRT-10.14.1.48.Linux.x86_64-gnu.cuda-13.0.tar.gz
export TRT_LIBPATH=`pwd`/TensorRT-10.14.1.48
    Example: Windows on x86-64 with cuda-12.9
    Expand-Archive-Path TensorRT-10.14.1.48.Windows.win10.cuda-12.9.zip
$env:TRT_LIBPATH="$pwd\TensorRT-10.14.1.48\lib"
Setting Up The Build Environment
For Linux platforms, we recommend that you generate a docker container for building TensorRT OSS as described below. For native builds, please install the prerequisiteSystem Packages.
  1. Generate the TensorRT-OSS build container.
    Example: Ubuntu 24.04 on x86-64 with cuda-13.0 (default)
    ./docker/build.sh --file docker/ubuntu-24.04.Dockerfile --tag tensorrt-ubuntu24.04-cuda13.0
    Example: Rockylinux8 on x86-64 with cuda-13.0
    ./docker/build.sh --file docker/rockylinux8.Dockerfile --tag tensorrt-rockylinux8-cuda13.0
    Example: Ubuntu 24.04 cross-compile for Jetson (aarch64) with cuda-13.0 (JetPack SDK)
    ./docker/build.sh --file docker/ubuntu-cross-aarch64.Dockerfile --tag tensorrt-jetpack-cuda13.0
    Example: Ubuntu 24.04 on aarch64 with cuda-13.0
    ./docker/build.sh --file docker/ubuntu-24.04-aarch64.Dockerfile --tag tensorrt-aarch64-ubuntu24.04-cuda13.0
  2. Launch the TensorRT-OSS build container.
    Example: Ubuntu 24.04 build container
    ./docker/launch.sh --tag tensorrt-ubuntu24.04-cuda13.0 --gpus all
    NOTE:

     1. Use the --tag corresponding to build container generated in Step 1.

     2. NVIDIA Container Toolkit is required for GPU access (running TensorRT applications) inside the build container.

     3. sudo password for Ubuntu build containers is 'nvidia'.

     4. Specify port number using --jupyter <port> for launching Jupyter notebooks.

     5. Write permission to this folder is required as this folder will be mounted inside the docker container for uid:gid of 1000:1000.
Building TensorRT-OSS
  • Generate Makefiles and build
    Example: Linux (x86-64) build with default cuda-13.0
    cd$TRT_OSSPATH
mkdir -p build &&cd build
cmake .. -DTRT_LIB_DIR=$TRT_LIBPATH -DTRT_OUT_DIR=`pwd`/out
make -j$(nproc)
    Example: Linux (aarch64) build with default cuda-13.0
    cd$TRT_OSSPATH
mkdir -p build &&cd build
cmake .. -DTRT_LIB_DIR=$TRT_LIBPATH -DTRT_OUT_DIR=`pwd`/out -DCMAKE_TOOLCHAIN_FILE=$TRT_OSSPATH/cmake/toolchains/cmake_aarch64-native.toolchain
make -j$(nproc)
    Example: Native build on Jetson Thor (aarch64) with cuda-13.0
    cd$TRT_OSSPATH
mkdir -p build &&cd build
cmake .. -DTRT_LIB_DIR=$TRT_LIBPATH -DTRT_OUT_DIR=`pwd`/out -DTRT_PLATFORM_ID=aarch64
CC=/usr/bin/gcc make -j$(nproc)
    NOTE: C compiler must be explicitly specified via CC= for native aarch64 builds of protobuf.
    Example: Ubuntu 24.04 Cross-Compile for Jetson Thor (aarch64) with cuda-13.0 (JetPack)
    cd$TRT_OSSPATH
mkdir -p build &&cd build
cmake .. -DTRT_LIB_DIR=$TRT_LIBPATH -DCMAKE_TOOLCHAIN_FILE=$TRT_OSSPATH/cmake/toolchains/cmake_aarch64_cross.toolchain
make -j$(nproc)
    Example: Ubuntu 24.04 Cross-Compile for DriveOS (aarch64) with cuda-13.0
    cd$TRT_OSSPATH
mkdir -p build &&cd build
cmake .. -DTRT_LIB_DIR=$TRT_LIBPATH -DCMAKE_TOOLCHAIN_FILE=$TRT_OSSPATH/cmake/toolchains/cmake_aarch64_dos_cross.toolchain
make -j$(nproc)
    Example: Native builds on Windows (x86) with cuda-13.0
    cd$TRT_OSSPATH
mkdir -p build
cd -p build
cmake .. -DTRT_LIB_DIR="$env:TRT_LIBPATH" -DTRT_OUT_DIR="$pwd\\out"
msbuild TensorRT.sln /property:Configuration=Release -m:$env:NUMBER_OF_PROCESSORS
    NOTE: The default CUDA version used by CMake is 13.0. To override this, for example to 12.9, append -DCUDA_VERSION=12.9 to the cmake command.
  • Required CMake build arguments are:
    • TRT_LIB_DIR: Path to the TensorRT installation directory containing libraries.
    • TRT_OUT_DIR: Output directory where generated build artifacts will be copied.
  • Optional CMake build arguments:
    • CMAKE_BUILD_TYPE: Specify if binaries generated are for release or debug (contain debug symbols). Values consists of [Release] | Debug
    • CUDA_VERSION: The version of CUDA to target, for example [12.9.9].
    • CUDNN_VERSION: The version of cuDNN to target, for example [8.9].
    • PROTOBUF_VERSION: The version of Protobuf to use, for example [3.20.1]. Note: Changing this will not configure CMake to use a system version of Protobuf, it will configure CMake to download and try building that version.
    • CMAKE_TOOLCHAIN_FILE: The path to a toolchain file for cross compilation.
    • BUILD_PARSERS: Specify if the parsers should be built, for example [ON] | OFF. If turned OFF, CMake will try to find precompiled versions of the parser libraries to use in compiling samples. First in ${TRT_LIB_DIR}, then on the system. If the build type is Debug, then it will prefer debug builds of the libraries before release versions if available.
    • BUILD_PLUGINS: Specify if the plugins should be built, for example [ON] | OFF. If turned OFF, CMake will try to find a precompiled version of the plugin library to use in compiling samples. First in ${TRT_LIB_DIR}, then on the system. If the build type is Debug, then it will prefer debug builds of the libraries before release versions if available.
    • BUILD_SAMPLES: Specify if the samples should be built, for example [ON] | OFF.
    • GPU_ARCHS: GPU (SM) architectures to target. By default we generate CUDA code for all major SMs. Specific SM versions can be specified here as a quoted space-separated list to reduce compilation time and binary size. Table of compute capabilities of NVIDIA GPUs can be found here. Examples: - NVidia A100: -DGPU_ARCHS="80" - RTX 50 series: -DGPU_ARCHS="120" - Multiple SMs: -DGPU_ARCHS="80 120"
    • TRT_PLATFORM_ID: Bare-metal build (unlike containerized cross-compilation). Currently supported options: x86_64 (default).
References
TensorRT Resources
Known Issues
About
NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT.
developer.nvidia.com/tensorrt
Resources
Readme
License
Contributing
Contributing
Activity
Custom properties
Stars
1 star
Watchers
0 watching
Forks
0 forks
Report repository
Releases
Packages
Used by
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Strip utm_, fbclid, gclid, etc. from all links on page
(function() {
var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',
'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',
'ref', 'ref_src', 'source', 'medium', 'campaign'];
function cleanUrl(url) {
try {
var u = new URL(url, window.location.origin);
var changed = false;
trackingParams.forEach(function(p) {
if (u.searchParams.has(p)) {
u.searchParams.delete(p);
changed = true;
}
});
return changed ? u.toString() : url;
} catch (e) {
return url;
}
}
function cleanLinks() {
document.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
cleanLinks();
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1) {
if (node.tagName === 'A') cleanLinks();
node.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '
GitHub - BlueRiverTechnology/TensorRT: NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT. · GitHub
Skip to content
BranchesTags
Repository files navigation
LicenseDocumentationRoadmap
TensorRT Open Source Software
This repository contains the Open Source Software (OSS) components of NVIDIA TensorRT. It includes the sources for TensorRT plugins and ONNX parser, as well as sample applications demonstrating usage and capabilities of the TensorRT platform. These open source software components are a subset of the TensorRT General Availability (GA) release with some extensions and bug-fixes.
Need enterprise support? NVIDIA global support is available for TensorRT with the NVIDIA AI Enterprise software suite. Check out NVIDIA LaunchPad for free access to a set of hands-on labs with TensorRT hosted on NVIDIA infrastructure.
Join the TensorRT and Triton community and stay current on the latest product updates, bug fixes, content, best practices, and more.
Prebuilt TensorRT Python Package
We provide the TensorRT Python package for an easy installation. 

To install:
pip install tensorrt
You can skip the Build section to enjoy TensorRT with Python.
Build
Prerequisites
To build the TensorRT-OSS components, you will first need the following software packages.
TensorRT GA build
  • TensorRT v10.14.1.48

    • Available from direct download links listed below
System Packages
Optional Packages
Downloading TensorRT Build
  1. Download TensorRT OSS
    git clone -b main https://github.com/nvidia/TensorRT TensorRT
cd TensorRT
git submodule update --init --recursive
  2. (Optional - if not using TensorRT container) Specify the TensorRT GA release build path
    If using the TensorRT OSS build container, TensorRT libraries are preinstalled under /usr/lib/x86_64-linux-gnu and you may skip this step.
    Else download and extract the TensorRT GA build from NVIDIA Developer Zone with the direct links below:
    Example: Ubuntu 22.04 on x86-64 with cuda-13.0
    cd~/Downloads
tar -xvzf TensorRT-10.14.1.48.Linux.x86_64-gnu.cuda-13.0.tar.gz
export TRT_LIBPATH=`pwd`/TensorRT-10.14.1.48
    Example: Windows on x86-64 with cuda-12.9
    Expand-Archive-Path TensorRT-10.14.1.48.Windows.win10.cuda-12.9.zip
$env:TRT_LIBPATH="$pwd\TensorRT-10.14.1.48\lib"
Setting Up The Build Environment
For Linux platforms, we recommend that you generate a docker container for building TensorRT OSS as described below. For native builds, please install the prerequisiteSystem Packages.
  1. Generate the TensorRT-OSS build container.
    Example: Ubuntu 24.04 on x86-64 with cuda-13.0 (default)
    ./docker/build.sh --file docker/ubuntu-24.04.Dockerfile --tag tensorrt-ubuntu24.04-cuda13.0
    Example: Rockylinux8 on x86-64 with cuda-13.0
    ./docker/build.sh --file docker/rockylinux8.Dockerfile --tag tensorrt-rockylinux8-cuda13.0
    Example: Ubuntu 24.04 cross-compile for Jetson (aarch64) with cuda-13.0 (JetPack SDK)
    ./docker/build.sh --file docker/ubuntu-cross-aarch64.Dockerfile --tag tensorrt-jetpack-cuda13.0
    Example: Ubuntu 24.04 on aarch64 with cuda-13.0
    ./docker/build.sh --file docker/ubuntu-24.04-aarch64.Dockerfile --tag tensorrt-aarch64-ubuntu24.04-cuda13.0
  2. Launch the TensorRT-OSS build container.
    Example: Ubuntu 24.04 build container
    ./docker/launch.sh --tag tensorrt-ubuntu24.04-cuda13.0 --gpus all
    NOTE:

     1. Use the --tag corresponding to build container generated in Step 1.

     2. NVIDIA Container Toolkit is required for GPU access (running TensorRT applications) inside the build container.

     3. sudo password for Ubuntu build containers is 'nvidia'.

     4. Specify port number using --jupyter <port> for launching Jupyter notebooks.

     5. Write permission to this folder is required as this folder will be mounted inside the docker container for uid:gid of 1000:1000.
Building TensorRT-OSS
  • Generate Makefiles and build
    Example: Linux (x86-64) build with default cuda-13.0
    cd$TRT_OSSPATH
mkdir -p build &&cd build
cmake .. -DTRT_LIB_DIR=$TRT_LIBPATH -DTRT_OUT_DIR=`pwd`/out
make -j$(nproc)
    Example: Linux (aarch64) build with default cuda-13.0
    cd$TRT_OSSPATH
mkdir -p build &&cd build
cmake .. -DTRT_LIB_DIR=$TRT_LIBPATH -DTRT_OUT_DIR=`pwd`/out -DCMAKE_TOOLCHAIN_FILE=$TRT_OSSPATH/cmake/toolchains/cmake_aarch64-native.toolchain
make -j$(nproc)
    Example: Native build on Jetson Thor (aarch64) with cuda-13.0
    cd$TRT_OSSPATH
mkdir -p build &&cd build
cmake .. -DTRT_LIB_DIR=$TRT_LIBPATH -DTRT_OUT_DIR=`pwd`/out -DTRT_PLATFORM_ID=aarch64
CC=/usr/bin/gcc make -j$(nproc)
    NOTE: C compiler must be explicitly specified via CC= for native aarch64 builds of protobuf.
    Example: Ubuntu 24.04 Cross-Compile for Jetson Thor (aarch64) with cuda-13.0 (JetPack)
    cd$TRT_OSSPATH
mkdir -p build &&cd build
cmake .. -DTRT_LIB_DIR=$TRT_LIBPATH -DCMAKE_TOOLCHAIN_FILE=$TRT_OSSPATH/cmake/toolchains/cmake_aarch64_cross.toolchain
make -j$(nproc)
    Example: Ubuntu 24.04 Cross-Compile for DriveOS (aarch64) with cuda-13.0
    cd$TRT_OSSPATH
mkdir -p build &&cd build
cmake .. -DTRT_LIB_DIR=$TRT_LIBPATH -DCMAKE_TOOLCHAIN_FILE=$TRT_OSSPATH/cmake/toolchains/cmake_aarch64_dos_cross.toolchain
make -j$(nproc)
    Example: Native builds on Windows (x86) with cuda-13.0
    cd$TRT_OSSPATH
mkdir -p build
cd -p build
cmake .. -DTRT_LIB_DIR="$env:TRT_LIBPATH" -DTRT_OUT_DIR="$pwd\\out"
msbuild TensorRT.sln /property:Configuration=Release -m:$env:NUMBER_OF_PROCESSORS
    NOTE: The default CUDA version used by CMake is 13.0. To override this, for example to 12.9, append -DCUDA_VERSION=12.9 to the cmake command.
  • Required CMake build arguments are:
    • TRT_LIB_DIR: Path to the TensorRT installation directory containing libraries.
    • TRT_OUT_DIR: Output directory where generated build artifacts will be copied.
  • Optional CMake build arguments:
    • CMAKE_BUILD_TYPE: Specify if binaries generated are for release or debug (contain debug symbols). Values consists of [Release] | Debug
    • CUDA_VERSION: The version of CUDA to target, for example [12.9.9].
    • CUDNN_VERSION: The version of cuDNN to target, for example [8.9].
    • PROTOBUF_VERSION: The version of Protobuf to use, for example [3.20.1]. Note: Changing this will not configure CMake to use a system version of Protobuf, it will configure CMake to download and try building that version.
    • CMAKE_TOOLCHAIN_FILE: The path to a toolchain file for cross compilation.
    • BUILD_PARSERS: Specify if the parsers should be built, for example [ON] | OFF. If turned OFF, CMake will try to find precompiled versions of the parser libraries to use in compiling samples. First in ${TRT_LIB_DIR}, then on the system. If the build type is Debug, then it will prefer debug builds of the libraries before release versions if available.
    • BUILD_PLUGINS: Specify if the plugins should be built, for example [ON] | OFF. If turned OFF, CMake will try to find a precompiled version of the plugin library to use in compiling samples. First in ${TRT_LIB_DIR}, then on the system. If the build type is Debug, then it will prefer debug builds of the libraries before release versions if available.
    • BUILD_SAMPLES: Specify if the samples should be built, for example [ON] | OFF.
    • GPU_ARCHS: GPU (SM) architectures to target. By default we generate CUDA code for all major SMs. Specific SM versions can be specified here as a quoted space-separated list to reduce compilation time and binary size. Table of compute capabilities of NVIDIA GPUs can be found here. Examples: - NVidia A100: -DGPU_ARCHS="80" - RTX 50 series: -DGPU_ARCHS="120" - Multiple SMs: -DGPU_ARCHS="80 120"
    • TRT_PLATFORM_ID: Bare-metal build (unlike containerized cross-compilation). Currently supported options: x86_64 (default).
References
TensorRT Resources
Known Issues
About
NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT.
developer.nvidia.com/tensorrt
Resources
Readme
License
Contributing
Contributing
Activity
Custom properties
Stars
1 star
Watchers
0 watching
Forks
0 forks
Report repository
Releases
Packages
Used by
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Auto-enable theater mode on YouTube
(function() {
function tryTheater() {
var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]');
if (btn && !btn.classList.contains('activated')) {
btn.click();
}
}
// Try immediately
tryTheater();
// Try after navigation (SPA)
var lastUrl = location.href;
setInterval(function() {
if (location.href !== lastUrl) {
lastUrl = location.href;
setTimeout(tryTheater, 500);
}
}, 1000);
// Also try on player load
var observer = new MutationObserver(tryTheater);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
GitHub - BlueRiverTechnology/TensorRT: NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT. · GitHub
Skip to content
BranchesTags
Repository files navigation
LicenseDocumentationRoadmap
TensorRT Open Source Software
This repository contains the Open Source Software (OSS) components of NVIDIA TensorRT. It includes the sources for TensorRT plugins and ONNX parser, as well as sample applications demonstrating usage and capabilities of the TensorRT platform. These open source software components are a subset of the TensorRT General Availability (GA) release with some extensions and bug-fixes.
Need enterprise support? NVIDIA global support is available for TensorRT with the NVIDIA AI Enterprise software suite. Check out NVIDIA LaunchPad for free access to a set of hands-on labs with TensorRT hosted on NVIDIA infrastructure.
Join the TensorRT and Triton community and stay current on the latest product updates, bug fixes, content, best practices, and more.
Prebuilt TensorRT Python Package
We provide the TensorRT Python package for an easy installation. 

To install:
pip install tensorrt
You can skip the Build section to enjoy TensorRT with Python.
Build
Prerequisites
To build the TensorRT-OSS components, you will first need the following software packages.
TensorRT GA build
  • TensorRT v10.14.1.48

    • Available from direct download links listed below
System Packages
Optional Packages
Downloading TensorRT Build
  1. Download TensorRT OSS
    git clone -b main https://github.com/nvidia/TensorRT TensorRT
cd TensorRT
git submodule update --init --recursive
  2. (Optional - if not using TensorRT container) Specify the TensorRT GA release build path
    If using the TensorRT OSS build container, TensorRT libraries are preinstalled under /usr/lib/x86_64-linux-gnu and you may skip this step.
    Else download and extract the TensorRT GA build from NVIDIA Developer Zone with the direct links below:
    Example: Ubuntu 22.04 on x86-64 with cuda-13.0
    cd~/Downloads
tar -xvzf TensorRT-10.14.1.48.Linux.x86_64-gnu.cuda-13.0.tar.gz
export TRT_LIBPATH=`pwd`/TensorRT-10.14.1.48
    Example: Windows on x86-64 with cuda-12.9
    Expand-Archive-Path TensorRT-10.14.1.48.Windows.win10.cuda-12.9.zip
$env:TRT_LIBPATH="$pwd\TensorRT-10.14.1.48\lib"
Setting Up The Build Environment
For Linux platforms, we recommend that you generate a docker container for building TensorRT OSS as described below. For native builds, please install the prerequisiteSystem Packages.
  1. Generate the TensorRT-OSS build container.
    Example: Ubuntu 24.04 on x86-64 with cuda-13.0 (default)
    ./docker/build.sh --file docker/ubuntu-24.04.Dockerfile --tag tensorrt-ubuntu24.04-cuda13.0
    Example: Rockylinux8 on x86-64 with cuda-13.0
    ./docker/build.sh --file docker/rockylinux8.Dockerfile --tag tensorrt-rockylinux8-cuda13.0
    Example: Ubuntu 24.04 cross-compile for Jetson (aarch64) with cuda-13.0 (JetPack SDK)
    ./docker/build.sh --file docker/ubuntu-cross-aarch64.Dockerfile --tag tensorrt-jetpack-cuda13.0
    Example: Ubuntu 24.04 on aarch64 with cuda-13.0
    ./docker/build.sh --file docker/ubuntu-24.04-aarch64.Dockerfile --tag tensorrt-aarch64-ubuntu24.04-cuda13.0
  2. Launch the TensorRT-OSS build container.
    Example: Ubuntu 24.04 build container
    ./docker/launch.sh --tag tensorrt-ubuntu24.04-cuda13.0 --gpus all
    NOTE:

     1. Use the --tag corresponding to build container generated in Step 1.

     2. NVIDIA Container Toolkit is required for GPU access (running TensorRT applications) inside the build container.

     3. sudo password for Ubuntu build containers is 'nvidia'.

     4. Specify port number using --jupyter <port> for launching Jupyter notebooks.

     5. Write permission to this folder is required as this folder will be mounted inside the docker container for uid:gid of 1000:1000.
Building TensorRT-OSS
  • Generate Makefiles and build
    Example: Linux (x86-64) build with default cuda-13.0
    cd$TRT_OSSPATH
mkdir -p build &&cd build
cmake .. -DTRT_LIB_DIR=$TRT_LIBPATH -DTRT_OUT_DIR=`pwd`/out
make -j$(nproc)
    Example: Linux (aarch64) build with default cuda-13.0
    cd$TRT_OSSPATH
mkdir -p build &&cd build
cmake .. -DTRT_LIB_DIR=$TRT_LIBPATH -DTRT_OUT_DIR=`pwd`/out -DCMAKE_TOOLCHAIN_FILE=$TRT_OSSPATH/cmake/toolchains/cmake_aarch64-native.toolchain
make -j$(nproc)
    Example: Native build on Jetson Thor (aarch64) with cuda-13.0
    cd$TRT_OSSPATH
mkdir -p build &&cd build
cmake .. -DTRT_LIB_DIR=$TRT_LIBPATH -DTRT_OUT_DIR=`pwd`/out -DTRT_PLATFORM_ID=aarch64
CC=/usr/bin/gcc make -j$(nproc)
    NOTE: C compiler must be explicitly specified via CC= for native aarch64 builds of protobuf.
    Example: Ubuntu 24.04 Cross-Compile for Jetson Thor (aarch64) with cuda-13.0 (JetPack)
    cd$TRT_OSSPATH
mkdir -p build &&cd build
cmake .. -DTRT_LIB_DIR=$TRT_LIBPATH -DCMAKE_TOOLCHAIN_FILE=$TRT_OSSPATH/cmake/toolchains/cmake_aarch64_cross.toolchain
make -j$(nproc)
    Example: Ubuntu 24.04 Cross-Compile for DriveOS (aarch64) with cuda-13.0
    cd$TRT_OSSPATH
mkdir -p build &&cd build
cmake .. -DTRT_LIB_DIR=$TRT_LIBPATH -DCMAKE_TOOLCHAIN_FILE=$TRT_OSSPATH/cmake/toolchains/cmake_aarch64_dos_cross.toolchain
make -j$(nproc)
    Example: Native builds on Windows (x86) with cuda-13.0
    cd$TRT_OSSPATH
mkdir -p build
cd -p build
cmake .. -DTRT_LIB_DIR="$env:TRT_LIBPATH" -DTRT_OUT_DIR="$pwd\\out"
msbuild TensorRT.sln /property:Configuration=Release -m:$env:NUMBER_OF_PROCESSORS
    NOTE: The default CUDA version used by CMake is 13.0. To override this, for example to 12.9, append -DCUDA_VERSION=12.9 to the cmake command.
  • Required CMake build arguments are:
    • TRT_LIB_DIR: Path to the TensorRT installation directory containing libraries.
    • TRT_OUT_DIR: Output directory where generated build artifacts will be copied.
  • Optional CMake build arguments:
    • CMAKE_BUILD_TYPE: Specify if binaries generated are for release or debug (contain debug symbols). Values consists of [Release] | Debug
    • CUDA_VERSION: The version of CUDA to target, for example [12.9.9].
    • CUDNN_VERSION: The version of cuDNN to target, for example [8.9].
    • PROTOBUF_VERSION: The version of Protobuf to use, for example [3.20.1]. Note: Changing this will not configure CMake to use a system version of Protobuf, it will configure CMake to download and try building that version.
    • CMAKE_TOOLCHAIN_FILE: The path to a toolchain file for cross compilation.
    • BUILD_PARSERS: Specify if the parsers should be built, for example [ON] | OFF. If turned OFF, CMake will try to find precompiled versions of the parser libraries to use in compiling samples. First in ${TRT_LIB_DIR}, then on the system. If the build type is Debug, then it will prefer debug builds of the libraries before release versions if available.
    • BUILD_PLUGINS: Specify if the plugins should be built, for example [ON] | OFF. If turned OFF, CMake will try to find a precompiled version of the plugin library to use in compiling samples. First in ${TRT_LIB_DIR}, then on the system. If the build type is Debug, then it will prefer debug builds of the libraries before release versions if available.
    • BUILD_SAMPLES: Specify if the samples should be built, for example [ON] | OFF.
    • GPU_ARCHS: GPU (SM) architectures to target. By default we generate CUDA code for all major SMs. Specific SM versions can be specified here as a quoted space-separated list to reduce compilation time and binary size. Table of compute capabilities of NVIDIA GPUs can be found here. Examples: - NVidia A100: -DGPU_ARCHS="80" - RTX 50 series: -DGPU_ARCHS="120" - Multiple SMs: -DGPU_ARCHS="80 120"
    • TRT_PLATFORM_ID: Bare-metal build (unlike containerized cross-compilation). Currently supported options: x86_64 (default).
References
TensorRT Resources
Known Issues
About
NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT.
developer.nvidia.com/tensorrt
Resources
Readme
License
Contributing
Contributing
Activity
Custom properties
Stars
1 star
Watchers
0 watching
Forks
0 forks
Report repository
Releases
Packages
Used by
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Remove or un-stick sticky/fixed headers that block content
(function() {
function unstick() {
document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) {
if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {
el.style.position = 'static';
el.style.top = 'auto';
el.style.zIndex = 'auto';
}
});
}
unstick();
var observer = new MutationObserver(unstick);
observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });
})();
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
GitHub - BlueRiverTechnology/TensorRT: NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT. · GitHub
Skip to content
BranchesTags
Repository files navigation
LicenseDocumentationRoadmap
TensorRT Open Source Software
This repository contains the Open Source Software (OSS) components of NVIDIA TensorRT. It includes the sources for TensorRT plugins and ONNX parser, as well as sample applications demonstrating usage and capabilities of the TensorRT platform. These open source software components are a subset of the TensorRT General Availability (GA) release with some extensions and bug-fixes.
Need enterprise support? NVIDIA global support is available for TensorRT with the NVIDIA AI Enterprise software suite. Check out NVIDIA LaunchPad for free access to a set of hands-on labs with TensorRT hosted on NVIDIA infrastructure.
Join the TensorRT and Triton community and stay current on the latest product updates, bug fixes, content, best practices, and more.
Prebuilt TensorRT Python Package
We provide the TensorRT Python package for an easy installation. 

To install:
pip install tensorrt
You can skip the Build section to enjoy TensorRT with Python.
Build
Prerequisites
To build the TensorRT-OSS components, you will first need the following software packages.
TensorRT GA build
  • TensorRT v10.14.1.48

    • Available from direct download links listed below
System Packages
Optional Packages
Downloading TensorRT Build
  1. Download TensorRT OSS
    git clone -b main https://github.com/nvidia/TensorRT TensorRT
cd TensorRT
git submodule update --init --recursive
  2. (Optional - if not using TensorRT container) Specify the TensorRT GA release build path
    If using the TensorRT OSS build container, TensorRT libraries are preinstalled under /usr/lib/x86_64-linux-gnu and you may skip this step.
    Else download and extract the TensorRT GA build from NVIDIA Developer Zone with the direct links below:
    Example: Ubuntu 22.04 on x86-64 with cuda-13.0
    cd~/Downloads
tar -xvzf TensorRT-10.14.1.48.Linux.x86_64-gnu.cuda-13.0.tar.gz
export TRT_LIBPATH=`pwd`/TensorRT-10.14.1.48
    Example: Windows on x86-64 with cuda-12.9
    Expand-Archive-Path TensorRT-10.14.1.48.Windows.win10.cuda-12.9.zip
$env:TRT_LIBPATH="$pwd\TensorRT-10.14.1.48\lib"
Setting Up The Build Environment
For Linux platforms, we recommend that you generate a docker container for building TensorRT OSS as described below. For native builds, please install the prerequisiteSystem Packages.
  1. Generate the TensorRT-OSS build container.
    Example: Ubuntu 24.04 on x86-64 with cuda-13.0 (default)
    ./docker/build.sh --file docker/ubuntu-24.04.Dockerfile --tag tensorrt-ubuntu24.04-cuda13.0
    Example: Rockylinux8 on x86-64 with cuda-13.0
    ./docker/build.sh --file docker/rockylinux8.Dockerfile --tag tensorrt-rockylinux8-cuda13.0
    Example: Ubuntu 24.04 cross-compile for Jetson (aarch64) with cuda-13.0 (JetPack SDK)
    ./docker/build.sh --file docker/ubuntu-cross-aarch64.Dockerfile --tag tensorrt-jetpack-cuda13.0
    Example: Ubuntu 24.04 on aarch64 with cuda-13.0
    ./docker/build.sh --file docker/ubuntu-24.04-aarch64.Dockerfile --tag tensorrt-aarch64-ubuntu24.04-cuda13.0
  2. Launch the TensorRT-OSS build container.
    Example: Ubuntu 24.04 build container
    ./docker/launch.sh --tag tensorrt-ubuntu24.04-cuda13.0 --gpus all
    NOTE:

     1. Use the --tag corresponding to build container generated in Step 1.

     2. NVIDIA Container Toolkit is required for GPU access (running TensorRT applications) inside the build container.

     3. sudo password for Ubuntu build containers is 'nvidia'.

     4. Specify port number using --jupyter <port> for launching Jupyter notebooks.

     5. Write permission to this folder is required as this folder will be mounted inside the docker container for uid:gid of 1000:1000.
Building TensorRT-OSS
  • Generate Makefiles and build
    Example: Linux (x86-64) build with default cuda-13.0
    cd$TRT_OSSPATH
mkdir -p build &&cd build
cmake .. -DTRT_LIB_DIR=$TRT_LIBPATH -DTRT_OUT_DIR=`pwd`/out
make -j$(nproc)
    Example: Linux (aarch64) build with default cuda-13.0
    cd$TRT_OSSPATH
mkdir -p build &&cd build
cmake .. -DTRT_LIB_DIR=$TRT_LIBPATH -DTRT_OUT_DIR=`pwd`/out -DCMAKE_TOOLCHAIN_FILE=$TRT_OSSPATH/cmake/toolchains/cmake_aarch64-native.toolchain
make -j$(nproc)
    Example: Native build on Jetson Thor (aarch64) with cuda-13.0
    cd$TRT_OSSPATH
mkdir -p build &&cd build
cmake .. -DTRT_LIB_DIR=$TRT_LIBPATH -DTRT_OUT_DIR=`pwd`/out -DTRT_PLATFORM_ID=aarch64
CC=/usr/bin/gcc make -j$(nproc)
    NOTE: C compiler must be explicitly specified via CC= for native aarch64 builds of protobuf.
    Example: Ubuntu 24.04 Cross-Compile for Jetson Thor (aarch64) with cuda-13.0 (JetPack)
    cd$TRT_OSSPATH
mkdir -p build &&cd build
cmake .. -DTRT_LIB_DIR=$TRT_LIBPATH -DCMAKE_TOOLCHAIN_FILE=$TRT_OSSPATH/cmake/toolchains/cmake_aarch64_cross.toolchain
make -j$(nproc)
    Example: Ubuntu 24.04 Cross-Compile for DriveOS (aarch64) with cuda-13.0
    cd$TRT_OSSPATH
mkdir -p build &&cd build
cmake .. -DTRT_LIB_DIR=$TRT_LIBPATH -DCMAKE_TOOLCHAIN_FILE=$TRT_OSSPATH/cmake/toolchains/cmake_aarch64_dos_cross.toolchain
make -j$(nproc)
    Example: Native builds on Windows (x86) with cuda-13.0
    cd$TRT_OSSPATH
mkdir -p build
cd -p build
cmake .. -DTRT_LIB_DIR="$env:TRT_LIBPATH" -DTRT_OUT_DIR="$pwd\\out"
msbuild TensorRT.sln /property:Configuration=Release -m:$env:NUMBER_OF_PROCESSORS
    NOTE: The default CUDA version used by CMake is 13.0. To override this, for example to 12.9, append -DCUDA_VERSION=12.9 to the cmake command.
  • Required CMake build arguments are:
    • TRT_LIB_DIR: Path to the TensorRT installation directory containing libraries.
    • TRT_OUT_DIR: Output directory where generated build artifacts will be copied.
  • Optional CMake build arguments:
    • CMAKE_BUILD_TYPE: Specify if binaries generated are for release or debug (contain debug symbols). Values consists of [Release] | Debug
    • CUDA_VERSION: The version of CUDA to target, for example [12.9.9].
    • CUDNN_VERSION: The version of cuDNN to target, for example [8.9].
    • PROTOBUF_VERSION: The version of Protobuf to use, for example [3.20.1]. Note: Changing this will not configure CMake to use a system version of Protobuf, it will configure CMake to download and try building that version.
    • CMAKE_TOOLCHAIN_FILE: The path to a toolchain file for cross compilation.
    • BUILD_PARSERS: Specify if the parsers should be built, for example [ON] | OFF. If turned OFF, CMake will try to find precompiled versions of the parser libraries to use in compiling samples. First in ${TRT_LIB_DIR}, then on the system. If the build type is Debug, then it will prefer debug builds of the libraries before release versions if available.
    • BUILD_PLUGINS: Specify if the plugins should be built, for example [ON] | OFF. If turned OFF, CMake will try to find a precompiled version of the plugin library to use in compiling samples. First in ${TRT_LIB_DIR}, then on the system. If the build type is Debug, then it will prefer debug builds of the libraries before release versions if available.
    • BUILD_SAMPLES: Specify if the samples should be built, for example [ON] | OFF.
    • GPU_ARCHS: GPU (SM) architectures to target. By default we generate CUDA code for all major SMs. Specific SM versions can be specified here as a quoted space-separated list to reduce compilation time and binary size. Table of compute capabilities of NVIDIA GPUs can be found here. Examples: - NVidia A100: -DGPU_ARCHS="80" - RTX 50 series: -DGPU_ARCHS="120" - Multiple SMs: -DGPU_ARCHS="80 120"
    • TRT_PLATFORM_ID: Bare-metal build (unlike containerized cross-compilation). Currently supported options: x86_64 (default).
References
TensorRT Resources
Known Issues
About
NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT.
developer.nvidia.com/tensorrt
Resources
Readme
License
Contributing
Contributing
Activity
Custom properties
Stars
1 star
Watchers
0 watching
Forks
0 forks
Report repository
Releases
Packages
Used by
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Universal Dark Mode - works on any site
(function() {
var enabled = true;
function applyDarkMode() {
if (!enabled) return;
// Create style element if it doesn't exist
var style = document.getElementById('universal-dark-mode-style');
if (!style) {
style = document.createElement('style');
style.id = 'universal-dark-mode-style';
document.head.appendChild(style);
}
// Dark mode CSS - inverts colors but preserves images/video
style.textContent = '
/* Invert everything except media */
html {
filter: invert(1) hue-rotate(180deg) !important;
background: #1a1a2e !important;
}
/* Restore images, videos, iframes, canvas */
img, video, iframe, canvas, svg, picture, [style*="background-image"] {
filter: invert(1) hue-rotate(180deg) !important;
}
/* Preserve specific elements that should not be inverted */
.no-dark-mode, .no-dark-mode *,
[data-theme="light"], [data-theme="light"],
.ace_editor, .ace_editor *,
.CodeMirror, .CodeMirror *,
.monaco-editor, .monaco-editor *,
.markdown-body pre, .markdown-body pre *,
.highlight, .highlight *,
pre code, pre code * {
filter: none !important;
}
/* Fix common UI elements */
.modal, .popup, .dropdown-menu, .tooltip, .popover {
filter: invert(1) hue-rotate(180deg) !important;
background: #2d2d44 !important;
border-color: #444 !important;
}
/* Scrollbars */
::-webkit-scrollbar { background: #1a1a2e !important; }
::-webkit-scrollbar-thumb { background: #444 !important; }
::-webkit-scrollbar-thumb:hover { background: #555 !important; }
/* Selection */
::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }
::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }
';
}
function removeDarkMode() {
var style = document.getElementById('universal-dark-mode-style');
if (style) style.remove();
}
// Toggle with Alt+Shift+D
document.addEventListener('keydown', function(e) {
if (e.altKey && e.shiftKey && e.key === 'D') {
e.preventDefault();
enabled = !enabled;
if (enabled) {
applyDarkMode();
console.log('[Universal Dark Mode] Enabled');
} else {
removeDarkMode();
console.log('[Universal Dark Mode] Disabled');
}
}
});
// Apply on load
applyDarkMode();
// Re-apply on dynamic content
var observer = new MutationObserver(function(mutations) {
if (enabled && !document.getElementById('universal-dark-mode-style')) {
applyDarkMode();
}
});
observer.observe(document.head, { childList: true });
console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');
})();
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();
GitHub - BlueRiverTechnology/TensorRT: NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT. · GitHub
Skip to content
BranchesTags
Repository files navigation
LicenseDocumentationRoadmap
TensorRT Open Source Software
This repository contains the Open Source Software (OSS) components of NVIDIA TensorRT. It includes the sources for TensorRT plugins and ONNX parser, as well as sample applications demonstrating usage and capabilities of the TensorRT platform. These open source software components are a subset of the TensorRT General Availability (GA) release with some extensions and bug-fixes.
Need enterprise support? NVIDIA global support is available for TensorRT with the NVIDIA AI Enterprise software suite. Check out NVIDIA LaunchPad for free access to a set of hands-on labs with TensorRT hosted on NVIDIA infrastructure.
Join the TensorRT and Triton community and stay current on the latest product updates, bug fixes, content, best practices, and more.
Prebuilt TensorRT Python Package
We provide the TensorRT Python package for an easy installation. 

To install:
pip install tensorrt
You can skip the Build section to enjoy TensorRT with Python.
Build
Prerequisites
To build the TensorRT-OSS components, you will first need the following software packages.
TensorRT GA build
  • TensorRT v10.14.1.48

    • Available from direct download links listed below
System Packages
Optional Packages
Downloading TensorRT Build
  1. Download TensorRT OSS
    git clone -b main https://github.com/nvidia/TensorRT TensorRT
cd TensorRT
git submodule update --init --recursive
  2. (Optional - if not using TensorRT container) Specify the TensorRT GA release build path
    If using the TensorRT OSS build container, TensorRT libraries are preinstalled under /usr/lib/x86_64-linux-gnu and you may skip this step.
    Else download and extract the TensorRT GA build from NVIDIA Developer Zone with the direct links below:
    Example: Ubuntu 22.04 on x86-64 with cuda-13.0
    cd~/Downloads
tar -xvzf TensorRT-10.14.1.48.Linux.x86_64-gnu.cuda-13.0.tar.gz
export TRT_LIBPATH=`pwd`/TensorRT-10.14.1.48
    Example: Windows on x86-64 with cuda-12.9
    Expand-Archive-Path TensorRT-10.14.1.48.Windows.win10.cuda-12.9.zip
$env:TRT_LIBPATH="$pwd\TensorRT-10.14.1.48\lib"
Setting Up The Build Environment
For Linux platforms, we recommend that you generate a docker container for building TensorRT OSS as described below. For native builds, please install the prerequisiteSystem Packages.
  1. Generate the TensorRT-OSS build container.
    Example: Ubuntu 24.04 on x86-64 with cuda-13.0 (default)
    ./docker/build.sh --file docker/ubuntu-24.04.Dockerfile --tag tensorrt-ubuntu24.04-cuda13.0
    Example: Rockylinux8 on x86-64 with cuda-13.0
    ./docker/build.sh --file docker/rockylinux8.Dockerfile --tag tensorrt-rockylinux8-cuda13.0
    Example: Ubuntu 24.04 cross-compile for Jetson (aarch64) with cuda-13.0 (JetPack SDK)
    ./docker/build.sh --file docker/ubuntu-cross-aarch64.Dockerfile --tag tensorrt-jetpack-cuda13.0
    Example: Ubuntu 24.04 on aarch64 with cuda-13.0
    ./docker/build.sh --file docker/ubuntu-24.04-aarch64.Dockerfile --tag tensorrt-aarch64-ubuntu24.04-cuda13.0
  2. Launch the TensorRT-OSS build container.
    Example: Ubuntu 24.04 build container
    ./docker/launch.sh --tag tensorrt-ubuntu24.04-cuda13.0 --gpus all
    NOTE:

     1. Use the --tag corresponding to build container generated in Step 1.

     2. NVIDIA Container Toolkit is required for GPU access (running TensorRT applications) inside the build container.

     3. sudo password for Ubuntu build containers is 'nvidia'.

     4. Specify port number using --jupyter <port> for launching Jupyter notebooks.

     5. Write permission to this folder is required as this folder will be mounted inside the docker container for uid:gid of 1000:1000.
Building TensorRT-OSS
  • Generate Makefiles and build
    Example: Linux (x86-64) build with default cuda-13.0
    cd$TRT_OSSPATH
mkdir -p build &&cd build
cmake .. -DTRT_LIB_DIR=$TRT_LIBPATH -DTRT_OUT_DIR=`pwd`/out
make -j$(nproc)
    Example: Linux (aarch64) build with default cuda-13.0
    cd$TRT_OSSPATH
mkdir -p build &&cd build
cmake .. -DTRT_LIB_DIR=$TRT_LIBPATH -DTRT_OUT_DIR=`pwd`/out -DCMAKE_TOOLCHAIN_FILE=$TRT_OSSPATH/cmake/toolchains/cmake_aarch64-native.toolchain
make -j$(nproc)
    Example: Native build on Jetson Thor (aarch64) with cuda-13.0
    cd$TRT_OSSPATH
mkdir -p build &&cd build
cmake .. -DTRT_LIB_DIR=$TRT_LIBPATH -DTRT_OUT_DIR=`pwd`/out -DTRT_PLATFORM_ID=aarch64
CC=/usr/bin/gcc make -j$(nproc)
    NOTE: C compiler must be explicitly specified via CC= for native aarch64 builds of protobuf.
    Example: Ubuntu 24.04 Cross-Compile for Jetson Thor (aarch64) with cuda-13.0 (JetPack)
    cd$TRT_OSSPATH
mkdir -p build &&cd build
cmake .. -DTRT_LIB_DIR=$TRT_LIBPATH -DCMAKE_TOOLCHAIN_FILE=$TRT_OSSPATH/cmake/toolchains/cmake_aarch64_cross.toolchain
make -j$(nproc)
    Example: Ubuntu 24.04 Cross-Compile for DriveOS (aarch64) with cuda-13.0
    cd$TRT_OSSPATH
mkdir -p build &&cd build
cmake .. -DTRT_LIB_DIR=$TRT_LIBPATH -DCMAKE_TOOLCHAIN_FILE=$TRT_OSSPATH/cmake/toolchains/cmake_aarch64_dos_cross.toolchain
make -j$(nproc)
    Example: Native builds on Windows (x86) with cuda-13.0
    cd$TRT_OSSPATH
mkdir -p build
cd -p build
cmake .. -DTRT_LIB_DIR="$env:TRT_LIBPATH" -DTRT_OUT_DIR="$pwd\\out"
msbuild TensorRT.sln /property:Configuration=Release -m:$env:NUMBER_OF_PROCESSORS
    NOTE: The default CUDA version used by CMake is 13.0. To override this, for example to 12.9, append -DCUDA_VERSION=12.9 to the cmake command.
  • Required CMake build arguments are:
    • TRT_LIB_DIR: Path to the TensorRT installation directory containing libraries.
    • TRT_OUT_DIR: Output directory where generated build artifacts will be copied.
  • Optional CMake build arguments:
    • CMAKE_BUILD_TYPE: Specify if binaries generated are for release or debug (contain debug symbols). Values consists of [Release] | Debug
    • CUDA_VERSION: The version of CUDA to target, for example [12.9.9].
    • CUDNN_VERSION: The version of cuDNN to target, for example [8.9].
    • PROTOBUF_VERSION: The version of Protobuf to use, for example [3.20.1]. Note: Changing this will not configure CMake to use a system version of Protobuf, it will configure CMake to download and try building that version.
    • CMAKE_TOOLCHAIN_FILE: The path to a toolchain file for cross compilation.
    • BUILD_PARSERS: Specify if the parsers should be built, for example [ON] | OFF. If turned OFF, CMake will try to find precompiled versions of the parser libraries to use in compiling samples. First in ${TRT_LIB_DIR}, then on the system. If the build type is Debug, then it will prefer debug builds of the libraries before release versions if available.
    • BUILD_PLUGINS: Specify if the plugins should be built, for example [ON] | OFF. If turned OFF, CMake will try to find a precompiled version of the plugin library to use in compiling samples. First in ${TRT_LIB_DIR}, then on the system. If the build type is Debug, then it will prefer debug builds of the libraries before release versions if available.
    • BUILD_SAMPLES: Specify if the samples should be built, for example [ON] | OFF.
    • GPU_ARCHS: GPU (SM) architectures to target. By default we generate CUDA code for all major SMs. Specific SM versions can be specified here as a quoted space-separated list to reduce compilation time and binary size. Table of compute capabilities of NVIDIA GPUs can be found here. Examples: - NVidia A100: -DGPU_ARCHS="80" - RTX 50 series: -DGPU_ARCHS="120" - Multiple SMs: -DGPU_ARCHS="80 120"
    • TRT_PLATFORM_ID: Bare-metal build (unlike containerized cross-compilation). Currently supported options: x86_64 (default).
References
TensorRT Resources
Known Issues
About
NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT.
developer.nvidia.com/tensorrt
Resources
Readme
License
Contributing
Contributing
Activity
Custom properties
Stars
1 star
Watchers
0 watching
Forks
0 forks
Report repository
Releases
Packages
Used by
Contributors
Languages