Added flux demo

[cehongwang]

Description

Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.

Fixes # (issue)

Type of change

Please delete options that are not relevant and/or add your own.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

Checklist:

• My code follows the style guidelines of this project (You can use the linters)
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas and hacks
• I have made corresponding changes to the documentation
• I have added tests to verify my fix or my feature
• New and existing unit tests pass locally with my changes
• I have added the relevant labels to my PR in so that relevant reviewers are notified

[narendasan]

Can the app display the inference time, might be nice to have some stats rendered live as you generate

[peri044]

use them from examples

[peri044]

I think we should avoid copying the whole model scripts for measuring perf. Try using the sys.path approach and importing the model and just a perf loop. something like

importsysimportossys.path.append(torchtrt_root+"examples/dynamo/apps")
fromflux_demoimport*model=<insertFLUXmodel (fp16orfp8) >results=measure_flux_perf(.... )

[github-actions]

There are some changes that do not conform to Python style guidelines:

--- /home/runner/work/TensorRT/TensorRT/py/torch_tensorrt/dynamo/conversion/impl/quantize.py	2025-06-12 23:00:13.565607+00:00+++ /home/runner/work/TensorRT/TensorRT/py/torch_tensorrt/dynamo/conversion/impl/quantize.py	2025-06-12 23:00:38.626299+00:00@@ -73,11 +73,10 @@
max_bound = 127
elif num_bits == 8 and exponent_bits == 4:
dtype = trt.DataType.FP8
max_bound = 448
-
axis = None
# int8 weight quantization is per-channel quantization(it can have one or multiple amax values)
if dtype == trt.DataType.INT8 and amax.numel() > 1:
# if the amax has more than one element, calculate the axis, otherwise axis value will be ignored
amax_init_shape = amax.shape
--- /home/runner/work/TensorRT/TensorRT/py/torch_tensorrt/dynamo/lowering/passes/constant_folding.py	2025-06-12 23:00:13.567607+00:00+++ /home/runner/work/TensorRT/TensorRT/py/torch_tensorrt/dynamo/lowering/passes/constant_folding.py	2025-06-12 23:00:39.100847+00:00@@ -98,16 +98,17 @@
class _TorchTensorRTConstantFolder(ConstantFolder): # type: ignore[misc]
def __init__(self, *args: Any, **kwargs: Any) -> None:
super().__init__(*args, **kwargs)
def is_impure(self, node: torch.fx.node.Node) -> bool:
- # Set of known quantization ops to be excluded from constant folding. + # Set of known quantization ops to be excluded from constant folding.
# Currently, we exclude all quantization ops coming from modelopt library.
quantization_ops = {}
try:
- # modelopt import ensures torch.ops.tensorrt.quantize_op.default is registered + # modelopt import ensures torch.ops.tensorrt.quantize_op.default is registered
import modelopt.torch.quantization as mtq
+
assert torch.ops.tensorrt.quantize_op.default
quantization_ops.add(torch.ops.tensorrt.quantize_op.default)
except Exception as e:
pass
if quantization_ops and node.target in quantization_ops:

[github-actions]

There are some changes that do not conform to Python style guidelines:

--- /home/runner/work/TensorRT/TensorRT/py/torch_tensorrt/dynamo/conversion/impl/quantize.py	2025-06-13 20:58:57.556250+00:00+++ /home/runner/work/TensorRT/TensorRT/py/torch_tensorrt/dynamo/conversion/impl/quantize.py	2025-06-13 20:59:27.374709+00:00@@ -73,11 +73,10 @@
max_bound = 127
elif num_bits == 8 and exponent_bits == 4:
dtype = trt.DataType.FP8
max_bound = 448
-
axis = None
# int8 weight quantization is per-channel quantization(it can have one or multiple amax values)
if dtype == trt.DataType.INT8 and amax.numel() > 1:
# if the amax has more than one element, calculate the axis, otherwise axis value will be ignored
amax_init_shape = amax.shape