onnxruntime optimizers fails

[xadupre]

Describe the issue

The attached model succeeds when onnxruntime optimization are disabled, and fails when all of them are.

  File "onnxruntime/capi/onnxruntime_inference_collection.py", line 491, in _create_inference_session
    sess.initialize_session(providers, provider_options, disabled_optimizers)
onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL : Node (MatMulBnFusion_Gemm) Op (Gemm) [ShapeInferenceError] First input does not have rank 2

model_levit_128-onnx_dynamo-ir-d0rt0.zip

To reproduce

import onnxruntime
onnxruntime.InferenceSession("<model.onnx>", providers=["CUDAExecutionProvider", "CPUExecutionProvider"])

Urgency

No response

Platform

Linux

OS Version

ubuntu-22.04

ONNX Runtime Installation

Built from Source

ONNX Runtime Version or Commit ID

7880342

ONNX Runtime API

Python

Architecture

X64

Execution Provider

CUDA

Execution Provider Library Version

CUDA 11.8

[xadupre]

The issue is here: https://github.com/microsoft/onnxruntime/blob/main/onnxruntime/core/optimizer/matmul_bn_fusion.cc#L207. A Gemm operator is introduced to fuse Add(MatMul(X,Y), Z) into Gemm(X,Y,Z). In my case, it fails because rank(X) == 3 and rank(Y) == 2 which is not allowed by this kernel.