[Performance] version 1.17.1 causes performance regression over 1.16.3 both with TRT EP and Cuda EP on Faster-RCNN model inference about onnxruntime HOT 3 OPEN

how can we make progress here ?

from onnxruntime.

I tried on Turing and ADA GPUs , I tried with different version of Cudnn , 1.17.1 inference on the faster-rcnn is always slower by 30% at least.
Note that TRT used is same between the 2 ONNXRT versions used
I guess I need to build a demonstrator I can share, but any guidance about determining which ONNX instruction is talking longer in 1.17.1 with my production model ?

from onnxruntime.

did not succeed in executing inference of torchvision fasterrcnn_resnet50_fpn opensource model with onnxruntime TensorRT ExecutionProvider (I used torchvision https://github.com/pytorch/vision/blob/main/test/test_onnx.py TestONNXExporter::test_faster_rcnn)

succeeded though in running with onnxruntime CUDAExecution provider but not with TensorRT Execution provider .

Torchvision test wants to fallback to CUDA Execution Provider because fasterRCNN exported model missed the inferred dimension that TensorRT requires.

Unfortunately onnxruntime/tools/symbolic_shape_infer.py tool also crashes on converting exported onnx fasterRCNN model for tensorRT (I needed to run this tool too on my moel to run it on TensorRT Execution Provider)

I am stuck in providing a non IP demonstration code of faster_rcnn for onnxrtuntiome TensorRT Execution Provider and hence help define which onnx operator regresses in Onnxrt 1.17.1 versus 1.16.3

Need instructions for determining myself the onnx operator that regresses on my model and indicate to onnxrt team: any help ?

from onnxruntime.