xuanandsix / clrnet-onnxruntime-and-tensorrt-demo Goto Github PK

I can't make the trtexec.exe

ERROR:[optimizer.cpp::computeCosts::2011] Error Code 10: Internal Error (Could not find any implementation for node {ForeignNode[538...Reshape_295]}.)
when I export the onnx model to tensorrt file with trtexec, this error occured.
tensorrt version : 8.4.0.6 download from: https://developer.nvidia.com/compute/machine-learning/tensorrt/secure/8.4.0/tars/tensorrt-8.4.0.6.linux.x86_64-gnu.cuda-11.6.cudnn8.3.tar.gz
comments like this: ./trtexec --onnx=tusimple_r18_opt.onnx --saveEngine=tusimple_r18_opt.engine --workspace=4096

An error occurred during trt operation
I didn't have a problem with the onnx reasoning. And successfully converted onnx to trt's.engine format, as shown below. I think the conversion process should be fine.

But when I run trt, this error occurs
Traceback (most recent call last): File "/snap/pycharm-educational/57/plugins/python-ce/helpers/pydev/pydevd.py", line 1496, in _exec pydev_imports.execfile(file, globals, locals) # execute the script File "/snap/pycharm-educational/57/plugins/python-ce/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile exec(compile(contents+"\n", file, 'exec'), glob, loc) File "/home/ysy/project/lane_detect/CLRNet-onnxruntime-and-tensorrt-demo-main/demo_trt.py", line 339, in <module> output = isnet.forward(image) File "/home/ysy/project/lane_detect/CLRNet-onnxruntime-and-tensorrt-demo-main/demo_trt.py", line 324, in forward self.context.execute_v2(self.allocations) AttributeError: 'NoneType' object has no attribute 'execute_v2'
I think there was an error in the engine model import process, but I don't know exactly what went wrong. Could you please give me some advice?

Thanks for your amazing work. It seems the inference time of onnx model is better than the tensorrt model. Is there anything wrong with my testing ? I got 150 ms time inference for the onnx model and 770 ms for the tenssorrt model.

Via https://github.com/xuanandsix/CLRNet-onnxruntime-and-tensorrt-demo/blob/main/demo_onnx.py
I get ONNX inference time about 5 times slower than pytorch model inference (I tried on "tusimple_r18.onnx", as in the above script).
Any ideas?

@xuanandsix

thanks for this demo!

I was trying to understand the implementation and I see that there is a modified softmax function
The original head uses the standard softmax implementation from pytorch:
https://github.com/Turoad/CLRNet/blob/7269e9d1c1c650343b6c7febb8e764be538b1aed/clrnet/models/heads/clr_head.py#L450

but I see that in your implementation you do something a bit different - you first subtract the max from each element in x. Why is that?

Can you provide tusimple_r18.onnx file，thanks

Hi,

I run:
python torch2onnx.py <config_file> --load_from <pth_file>
and get crash:-

pretrained model: https://download.pytorch.org/models/resnet101-5d3b4d8f.pth
Traceback (most recent call last):
File "torch2onnx.py", line 49, in
main()
File "torch2onnx.py", line 28, in main
net.load_state_dict(new_state_dict)
File "/home/msaha/anaconda3/envs/clrnet/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1223, in load_state_dict
raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for Detector:
Unexpected key(s) in state_dict: "heads.sample_x_indexs", "heads.prior_feat_ys", "heads.prior_ys", "heads.criterion.weight".

Any ideas? @xuanandsix
Thanks!

2onnx.log
Hello author, great work, but when I refer to your reademe for the conversion of pth to onnx, there seem to be some errors, see that the mailbox attachment you converted before tusimple_r18.onnx is 60M, and I only have 40M this one, is there something wrong? After the conversion is completed, use test .jpg can deduce the effect normally, but when I select some other lane line pictures to test, the effect is very unsatisfactory, Can I trouble you to send tusimple_r18 .onnx file again?

Hello, is there any plan to add C + + support

help qwq

Can you support a C++ version

Hi,

When running python torch2onnx.py <config_file> --load_from <pth_file>
I get this crash:-

RuntimeError: Exporting the operator grid_sampler to ONNX opset version 11 is not supported. Please feel free to request support or submit a pull request on PyTorch GitHub.

@xuanandsix, How did you resolve this at your end? Thanks!