[Feature Request] TensorRT support about comfyui HOT 15 OPEN

Yes it's planned. There's a few more things I want to implement before. Unless something better comes out in the meantime I'm going to implement it.

from comfyui.

SDXL models now have a TensorRT variant. https://huggingface.co/stabilityai/stable-diffusion-xl-1.0-tensorrt

from comfyui.

Aaaand new drivers just dropped promising 2X performance boost with TRT
https://www.nvidia.com/en-us/geforce/news/game-ready-driver-dlss-3-naraka-vermintide-rtx-vsr/

from comfyui.

support!
https://github.com/NVIDIA/Stable-Diffusion-WebUI-TensorRT

from comfyui.

advanced users can try my node: https://github.com/phineas-pta/comfy-trt-test

from comfyui.

I just tested Nvidia's A1111 TRT extension and results are 2x faster indeed (at least for 512x768 simple generation).

from comfyui.

There was a pull for automatic that references the limits of TRT.

AUTOMATIC1111/stable-diffusion-webui-tensorrt#36

There is an upper limit of what it can do. As an example, if you have your batch size set to 8, you may not be able to generate dynamic images greater than 512x480 and the like.

I took note of some issues I've found during the year of TRT implementations flooding GitHub.

AUTOMATIC1111/stable-diffusion-webui-tensorrt#46 (comment)

Max dimensions I got from the calculated limit is Batch size 1, 858 x 858.

from comfyui.

Be prepared to pull your hair out if you get to it. Getting it setup on one machine is a chore, but getting it to work on someone else's is worse. Many TensorRT implementations have fallen because of it. Volta-ML and SDA-node are the larger examples.

from comfyui.

I would love to see tensorrt support, because SD XL is quite slow.
Also, tensorrt only seems to support max 768x768px. Do you think it is somehow possible to pass the SDXL 1024 in?

from comfyui.

Yeah I'm going to go with AITemplate instead of TRT unless they add a way to replace the weights at runtime.

from comfyui.

Yeah AiTemplate seems good too.

from comfyui.

yeah its there! any idea how make it work in comfyui

from comfyui.

from comfyui.

Aaaand new drivers just dropped promising 2X performance boost with TRT https://www.nvidia.com/en-us/geforce/news/game-ready-driver-dlss-3-naraka-vermintide-rtx-vsr/

and with that I'd love to see the fruits of this 2x speed boost as that could greatly improve my workflow on my 3090.

from comfyui.

Ok so how do i translate that support instructions for auto1111 to compfy. As I was using auto1111 but I was getting issues creating and using the optimized versions

from comfyui.