Could not load dynamic library 'cudart64_110.dll'; dlerror: cudart64_110.dll not found about koboldai-client HOT 6 CLOSED

I've had some folks report this before; it almost always still allows folks to use GPU acceleration (the "Looking for GPU support...FOUND!" in your log means PyTorch is detecting your CUDA install.) I haven't tracked down the source of the issue yet. A couple questions:

1. Do you have finetuneanon's transformers library installed?
2. Start Task Manager, then go to the Performance tab and click on your GPU on the left. Then spin up a model in Kobold. Does the 'Dedicated GPU Memory Usage' gauge increase as the model is initialized?

from koboldai-client.

1. I haven't installed it separately, no. I saw mention of it in the readme, but nothing about it in your SETUP instructions, so I didn't do anything related to it. Come to think of it, I have briefly run Clover Edition before, but I don't think it installed anything like that globally. I could certainly be wrong about that, though.
2. Hmm, yeah, I see my GPU memory usage jump from 1.0 GB to 2.1 GB when it's loaded with GPT-2, And once I close it, GPU memory usage drops back down to 1.0 GB. So I guess it is working through the GPU.

from koboldai-client.

Thanks, just trying to figure out what's injecting the call to CUDA 11. Since PyTorch is installed with an explicit reference to a CUDA version, I'm going to guess it's coming from tensorflow-gpu. When you submit text to the AI, do you get a result back in a few seconds, or closer to 20 seconds?

from koboldai-client.

It was responding within a second or two when I briefly tested it with GPT-2.

from koboldai-client.

Gotcha, that's definitely GPU computed. If you want to get rid of the warning, upgrading to CUDA 11 shouldn't hurt anything, but you'll need to uninstall PyTorch and reinstall it with the CUDA 11 arguments

from koboldai-client.

Just as a followup, I installed CUDA 11.1 and the associated PyTorch, and it indeed loads without error now, instead outputting a success message at that point.
2021-05-19 00:17:50.686998: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library cudart64_110.dll

from koboldai-client.