How do I make h2ogpt fast in responding? Do the device’s resources have anything to do with the slowness of the response? about h2ogpt HOT 7 CLOSED

You should just download GGUF files from TheBloke at HF portal, e.g. https://huggingface.co/TheBloke/zephyr-7B-beta-GGUF

from h2ogpt.

يجب عليك فقط تنزيل ملفات GGUF من TheBloke على بوابة HF، على سبيل المثال https://huggingface.co/TheBloke/zephyr-7B-beta-GGUF

Is there an Arabic model in GGUF format?

from h2ogpt.

Use GGUF models and have a GPU.

from h2ogpt.

How could i convert the models, which in bin format, to gguf format؟

from h2ogpt.

jais is native arabic model, but you might get ok results using zephyr if you use the CLI options I shared for jais. Just change the --base_model to https://huggingface.co/TheBloke/zephyr-7B-beta-GGUF

from h2ogpt.

jais هو نموذج عربي أصلي، ولكن قد تحصل على نتائج جيدة باستخدام zephyr إذا كنت تستخدم خيارات CLI التي شاركتها مع jais. ما عليك سوى تغيير --base_model إلى https://huggingface.co/TheBloke/zephyr-7B-beta-GGUF

This is an Arabic model. Can you take a picture while you are using it?🙏

from h2ogpt.

Ok nevermind, zephyr doesn't do good with arabic.

So please share your run command, document, and question, and I can see if we can improve stopping behavior with jais.

from h2ogpt.