cannot start new chat after the update about serge HOT 14 CLOSED

@nsarrazin yup, all works fine now. thank you very much! 💛

from serge.

Thanks main works(for me). It's actually the first container build that has worked for me thus far. Worked flawlessly.

from serge.

Looks like your models are not loaded. Did you download one ?

from serge.

I'm encountering the exact same issue with Nginx as an IngressController in K8S. As soon as I try to put an ingress in front of the serge service (reverse proxy to http://serge:8008) with a TLS certificate I get a 405 Method not allowed.

from serge.

same problem. Already downloaded a model with
docker compose exec serge python3 /usr/src/app/api/utils/download.py tokenizer 7B

from serge.

Looks like your models are not loaded. Did you download one?

yes, I do have the models downloaded, it was working before the update, but now it doesn't. Also, I am unable to run it on my GPU when after installing stuff to run it on GPU

from serge.

Yeah, I have the same problem

from serge.

Same problem. Fresh deploy, latest release. After click to Start New Chat serge throws a 405 Method Not Allowed in log output.
My instance is behind Nginx Proxy Manager. Here is the related log output.

"GET /chat?temp=0.1&top_k=50&max_length=256&top_p=0.95&ctx_length=512&repeat_last_n=64&model=ggml-alpaca-7B-q4_0.bin&n_threads=4&repeat_penalty=1.3&init_prompt=Below+is+an+instruction+that+describes+a+task.+Write+a+response+that+appropriately+completes+the+request.+The+response+must+be+accurate%2C+concise+and+evidence-based+whenever+possible.+A+complete+answer+is+always+ended+by+%5Bend+of+text%5D. HTTP/1.1" 405 Method Not Allowed

from serge.

+1

from serge.

have same issue when using nginx proxy with TLS. direct 8008 access works.
maybe it is related to address name? it expects localhost, but gets whatever proxy is passing.

also I get proxy errors when my service in docker-compose is not called 'serge'.

from serge.

Hey everyone! I might have improved the situation with the latest version on main. Can you pull, rebuild and see if it fixed things ?

from serge.

I've just tried with the new released production image, it now works on K8S with a reverse proxy and TLS like a charm !
@nsarrazin I think the issue can be closed :)

from serge.

Good to hear, feel free to open it if you have issues again

from serge.

<3 working :) #FunResumed

from serge.