Comments (14)
@nsarrazin yup, all works fine now. thank you very much! 💛
from serge.
Thanks main
works(for me). It's actually the first container build that has worked for me thus far. Worked flawlessly.
from serge.
Looks like your models are not loaded. Did you download one ?
from serge.
I'm encountering the exact same issue with Nginx as an IngressController in K8S. As soon as I try to put an ingress in front of the serge service (reverse proxy to http://serge:8008) with a TLS certificate I get a 405 Method not allowed.
from serge.
same problem. Already downloaded a model with
docker compose exec serge python3 /usr/src/app/api/utils/download.py tokenizer 7B
from serge.
Looks like your models are not loaded. Did you download one?
yes, I do have the models downloaded, it was working before the update, but now it doesn't. Also, I am unable to run it on my GPU when after installing stuff to run it on GPU
from serge.
Yeah, I have the same problem
from serge.
Same problem. Fresh deploy, latest release. After click to Start New Chat
serge throws a 405 Method Not Allowed
in log output.
My instance is behind Nginx Proxy Manager. Here is the related log output.
"GET /chat?temp=0.1&top_k=50&max_length=256&top_p=0.95&ctx_length=512&repeat_last_n=64&model=ggml-alpaca-7B-q4_0.bin&n_threads=4&repeat_penalty=1.3&init_prompt=Below+is+an+instruction+that+describes+a+task.+Write+a+response+that+appropriately+completes+the+request.+The+response+must+be+accurate%2C+concise+and+evidence-based+whenever+possible.+A+complete+answer+is+always+ended+by+%5Bend+of+text%5D. HTTP/1.1" 405 Method Not Allowed
from serge.
+1
from serge.
have same issue when using nginx proxy with TLS. direct 8008 access works.
maybe it is related to address name? it expects localhost, but gets whatever proxy is passing.
also I get proxy errors when my service in docker-compose is not called 'serge'.
from serge.
Hey everyone! I might have improved the situation with the latest version on main. Can you pull, rebuild and see if it fixed things ?
from serge.
I've just tried with the new released production image, it now works on K8S with a reverse proxy and TLS like a charm !
@nsarrazin I think the issue can be closed :)
from serge.
Good to hear, feel free to open it if you have issues again
from serge.
<3 working :) #FunResumed
from serge.
Related Issues (20)
- 🚀 [Feature]: Add OpenVino / OpenVino Model Server HOT 1
- 🐛 [Bug]: Web interface does not render properly on mobile devices HOT 1
- 🚀 [Feature]: Add LINCE-Mistal model HOT 1
- 🐛 [Bug]: UI components are missing accessibility labels HOT 2
- 🐛 [Bug]: response text generated by a model sometimes disappears after computer/browser is woken up from a 'sleep' HOT 4
- have a separate page which displays downloaded moddles. HOT 1
- 🚀 [Feature]: Add support for Intel ARC GPUs A750 and A770 (If Possible) HOT 2
- bug: Allow loading .gguf and .bin files HOT 3
- 🚀 [Feature]: add eagle 7b HOT 3
- 🐛 [Bug]: system reachable via ICMP and via Port 8008 but screen "navy blue" with no text whatsoever HOT 14
- 🚀 [Feature]: Add Nous-Hermes-2-Mistral-7B-DPO HOT 8
- 🚀 [Feature]: Add support for uploading files during chat conversation
- 🐛 [Bug]: New install - response keeps repeating the last line HOT 7
- 🚀 [Feature]: add characters HOT 6
- 🚀 [Feature]: Please add Gorilla: Large Language Model Connected with Massive APIs HOT 3
- 🤗 [Question]: Whats the difference between the... models?
- 🚀 [Feature]: Add meta-llama/Meta-Llama-3-70B-Instruct HOT 7
- 🐛 [Bug]: Can't use pre-existing model at /weights HOT 1
- 🚀 [Feature]: Add support for Llama 3.1 Models HOT 2
- how to use mixtral-8x7b-v0.1🤗 [Question]: HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from serge.