Comments (15)
nsarrazin
commented on July 21, 2024
Looks like you're running python 3.8 for some reason which doesn't support the typings we use.
Can you give me some details about your OS, and your docker version ?
from serge.
arjones85
commented on July 21, 2024
Sure - Rocky 8.7 and Docker CE 23
from serge.
arjones85
commented on July 21, 2024
Also, I'm actually running Python 3.6. The errors about 3.8 is the version of Python inside the container.
Python 3.9 is available in Rocky. If you think the host python version is the issue, I'll upgrade and try it.
Edit: Made Python 3.9 default python, tried it again, same errors unfortunately.
from serge.
OilProducts
commented on July 21, 2024
Its a problem with the python version inside the container. Simply running docker compose up builds a container with 3.8. not sure how anyone is getting it to work...
Edit: the "docker:latest" on my machine was 2 years old, pulling a new one works fine.
from serge.
arjones85
commented on July 21, 2024
the "docker:latest" on my machine was 2 years old, pulling a new one works fine.
Sorry - Pulling a new one of what? I went through and did a docker pull of nginx:alpine, ubuntu:latest, node:latest, and docker:latest and it hasn't made any difference for me. Still the same error, it's still running 3.8 in the containers.
from serge.
OilProducts
commented on July 21, 2024
Sorry I mis typed. "ubuntu:latest" was the problem.
from serge.
arjones85
commented on July 21, 2024
Thanks! I ended up stopping all my other containers, pruning all images and the containers themselves, then doing a pull. That seems to have resolved that error.
Now, when I try to send anything to it, it is erroring out with:
serge-nginx-1 | 10.150.1.5 - - [23/Mar/2023:17:11:35 +0000] "GET /__data.json?x-sveltekit-invalidated=_1 HTTP/1.1" 200 82 "http://192.168.1.110:8008/chat/9b82505e-f78d-4fe1-97a3-1323117cc6de" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/111.0.0.0 Safari/537.36" "-"
serge-api-1 | INFO: 172.27.0.3:47002 - "GET /models HTTP/1.1" 200 OK
serge-nginx-1 | 10.150.1.5 - - [23/Mar/2023:17:11:35 +0000] "GET /api/models HTTP/1.1" 200 2 "http://192.168.1.110:8008/chat/9b82505e-f78d-4fe1-97a3-1323117cc6de" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/111.0.0.0 Safari/537.36" "-"
serge-api-1 | INFO: 172.27.0.3:47004 - "GET /chat/9b82505e-f78d-4fe1-97a3-1323117cc6de HTTP/1.1" 200 OK
serge-nginx-1 | 10.150.1.5 - - [23/Mar/2023:17:11:36 +0000] "GET /api/chat/9b82505e-f78d-4fe1-97a3-1323117cc6de HTTP/1.1" 200 497 "http://192.168.1.110:8008/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/111.0.0.0 Safari/537.36" "-"
serge-api-1 | INFO: 172.27.0.3:53868 - "GET /chat/9b82505e-f78d-4fe1-97a3-1323117cc6de/question?prompt=Test HTTP/1.1" 200 OK
serge-nginx-1 | 2023/03/23 17:11:43 [error] 34#34: *78 upstream prematurely closed connection while reading upstream, client: 10.150.1.5, server: localhost, request: "GET /api/chat/9b82505e-f78d-4fe1-97a3-1323117cc6de/question?prompt=Test HTTP/1.1", upstream: "http://172.27.0.5:9124/chat/9b82505e-f78d-4fe1-97a3-1323117cc6de/question?prompt=Test", host: "192.168.1.110:8008", referrer: "http://192.168.1.110:8008/chat/9b82505e-f78d-4fe1-97a3-1323117cc6de"
serge-api-1 | ERROR: Exception in ASGI application
serge-api-1 | Traceback (most recent call last):
serge-api-1 | File "/usr/local/lib/python3.10/dist-packages/uvicorn/protocols/http/httptools_impl.py", line 436, in run_asgi
serge-api-1 | result = await app( # type: ignore[func-returns-value]
serge-api-1 | File "/usr/local/lib/python3.10/dist-packages/uvicorn/middleware/proxy_headers.py", line 78, in __call__
serge-api-1 | return await self.app(scope, receive, send)
serge-api-1 | File "/usr/local/lib/python3.10/dist-packages/fastapi/applications.py", line 276, in __call__
serge-api-1 | await super().__call__(scope, receive, send)
serge-api-1 | File "/usr/local/lib/python3.10/dist-packages/starlette/applications.py", line 122, in __call__
serge-api-1 | await self.middleware_stack(scope, receive, send)
serge-api-1 | File "/usr/local/lib/python3.10/dist-packages/starlette/middleware/errors.py", line 184, in __call__
serge-api-1 | raise exc
serge-api-1 | File "/usr/local/lib/python3.10/dist-packages/starlette/middleware/errors.py", line 162, in __call__
serge-api-1 | await self.app(scope, receive, _send)
serge-api-1 | File "/usr/local/lib/python3.10/dist-packages/starlette/middleware/cors.py", line 84, in __call__
serge-api-1 | await self.app(scope, receive, send)
serge-api-1 | File "/usr/local/lib/python3.10/dist-packages/starlette/middleware/exceptions.py", line 79, in __call__
serge-api-1 | raise exc
serge-api-1 | File "/usr/local/lib/python3.10/dist-packages/starlette/middleware/exceptions.py", line 68, in __call__
serge-api-1 | await self.app(scope, receive, sender)
serge-api-1 | File "/usr/local/lib/python3.10/dist-packages/fastapi/middleware/asyncexitstack.py", line 21, in __call__
serge-api-1 | raise e
serge-api-1 | File "/usr/local/lib/python3.10/dist-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
serge-api-1 | await self.app(scope, receive, send)
serge-api-1 | File "/usr/local/lib/python3.10/dist-packages/starlette/routing.py", line 718, in __call__
serge-nginx-1 | 10.150.1.5 - - [23/Mar/2023:17:11:43 +0000] "GET /api/chat/9b82505e-f78d-4fe1-97a3-1323117cc6de/question?prompt=Test HTTP/1.1" 200 0 "http://192.168.1.110:8008/chat/9b82505e-f78d-4fe1-97a3-1323117cc6de" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/111.0.0.0 Safari/537.36" "-"
serge-api-1 | await route.handle(scope, receive, send)
serge-api-1 | File "/usr/local/lib/python3.10/dist-packages/starlette/routing.py", line 276, in handle
serge-api-1 | await self.app(scope, receive, send)
serge-api-1 | File "/usr/local/lib/python3.10/dist-packages/starlette/routing.py", line 69, in app
serge-api-1 | await response(scope, receive, send)
serge-api-1 | File "/usr/local/lib/python3.10/dist-packages/sse_starlette/sse.py", line 227, in __call__
serge-api-1 | async with anyio.create_task_group() as task_group:
serge-api-1 | File "/usr/local/lib/python3.10/dist-packages/anyio/_backends/_asyncio.py", line 662, in __aexit__
serge-api-1 | raise exceptions[0]
serge-api-1 | File "/usr/local/lib/python3.10/dist-packages/sse_starlette/sse.py", line 230, in wrap
serge-api-1 | await func()
serge-api-1 | File "/usr/local/lib/python3.10/dist-packages/sse_starlette/sse.py", line 219, in stream_response
serge-api-1 | async for data in self.body_iterator:
serge-api-1 | File "/usr/src/app/main.py", line 161, in event_generator
serge-api-1 | async for output in generate(
serge-api-1 | File "/usr/src/app/utils/generate.py", line 65, in generate
serge-api-1 | raise ValueError(error_output.decode("utf-8"))
serge-api-1 | ValueError: main: seed = 1679591503
serge-api-1 | llama_model_load: loading model from '/usr/src/app/weights/ggml-alpaca-7B-q4_0.bin' - please wait ...
serge-api-1 | llama_model_load: failed to open '/usr/src/app/weights/ggml-alpaca-7B-q4_0.bin'
serge-api-1 | llama_init_from_file: failed to load model
serge-api-1 | main: error: failed to load model '/usr/src/app/weights/ggml-alpaca-7B-q4_0.bin'
I think I'll wait a bit before trying it out again, let this project get industrialized a bit :-)
from serge.
nsarrazin
commented on July 21, 2024
I've fixed the dependencies in my latest commits. Feel free to pull the latest from main and try again, hopefully it works better now.
from serge.
pabl-o-ce
commented on July 21, 2024
Hi @nsarrazin , I'm using at host Python 3.10.10 get same 502 error at browser and in the logs I found this errors
serge-api-1 | INFO: Started reloader process [1] using WatchFiles
serge-api-1 | INFO: Started server process [8]
serge-api-1 | INFO: Waiting for application startup.
serge-api-1 | INFO: main initializing database connection
serge-api-1 | ERROR: Traceback (most recent call last):
serge-api-1 | File "/usr/local/lib/python3.10/dist-packages/starlette/routing.py", line 677, in lifespan
serge-api-1 | async with self.lifespan_context(app) as maybe_state:
serge-api-1 | File "/usr/local/lib/python3.10/dist-packages/starlette/routing.py", line 566, in __aenter__
serge-api-1 | await self._router.startup()
serge-api-1 | File "/usr/local/lib/python3.10/dist-packages/starlette/routing.py", line 654, in startup
serge-api-1 | await handler()
serge-api-1 | File "/usr/src/app/main.py", line 96, in start_database
serge-api-1 | await initiate_database()
serge-api-1 | File "/usr/src/app/utils/initiate_database.py", line 25, in initiate_database
serge-api-1 | await init_beanie(
serge-api-1 | File "/usr/local/lib/python3.10/dist-packages/beanie/odm/utils/init.py", line 528, in init_beanie
serge-api-1 | await Initializer(
serge-api-1 | File "/usr/local/lib/python3.10/dist-packages/beanie/odm/utils/init.py", line 89, in __await__
serge-api-1 | yield from self.init_class(model).__await__()
serge-api-1 | File "/usr/local/lib/python3.10/dist-packages/beanie/odm/utils/init.py", line 498, in init_class
serge-api-1 | await self.init_document(cls)
serge-api-1 | File "/usr/local/lib/python3.10/dist-packages/beanie/odm/utils/init.py", line 322, in init_document
serge-api-1 | build_info = await self.database.command({"buildInfo": 1})
serge-api-1 | File "/usr/lib/python3.10/concurrent/futures/thread.py", line 58, in run
serge-api-1 | result = self.fn(*self.args, **self.kwargs)
serge-api-1 | File "/usr/local/lib/python3.10/dist-packages/pymongo/_csot.py", line 105, in csot_wrapper
serge-api-1 | return func(self, *args, **kwargs)
serge-api-1 | File "/usr/local/lib/python3.10/dist-packages/pymongo/database.py", line 805, in command
serge-api-1 | with self.__client._socket_for_reads(read_preference, session) as (
serge-api-1 | File "/usr/local/lib/python3.10/dist-packages/pymongo/mongo_client.py", line 1296, in _socket_for_reads
serge-api-1 | server = self._select_server(read_preference, session)
serge-api-1 | File "/usr/local/lib/python3.10/dist-packages/pymongo/mongo_client.py", line 1257, in _select_server
serge-api-1 | server = topology.select_server(server_selector)
serge-api-1 | File "/usr/local/lib/python3.10/dist-packages/pymongo/topology.py", line 272, in select_server
serge-api-1 | server = self._select_server(selector, server_selection_timeout, address)
serge-api-1 | File "/usr/local/lib/python3.10/dist-packages/pymongo/topology.py", line 261, in _select_server
serge-api-1 | servers = self.select_servers(selector, server_selection_timeout, address)
serge-api-1 | File "/usr/local/lib/python3.10/dist-packages/pymongo/topology.py", line 223, in select_servers
serge-api-1 | server_descriptions = self._select_servers_loop(selector, server_timeout, address)
serge-api-1 | File "/usr/local/lib/python3.10/dist-packages/pymongo/topology.py", line 238, in _select_servers_loop
serge-api-1 | raise ServerSelectionTimeoutError(
serge-api-1 | pymongo.errors.ServerSelectionTimeoutError: mongodb:27017: [Errno -2] Name or service not known, Timeout: 30s, Topology Description: <TopologyDescription id: 641d1cdd03bb7fd9fe7b2a4e, topology_type: Unknown, servers: [<ServerDescription ('mongodb', 27017) server_type: Unknown, rtt: None, error=AutoReconnect('mongodb:27017: [Errno -2] Name or service not known')>]>
serge-api-1 |
serge-api-1 | ERROR: Application startup failed. Exiting.
any tip? what version you need of python to work it?
I'm with the latest commit
Psd: Amazing work
from serge.
pabl-o-ce
commented on July 21, 2024
forget it I resolved... I was trying to skip nginx and somehow it get very messy
from serge.
voronind-com
commented on July 21, 2024
I get the same TypeError: fetch failed on my server with nginx proxy with TLS. same docker image on my workstation works (no nginx, direct access to 8008 port). waiting for proxy support.
from serge.
pabl-o-ce
commented on July 21, 2024
@cakee-ru i make it work for the current version serge are with this config on nginx
server {
listen [::]:443 ssl http2;
listen 443 ssl http2;
# The host name to respond to
server_name ai.yourdomain.com;
...your config...
# Path for static files
location / {
proxy_pass http://xxx.xxx.xxx.xxx:8008/;
proxy_redirect off;
}
}
server {
listen [::]:80;
listen 80;
server_name ai.yourdomain.com;
# (1)
return 301 https://$host$request_uri;
# return 444;
}
xxx.xxx.xxx.xxx is the ip your are using local o docker network
from serge.
voronind-com
commented on July 21, 2024
@pabl-o-ce the key part is proxy_redirect off;? cause I have pretty much the same basic config.
from serge.
pabl-o-ce
commented on July 21, 2024
you can skip it.. i was messing around the best way to make it work.. the important part right now (but it will resolve soon #84 84) is to put / and the end of proxy_pass http://ip:port/... I'm not into svelte but it work for proxy nginx with a domain
from serge.
gaby
commented on July 21, 2024
@arjones85 Still having issues with this?
from serge.
Related Issues (20)
- 🚀 [Feature]: Add OpenVino / OpenVino Model Server HOT 1
- 🐛 [Bug]: Web interface does not render properly on mobile devices HOT 1
- 🚀 [Feature]: Add LINCE-Mistal model HOT 1
- 🐛 [Bug]: UI components are missing accessibility labels HOT 2
- 🐛 [Bug]: response text generated by a model sometimes disappears after computer/browser is woken up from a 'sleep' HOT 4
- have a separate page which displays downloaded moddles. HOT 1
- 🚀 [Feature]: Add support for Intel ARC GPUs A750 and A770 (If Possible) HOT 2
- bug: Allow loading .gguf and .bin files HOT 3
- 🚀 [Feature]: add eagle 7b HOT 3
- 🐛 [Bug]: system reachable via ICMP and via Port 8008 but screen "navy blue" with no text whatsoever HOT 14
- 🚀 [Feature]: Add Nous-Hermes-2-Mistral-7B-DPO HOT 8
- 🚀 [Feature]: Add support for uploading files during chat conversation
- 🐛 [Bug]: New install - response keeps repeating the last line HOT 7
- 🚀 [Feature]: add characters HOT 6
- 🚀 [Feature]: Please add Gorilla: Large Language Model Connected with Massive APIs HOT 3
- 🤗 [Question]: Whats the difference between the... models?
- 🚀 [Feature]: Add meta-llama/Meta-Llama-3-70B-Instruct HOT 7
- 🐛 [Bug]: Can't use pre-existing model at /weights HOT 1
- 🐛 [Bug]: DLLAMA_BLAS_VENDOR=OpenBLAS build with pip is not enabling OpenBlas HOT 3
- how to use mixtral-8x7b-v0.1🤗 [Question]: HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from serge.