sebby37 / dead-internet Goto Github PK

Just wanted to bring this to your attention as I saw some of the code comments about context limitations:

https://huggingface.co/MaziyarPanahi/Llama-3-8B-Instruct-32k-v0.1-GGUF

Hello, I would like to invite you to improve the script for synthetic data. Would you like to help me?
script:
https://github.com/win10ogod/AthenaGenSynth/blob/main/Synth.py
README.md
https://github.com/win10ogod/AthenaGenSynth

I modified the port in "ReaperEngine.py" to be able to work with koboldcpp, however nothing happens after the text generation has finished.
Video is here:
https://github.com/Sebby37/Dead-Internet/assets/53271427/c1376062-f1f9-4e2e-84c9-132ec2b6c0dd

In the cmd terminal (not the koboldcpp terminal), this pops up after the generation finishes:
127.0.0.1 - - [24/Apr/2024 21:52:37] "GET / HTTP/1.1" 200 -
[2024-04-24 21:52:48,432] ERROR in app: Exception on / [GET]
Traceback (most recent call last):
File "D:\Programme\Miniconda3\envs\DeadInternet\lib\site-packages\flask\app.py", line 1473, in wsgi_app
response = self.full_dispatch_request()
File "D:\Programme\Miniconda3\envs\DeadInternet\lib\site-packages\flask\app.py", line 882, in full_dispatch_request
rv = self.handle_user_exception(e)
File "D:\Programme\Miniconda3\envs\DeadInternet\lib\site-packages\flask\app.py", line 880, in full_dispatch_request
rv = self.dispatch_request()
File "D:\Programme\Miniconda3\envs\DeadInternet\lib\site-packages\flask\app.py", line 865, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) # type: ignore[no-any-return]
File "D:\Dead-Internet\main.py", line 17, in index
return engine.get_search(query)
File "D:\Dead-Internet\ReaperEngine.py", line 76, in get_search
search_page_completion = self.client.chat.completions.create(messages=[
File "D:\Programme\Miniconda3\envs\DeadInternet\lib\site-packages\openai_utils_utils.py", line 277, in wrapper
return func(*args, **kwargs)
File "D:\Programme\Miniconda3\envs\DeadInternet\lib\site-packages\openai\resources\chat\completions.py", line 579, in create
return self._post(
File "D:\Programme\Miniconda3\envs\DeadInternet\lib\site-packages\openai_base_client.py", line 1232, in post
return cast(ResponseT, self.request(cast_to, opts, stream=stream, stream_cls=stream_cls))
File "D:\Programme\Miniconda3\envs\DeadInternet\lib\site-packages\openai_base_client.py", line 921, in request
return self._request(
File "D:\Programme\Miniconda3\envs\DeadInternet\lib\site-packages\openai_base_client.py", line 997, in _request
return self._retry_request(
File "D:\Programme\Miniconda3\envs\DeadInternet\lib\site-packages\openai_base_client.py", line 1045, in _retry_request
return self._request(
File "D:\Programme\Miniconda3\envs\DeadInternet\lib\site-packages\openai_base_client.py", line 997, in _request
return self._retry_request(
File "D:\Programme\Miniconda3\envs\DeadInternet\lib\site-packages\openai_base_client.py", line 1045, in _retry_request
return self._request(
File "D:\Programme\Miniconda3\envs\DeadInternet\lib\site-packages\openai_base_client.py", line 1012, in _request
raise self._make_status_error_from_response(err.response) from None
openai.InternalServerError: Error code: 503 - {'detail': {'msg': 'Server is busy; please try again later.', 'type': 'service_unavailable'}}
127.0.0.1 - - [24/Apr/2024 21:52:48] "GET /?query=Cake HTTP/1.1" 500 -
127.0.0.1 - - [24/Apr/2024 21:58:21] "GET /favicon.ico HTTP/1.1" 200 -
127.0.0.1 - - [24/Apr/2024 22:04:56] "GET /?query=Cake HTTP/1.1" 200 -
127.0.0.1 - - [24/Apr/2024 22:05:11] "GET / HTTP/1.1" 200 -
127.0.0.1 - - [24/Apr/2024 22:05:11] "GET /favicon.ico HTTP/1.1" 200 -
127.0.0.1 - - [24/Apr/2024 22:12:09] "GET /?query=Cake HTTP/1.1" 200 -

When generating the URLs, generate a website description and use that description to guide the generation of the web page.
Consider using GBNF for this.

The Inference speed of those model is almost instant,
Adding a small image to webpages could add to the immersion with a small cost in time to generate, can this be done? maybe as an option?

Can I use ollama?

FYI -- At least on my M1 Mac, running server (from llama.cpp) with Llama 3 8B is much faster for browsing the Dead Internet. I also tried ollama and LM studio with the same model. I guess I shouldn't be too surprised, but thought it might be useful for others.

haha this is neat, so here's a tip for making running this project simpler:

add a requirements.txt to the root of this project with the following:

beautifulsoup4
openai
Flask

and change the readme.md for people to run:

pip install -r requirements.txt

nice job, ai friend