rhohndorf / auto-llama-cpp Goto Github PK
View Code? Open in Web Editor NEWUses Auto-GPT with Llama.cpp
License: MIT License
Uses Auto-GPT with Llama.cpp
License: MIT License
I pulled the git repository, edited my copy of .env to suit my needs, and even recreated a run.bat from the original AutoGPT project, adding the check_requirements.py script to the scripts (why was it actually removed?).
Using a conda environment with Python 3.10.10
Pip successfully installed all requirements
now calling
python scripts/main.py
results in
Traceback (most recent call last):
File "E:\LLama\Auto-Llama-cpp\scripts\main.py", line 3, in <module>
import commands as cmd
File "E:\LLama\Auto-Llama-cpp\scripts\commands.py", line 1, in <module>
import browse
File "E:\LLama\Auto-Llama-cpp\scripts\browse.py", line 4, in <module>
from llm_utils import create_chat_completion
File "E:\LLama\Auto-Llama-cpp\scripts\llm_utils.py", line 7
def create_chat_completion(messages[0]["content"], model=None, temperature=cfg.temperature, max_tokens=0)->str:
^
SyntaxError: invalid syntax
What am i doing wrong?
It never executes as it should and doesn't seem to find my model.
Im not sure if that is the problem or if it's even earlier.
There is no last_run_ai_settings.yaml, because it never executes.
What am i doing wrong?
ggml-vicuna-13b-4bit.bin
model.env
file (From the default)
SMART_LLM_MODEL=./models/ggml-vicuna-13b-4bit.bin
FAST_LLM_MODEL=./models/ggml-vicuna-13b-4bit.bin
EMBED_DIM = 8192
docker build -t foo/auto-llama .
docker run -p80:3000 foo/auto-llama
docker run -p80:3000 foo/auto-llama
Traceback (most recent call last):
File "/app/main.py", line 3, in <module>
import commands as cmd
File "/app/commands.py", line 1, in <module>
import browse
File "/app/browse.py", line 4, in <module>
from llm_utils import create_chat_completion
File "/app/llm_utils.py", line 4, in <module>
cfg = Config()
^^^^^^^^
File "/app/config.py", line 18, in __call__
cls._instances[cls] = super(
^^^^^^
File "/app/config.py", line 69, in __init__
self.EMBED_DIM = int(os.getenv("EMBED_DIM"))
^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: int() argument must be a string, a bytes-like object or a real number, not 'NoneType'
(base)
Was hoping the app would run after the steps above.
I'm sure I'm misconfiguring the setup.
# Paste your prompt here
I have the same problem when running any model, I tried running different versions of Vicuna since with the original 13b it also gives the same problem, I'm running it as it comes, I just added the model to the .env file, the prompt it has is the one it comes with default
Llama.generate: prefix-match hit
| Thinking...
llama_print_timings: load time = 1466.45 ms
llama_print_timings: sample time = 30.25 ms / 31 runs ( 0.98 ms per run)
llama_print_timings: prompt eval time = 768713.51 ms / 987 tokens ( 778.84 ms per token)
llama_print_timings: eval time = 24162.74 ms / 30 runs ( 805.42 ms per run)
llama_print_timings: total time = 794693.56 ms
Assistent Reply If you understand these rules, enter 'Ready' and I will start the game.
json If you understand these rules, enter 'Ready' and I will start the game.
json loads error Expecting value: line 1 column 1 (char 0)
Error:
Traceback (most recent call last): File "scripts/main.py", line 79, in print_assistant_thoughts assistant_reply_json = fix_and_parse_json(assistant_reply) File "/root/llama.cpp/Auto-Llama-cpp/scripts/json_parser.py", line 52, in fix_and_parse_json brace_index = json_str.index("{") ValueError: substring not found
json If you understand these rules, enter 'Ready' and I will start the game.
json loads error Expecting value: line 1 column 1 (char 0)
NEXT ACTION: COMMAND = Error: ARGUMENTS = substring not found
Enter 'y' to authorise command, 'y -N' to run N continuous commands, 'n' to exit program, or enter feedback for Entrepreneur-GPT...
Input:
Default
Default
Defaulr prompt
I don't think that is necessary anymore, llama.cpp can be compiled for BLAS support per https://github.com/ggerganov/llama.cpp/tree/master#blas-build
=> => exporting layers 8.5s
=> => exporting manifest sha256:81b29524e6ca86716c44c2fa16b8dc312af04dd88c3e3c03af98f087b650c8f4 0.0s
=> => exporting config sha256:00a84a43fa0487344f94620038375e6d5607fc5f482c144bf8e938ceb7c76803 0.0s
=> => naming to dangling@sha256:81b29524e6ca86716c44c2fa16b8dc312af04dd88c3e3c03af98f087b650c8f4 0.0s
=> => unpacking to dangling@sha256:81b29524e6ca86716c44c2fa16b8dc312af04dd88c3e3c03af98f087b650c8f4
but something like perks it right up, the source image can be swapped for rocm, intel, arm etc. i habve a cuda gpu so i played to my strong suite.
FROM nvidia/cuda:11.5.0-runtime-ubuntu20.04
RUN apt-get update && apt-get install python3.11 python3-pip -y
WORKDIR /app
COPY scripts/ /app/scripts/
COPY requirements.txt /app/
RUN ls //requirements.txt|xargs -n 1 -P 3 pip install -r
ENV CUDA_VISIBLE_DEVICES=all
CMD ["python3.11", "/bin/bash"]
failure to store and run image
store and run image
# Paste your prompt here
``` doesnt get that far
in scripts/llm_utils.py:grammar
, there is a line of code
grammar = LlamaGrammar.from_file("/home/ruben/Code/Auto-Llama-cpp/grammars/json.gbnf")
The line of code should not read a file with absolute path, but I am not should what should be used instead.
Error is prompt and program terminated. As I use Docker, I need to modify Dockerfile and add the lines below and build the image:
RUN mkdir -p /home/ruben/Code/Auto-Llama-cpp
COPY grammars /home/ruben/Code/Auto-Llama-cpp/grammars
The application should be able to build by
docker build -t auto-llama .
And run with
docker run -it --env-file "./.env" -v "<MODEL_PATH>:/models" auto-llama
# Paste your prompt here
I think it will be useful to add support for petals.dev.
I think it will run faster and work with bigger models like llama 2 70b.
No response
No response
Hi, I noticed that when calling LLM in the code, only the first item in the messages list is passed as the prompt. Is this an error?
Auto-Llama-cpp/scripts/llm_utils.py
Line 8 in e2aa7c4
No response
No response
npm start
./test-installation.sh
On Mac Mini M1, at 8 threads, llama.cpp is way slower than expected.
It only uses 20-30% of available resources for each worker.
Should use 100% resources for each thread.
N/A
You start the Program by executing the main.py File.
Then you press "y" and Enter.
After a couple of minutes you will get an Error Message.
I am using the Vicuna 13b Model.
I press enter and this is the Output after letting it "think".
AutoGPT INFO Error:
: Traceback (most recent call last):
File "C:\Users\alexr\Documents\Auto-Llama-cpp\scripts\main.py", line 79, in print_assistant_thoughts
assistant_reply_json = fix_and_parse_json(assistant_reply)
File "C:\Users\alexr\Documents\Auto-Llama-cpp\scripts\json_parser.py", line 52, in fix_and_parse_json
brace_index = json_str.index("{")
ValueError: substring not found
I think it should continue with the Process.
# Paste your prompt here
as newer version of llama.cpp support GPU how can we use that wiht this
iam new here
when I ran docker run -p80:3000 auto-llama1, I got the following errors:
Welcome to Auto-Llama! Enter the name of your AI and its role below. Entering nothing will load defaults.
Name your AI: For example, 'Entrepreneur-GPT'
AI Name: Traceback (most recent call last):
File "/app/main.py", line 313, in
prompt = construct_prompt()
^^^^^^^^^^^^^^^^^^
File "/app/main.py", line 205, in construct_prompt
config = prompt_user()
^^^^^^^^^^^^^
File "/app/main.py", line 231, in prompt_user
ai_name = utils.clean_input("AI Name: ")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/utils.py", line 3, in clean_input
return input(prompt)
^^^^^^^^^^^^^
EOFError: EOF when reading a line
Any idea how to fix it?
No response
No response
# Paste your prompt here
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.