Comments (5)
Hello, I think it's this way:
Different models and different use cases have different prompt structures. In the "Chat" tab you have somewhere three options for three simple use cases: "Action" with the entries "Query", "Summarize", and "Extract".
Depending on the use case, the useful prompt parts (the content of those fields you asked about) are sent together with your "visible" prompt - before and after.
One "soft" use case within "Query" is ICL (In-context Learning), where you give examples as context for your actual request.
An extension of ICL (and a focal point of H20 GPT) is RAG (Retrieval-augmented generation), where the context for the query is retrieved from relevant chunks of uploaded documents. Those document chunks are refered to as "documents".
A typical prompt for a RAG use case would consist of those three prompt parts:
- Task: Respond to the request below by taking the context into consideration.
- Context: [the document chunk about sumfin]
- Request: [your visible query, e.g. "What is foo in regards of sumfin?"]
Your "Task" part would be entered in "Query Pre-Prompt", because it is "Added before documents".
Your "Request" part of the prompt would be entered in "Query Prompt", because it is "Added after documents".
"Summary Pre-Prompt" is meant for the "Summarize" use case option (with RAG).
"Summary Prompt" is meant for the "Summarize" use case option (without RAG, without considering your own documents).
HYDE is a different prompt approach to get better results, but I don't know how to activate it in H20GPT. https://medium.com/prompt-engineering/hyde-revolutionising-search-with-hypothetical-document-embeddings-3474df795af8
System prompt is a prompt part that is added to all of your prompts (systematically).
from h2ogpt.
Thanks!
I'll update this to include some of the expert options.
https://github.com/h2oai/h2ogpt/blob/main/docs/README_ui.md#expert-tab
The expert options change alot more than other things, so I can't promise to document all items or keep the docs fully up to date, but I understand which ones could have some help and/or longer info strings in the UI itself.
from h2ogpt.
Great stuff, thank you both. I have been doing AI research all day as I'm new to the field, but we're basically wanting to query internal documents for my work and allow our technicians to ask questions. The reason I was looking into these prompts is because on occasion, it will start associating things together it shouldn't or answer questions in an odd way, and after seeing the amount of prompt options, I assumed we weren't giving it enough instruction of how we wanted it to respond.
Of course I can just enter these values in the h2o UI, but without know what these options meant, it didn't make much sense to me and I had no confidence in what I was doing. A revelation I (think) I had is that an LLM doesn't actually accept anything other than text. So for example, if I'm utilizing langchain to process the contents of an image, from my understanding, it would run the image through some specialized program that is designed to simply describe the image. Then, this description is effectively imported into the prompt, on top of what I wrote in the standard query input. So no matter what kind of document you're uploading, it's ultimately getting converted into a useful text format that the LLM can deal with. Is my understanding of this correct?
I still need some way to automatically populate these prompts on startup. I'm running h2o in a docker container and I launch it using this command:
sudo docker run -p 7860:7860 --rm --init \
-v "${HOME}"/.cache:/workspace/.cache \
-v "${HOME}"/save:/workspace/save \
-v "${HOME}"/user_path:/workspace/user_path \
-v "${HOME}"/db_dir_UserData:/workspace/db_dir_UserData \
-v "${HOME}"/users:/workspace/users \
-v "${HOME}"/db_nonusers:/workspace/db_nonusers \
-v "${HOME}"/llamacpp_path:/workspace/llamacpp_path \
gcr.io/vorvan/h2oai/h2ogpt-runtime:0.1.0 /workspace/generate.py \
--use_safetensors=True \
--prompt_type=zephyr \
--save_dir='/workspace/save/' \
--user_path /workspace/user_path \
--system_prompt "$(cat prompt.txt)" \
--base_model=stabilityai/stablelm-zephyr-3b \
--max_quality=True \
--langchain_mode="UserData"
As you can see, I'm setting the system prompt by reading a prompt.txt file. This way I can easily tweak it and just relaunch the docker container. However, the trouble I'm having is when I try to set the other prompts, so they're automatically populated as well. I couldn't find any argument that would directly set some of these other prompts. Based on what was said, it sounds like I'm interested in changing Query Pre Prompt and Query Prompt. Can anyone help me understand how to set the prompts as an argument, so they're already populated when I start the container?
I have one another related problem that I'd like to know more about. I have a python program that utilizes the grclient.py file to send queries. Here's a snippet of it:
if __name__ == "__main__":
instruction = sys.argv[1]
chat_id = sys.argv[2]
system_prompt = read_prompt_from_file("prompt.txt")
chat_history = get_chat_history(chat_id)
kwargs = {
'system_prompt': system_prompt,
'langchain_mode': "UserData",
'document_subset': "Relevant",
'chat_conversation': chat_history
}
client = GradioClient("http://localhost:7860/")
response = client.query(instruction, **kwargs)
update_chat_history(chat_id, instruction, response)
print(response)
As you can see here, I am defining the system prompt, which gets pulled from the same prompt.txt file that the docker command pulls from. This way they have the same prompt, however, if I already have the docker container launched with the correct system prompt, do I need to supply it as a kwarg still? I'm not sure if the fact that I launched the docker container with all the settings I wanted, will the grclient.py file still obey these values set by docker? Or is the grclient.py file a separate "instance"? If I recall correctly, I had to set the document_subset and langchain_mode in order for this program to work the same as the h2o UI, so I would assume the UI settings don't carry over?
Same as the docker container arguments, I cannot find any option for setting the Query Pre Prompt and Query Prompt in the grclient.py file. Am I able to set them as kwags as well?
I tired to run the docker container with this command as a test, and I didn't see anything update in the expert tab, but in the models tab, I did see them in the "Current or Custom Model Prompt" section. Is this the same as setting them in the Expert tab and still apply these prompts to my query?
sudo docker run -p 7860:7860 --rm --init \
-v "${HOME}"/.cache:/workspace/.cache \
-v "${HOME}"/save:/workspace/save \
-v "${HOME}"/user_path:/workspace/user_path \
-v "${HOME}"/db_dir_UserData:/workspace/db_dir_UserData \
-v "${HOME}"/users:/workspace/users \
-v "${HOME}"/db_nonusers:/workspace/db_nonusers \
-v "${HOME}"/llamacpp_path:/workspace/llamacpp_path \
gcr.io/vorvan/h2oai/h2ogpt-runtime:0.1.0 /workspace/generate.py \
--use_safetensors=True \
--prompt_type=custom \
--prompt_dict="{'promptA': 'value1', 'promptB': 'value2', 'PreInstruct': 'value3', 'PreInput': 'value4', 'PreResponse': 'value5', 'terminate_response': 'value6', 'chat_sep': 'value7', 'chat_turn_sep': 'value8', 'humanstr': 'value9', 'botstr': 'value10'}" \
--save_dir='/workspace/save/' \
--user_path /workspace/user_path \
--system_prompt "$(cat prompt.txt)" \
--base_model=stabilityai/stablelm-zephyr-3b \
--max_quality=True \
--langchain_mode="UserData"
So sorry for all the questions, but these are things I've tried to look up but either didn't find anything or I simply didn't understand it. I really appreciate the assistance.
from h2ogpt.
An example of setting all the prompts is here:
from h2ogpt.
For how the prompt templates look like, see:
Lines 926 to 945 in f505c8f
from h2ogpt.
Related Issues (20)
- Anyway to "contain" the installation? HOT 3
- [Q] Which prompt type should I use for Starling? HOT 2
- Without Line breaks (\n) issue of extract response HOT 1
- planning prompt example HOT 1
- Load evaluations json in UTF HOT 4
- Model weights on VRAM and Embeddings on RAM HOT 1
- Maintain context with Gradio Client HOT 3
- Windows manual install, some error and strange behaviour HOT 6
- async to vllm broken, probably since transition to openai 1.x HOT 1
- Unable to import files on MacOS HOT 2
- DocTR OCR seems broken -- layout aware or not. HOT 1
- Windows 10 Manual Install: Failed to load shared library: llama_cpp_cuda\llama.dll HOT 2
- ModuleNotFoundError: No module named 'langchain_core.tracers.langchain_v1' HOT 4
- Understanding h2oGpt and inference server relationship HOT 7
- Followed instructions exactly, error about H2O_Fire and Fire on run? HOT 9
- Download folder / .cache file settings==> MISSING!! HOT 4
- Can't change model, httpx module not found HOT 17
- make model list return only alive connections
- Avatars do not display HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from h2ogpt.