Comments (5)
This is a good feature request. Thanks.
I haven't found a way to explain the template formatting in a simple way; even the code that reads the template is a bit messy...
But let's try:
I thought about the flexibility a little when making the template function. Like you said, most templates are very rigid but since its just "text" the user should be able to make any template they want.
The simplest way I could think of was just that the template does not care how the lines are formatted. Only the %1
(user input) line is important.
Now, it turned out that parsing a text file that could be of any length and have whatever characters is quite tricky :), so I separated it to prefix, header, input, and footer lines like this:
default_prefix
default_prefix
default_prefix
default_header
%1
default_footer
default_footer
default_footer
default_footer
And what it gives to the model as prompt is: (default_prefix + default_header + input + default_footer)
default_prefix
is basically the "instruction" lines at the start. You can describe whatever there and you can have as many lines as you like.
default_header
is the "prompt:" call, or you could have the user have a name here. Only 1 line, but of any length.
Then there is the %1
line that is the user input line.
default_footer
is the response, so you can say "response:" for instruction-type AIs or for example you can name your AI. You can have as many footer lines as you like.
For example, this template should also work:
This is a conversation between Alice and Bob.
Alice is an all-knowing AI that is always right.
She is especially good at counting words in sentences.
Bob: How many words the following sentence has?
%1
Alice:
Now the AI responds as Alice would in a conversation between Alice and Bob (user).
Some models (I believe those are called instruction-type models) like the words "Instruction:", "Prompt:" and "Response:" and/or the #
symbols, because they were trained on using them. The template does not impose any restrictions on the words or the use of #
symbols. But with models you just have to experiment and see what works.
I hope this helped. :)
from llamagptj-chat.
Wrapping the response is a good idea. :)
I need to think if there is a nice and simple way to do it without changing the response function. Right now it just dumps the response on stdout.
from llamagptj-chat.
I need to think if there is a nice and simple way to do it without changing the response function. Right now it just dumps the response on stdout.
Perhaps this is common enough to just add it to the CLI parameters.
The simplest way I could think of was just that the template does not care how the lines are formatted. Only the
%1
(user input) line is important.
I thought this might be the case but with LLMs it is a bit hard to test and know for sure. A couple follow ups: If the template is given to the LLM as is there should be no issues with reducing the template to just %1
allowing the prompter to specify the template with every prompt? If dynamic templates can't be done this way then could there be a way to load templates in and out perhaps in a similar fashion to /save
& /load
?
from llamagptj-chat.
Perhaps this is common enough to just add it to the CLI parameters.
Yup. Now in v0.2.5 with --b_token
and --e_token
.
The simplest way I could think of was just that the template does not care how the lines are formatted. Only the
%1
(user input) line is important.I thought this might be the case but with LLMs it is a bit hard to test and know for sure. A couple follow ups: If the template is given to the LLM as is there should be no issues with reducing the template to just
%1
allowing the prompter to specify the template with every prompt?
Yeah, should be possible. You might need to add some empty newlines in case the loading function happens to give errors:
%1
And then just have a way more elaborate prompt.
If dynamic templates can't be done this way then could there be a way to load templates in and out perhaps in a similar fashion to
/save
&/load
?
Good point. This is something I could maybe add in a future version.
from llamagptj-chat.
Yup. Now in v0.2.5 with
--b_token
and--e_token
.
Thanks, that makes things much easier.
Yeah, should be possible. You might need to add some empty newlines in case the loading function happens to give errors:
Thanks, I was missing the newlines.
If dynamic templates can't be done this way then could there be a way to load templates in and out perhaps in a similar fashion to
/save
&/load
?Good point. This is something I could maybe add in a future version.
Since empty templates work this is probably not that important since users can wrap the program and provide their own system for changing templates.
Thanks heaps for all your fast work, this now has all the required functionality to do anything I can think of for now, so I'm closing this issue.
from llamagptj-chat.
Related Issues (20)
- Way to reset the chat? HOT 6
- Error report: Terminated when loading configuration from a json file HOT 2
- What does save/load do? HOT 15
- bug: Seed parameter doesn't work. HOT 3
- Windows Defender Treats it as Virus HOT 6
- High cpu when launching as a process, and parent app quits HOT 5
- How to make it generate longer results? HOT 3
- Ingest local documents for analisys? HOT 6
- Illegal instruction HOT 2
- how to get rid of AVX? -DAVX2=OFF doesn't work HOT 9
- Here's how to compile and run under MINGW64 from Msys2 HOT 5
- nm asdf
- Update GPT4All/llama.cpp HOT 1
- Cannot build the app HOT 1
- Converting Models from .pth to .ggml HOT 2
- Functionality to have load history from a chat log file
- Functionality to have "memory" from loading a chat log file HOT 5
- How give context HOT 4
- Can't compile, llamamodel errors. HOT 5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from llamagptj-chat.