glorat / gptapps Goto Github PK

From GPT's own suggestion on this algorithm:

1. Split the user-supplied content into smaller chunks or segments that fit within the maximum token limit. Each segment should have a context that makes sense independently.
2. Start with the initial question and the first chunk of content. Send this as a prompt to the OpenAI API to get the answer for that specific context.
3. Take the generated answer and append it to the context for the next chunk of content. Repeat this process until you have processed all the chunks.
4. Finally, combine all the answers generated for each chunk to form a coherent response to the original question.

Currently only the last asked questions are remembered. Provide a way of saving different profiles

Currently, this is hardcoded to

Answer the question as truthfully as possible using the provided text, and if the answer is not contained within the text below, say "I don't know"

This seems to work well for most things but it could be tweakable to be better

The embeds search is relying on the content of the question to match against the content. But the question may lack sufficient search context - it may help to enhance the question with additional context specific to the question. In simple terms, it could act like a keyword search to find the right part of the document

E.g.
Q: What is the name of the buyer and seller
C: buyer, purchases, seller, supplier, agreement

so that repeated questioning of unchanged document goes faster

The embeddings endpoint has a generous rate limit but it can still be hit if the content is very long and a free account is being used. It should be wrapped with retry, just like the existin chatCompletion call

May need to merge with a default either here or in some pinia store

so that user can see where the AI is getting its answers from and interactively debug