I noticed that when I search for "In-situ Flow Information Telemetry", I will encounter an error written below:

openai.error.InvalidRequestError: This model's maximum context length is 8192 tokens, however you requested 12301 tokens (12301 in your prompt; 0 for the completion). Please reduce your prompt; or completion length.

the same thing happened when I run it locally.
Please check your text length limit to see if the restriction you define don't meet with the token length limit.

• along with various API choices

Hi! Thanks for the project, it works surprisingly nicely. I checked the code and it seems that for now you check how much context can fit with the character length, which is really limiting the capabilities. In OpenAI, especially with gpt-3.5-turbo, most English words are really 1 token, so with 3000 token context you can get ~3k English words (in reality a bit less, but still close), not characters. This means that searchGPT will be able to pass 2-3x more web page content to the request so the results will improve, albeit it'll cost more.

The best library to use for tokenization for OpenAI right now is https://github.com/openai/tiktoken. For older models (everything except gpt-3.5-turbo) you can use the gpt2 encoding, for Turbo you have to use cl100k_base.

When I type search terms such as "spacex" and "apple", searchGPT doesn't allow that. To me this is pretty limiting and force the users to make the search term much longer than it should be.

• Large language model API integration
• MVP: OpenAI API

Allow the option for users to not use OpenAI API (which need API key)

• Fine-tuned GPT-NeoX 20B (@michaelthwan) blocked, too expensive
• GPT-NeoX 20B (from Goose AI) (@michaelthwan )
• Bloom xxB (created another card)

Sometimes the footnote have wrong order.
@panglydia know why.

• langchain-faiss -> vectorstore
• whoosh (cancelled)
• https://haystack.deepset.ai/ (cancelled)

You should consider to split backend from frontend to give the project better integrability with other projects.
The backend could be published in a pip package and be usable without cloning the frontend part too.
For example I would like to integrate the backend part into my discord bot but at the time I have to clone the entire project.

To get the best at all, providing an interface with the backend with REST APIs could make the development of a frontend way faster and easier and above all i think it would have more interest by frontend developers which they can use the framework they want.
Flask is a good framework but I think is not so used by developers, instead for example of React or Angular that are so much used.

I think CHATGPT writes as if it is September 2021 because we do not specify the current time in the promt. This problem can be solved with a python script that will pull the current time information from a dual redundant NTP server and keep it updated daily in the promt. The easy solution would be to add the current date to the promt in this way: The current date: (currentDate).

Win app

• Electron (blocked)
• React native (blocked)

Web server

• Gradio
• Streamlit (blocked)
• Flask
• Flask + next.js (blocked)
• MERN stack (blocked)
• MERN stack + next.js (blocked)

We support some API call (OpenAI / GooseAI) but it would be nice if we can select a model small but good enough to do the summarization work in local GPU.

• Study models
• Code the local GPU call LLM module

The UI is built by non-frontend people so it is relatively plain now.

Enable OpenAPI and get Davinci model for LLM API
- Don't push API key to git!

• Enable OpenAPI subscription
• Comparison of different grade API:Ada/Babbage/Curie/Davinci
• Use Python to connect the API

Can specify domain name or even one URL to limit search.

e.g. I think the 5x scrap website is quite slow

Study how to get web result. Able to extract Bing result

• Azure Bing API subscription
• Python code to call Bing API

Are there any plans to utilize a vector DB like Pinecone?

Currently using Tf, score >= 5 as the rule for filtering correct reference.
See if there is needs for tuning. (e.g. too many responses have no reference)

Related code, in FootnoteService

SCORE_THRESHOLD = 5
result_within_scope_df = result_df[result_df['score'] >= SCORE_THRESHOLD]

Reference new bing.
To improve the efficiency/accuracy of searching

https://peps.python.org/pep-0008/
https://github.com/psf/black

It is possible to retrieve document index by pre-calculation, especially for offline documents.

• Index online website - get document location, sentences, reference source
• Store DB Index

E.g. retrieve doc index from ChatGPT
/xx/xx/ChatGPT.pptx, , Slide3
xx.com/xxx/chatgpt, , xx.com

So that we may analyze users' input

• Play with Davinci API for summarization power
• Try different prompt for best result
• Try intrinsic reference[1] footnote approach [blocked]

[1] i.e. instead of giving footnote by another similarity module, try to let model provide so.

It is possible to retrieve document index by pre-calculation, especially for offline documents.

• Index offline document - get document location, sentences, reference source
• Store in DB
• Using pyterrier

E.g. retrieve doc index from ChatGPT
/xx/xx/ChatGPT.pptx, , Slide3
xx.com/xxx/chatgpt, , xx.com

Look like sometimes user don't know what to search. Give some random examples?

• Make config things selectable from UI
• possible to prefill certain fields (from query string)

pro1: We don't use a lot of faiss api
pro2: Stupid duplicated call
pro3: enable distance -> can tune the footnote

Difficulty: how to effectively batch call

The user experience is not good when searching things. Can show intermediate result like new bing.

Have no idea why the app use up >512 MB memory.
As you can see, I restarted the app at ~12pm and after some calls from users, it grow very fast (and >100% and using swap memory)

Instead of giving a error page like this.

Many input will make the program die now.
E.g. symbol. Let to make it more robust

Hi,
Any chance of integrating Cohere's API for the LLM? It's free (upto a point), so would be a cost-effective way of doing this.
Here's a playground: https://dashboard.cohere.ai/playground/generate

Wouldn't using Google CSE be a more cost effective solution than using the bing search api? Btw Langchain supports Google CSE.
Link: https://programmablesearchengine.google.com/controlpanel/all

Show explanation of why a news is marked as reference

Currently using p extraction which have many useless text (sentences) in the text_df.
Need to trim only useful one.
BTW if it is still too long, the get_prompt will trim it using config's prompt_length_limit

Checklist

• beautifulsoup
• trafilatura
• other method?

We may have a gut feeling of how is the quality of response from current pipeline.
But we may want a boarder range of prompts with different context.

For example

• Normal question (What is ChatGPT?)
• Strange question (What is the best way to eat metal ball?)

I built it on the server and it works fine

(searchChat) root@fusang:~/docker/searchGPT/src# python flask_app.py
 * Serving Flask app 'website'
 * Debug mode: on
WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
 * Running on http://127.0.0.1:5000
Press CTRL+C to quit
 * Restarting with stat
 * Debugger is active!
 * Debugger PIN: 790-193-721

How to change access privileges from 127.0.0.1 and 5000