Hi,

I got warnings for importing few classes from langchain. Seems that some classes were deprecated from langchain and moved to langchain_community. (Reference)

Example Warning:

LangChainDeprecationWarning: Importing document loaders from langchain is deprecated. Importing from langchain will no longer be supported as of langchain==0.2.0. Please import from langchain-community instead:

from langchain_community.document_loaders import WikipediaLoader.

To install langchain-community run pip install -U langchain-community.

I got rid of such warnings by importing few clasess from langchain_community instead of langchain in main_driver.py.
from langchain_community.document_loaders import WikipediaLoader
from langchain_community.vectorstores import Chroma

And for importing VertexAIEmbeedings, the warning was:

LangChainDeprecationWarning: The class langchain_community.embeddings.vertexai.VertexAIEmbeddings was deprecated in langchain-community 0.0.12 and will be removed in 0.2.0. An updated version of the class exists in the langchain-google-vertexai package and should be used instead. To use it run pip install -U langchain-google-vertexai and import as from langchain_google_vertexai import VertexAIEmbeddings.
warn_deprecated(

So, installing langchain-google-vertexai and using below import resolved the above warning.
from langchain_google_vertexai import VertexAIEmbeddings

Perhaps needs an update in the master branch code of ask_wikipedia to use the langchain_community for imports of relevant classes?

'Ask Bharat' chatbot is using simple sentence splitting. Need to have some semantic, theme based, or at least section based segmentation.

Source: https://github.com/yogeshhk/Sarvadnya/tree/master/src/ask_bharat

'Ask Bharat' chatbot is not working well. Need to try different open-source llms and test the chatbot thoroughly.

Source: https://github.com/yogeshhk/Sarvadnya/tree/master/src/ask_bharat

'Ask Bharat' chatbot is using single call LLM. Need to shift to Chat/Conversational LLM, so that related/coreference queries can be handled.

e.g.

• What are the archeological sites in India?
• What are their locations?

Source: https://github.com/yogeshhk/Sarvadnya/tree/master/src/ask_bharat

Update all chatbots to work on open-source models such as Llama and comment usage of openai or any such paid API.
Don't delete the code. As sometimes anyone with requisite credentials can uncomment and start using it.

Employ a bit advance techniques for segmentation and reranking in 'ask_bharat' bot.

Reference can be https://lightning.ai/lightning-ai/studios/document-search-and-retrieval-using-rag?view=public&section=recent

Code: https://github.com/yogeshhk/Sarvadnya/tree/master/src/ask_manim

Currently it uses openai

    response = openai.ChatCompletion.create(
        model=openai_model.lower(),
        messages=[
            {"role": "system", "content": GPT_SYSTEM_INSTRUCTIONS},
            {"role": "user", "content": wrap_prompt(prompt)}
        ],
        max_tokens=max_tokens
    )

Need to remove dependancy on Open AI (and its KEY), and add use of open source models like Lllama 2, in place:

from langchain_community.llms import CTransformers

    # Load the locally downloaded model here
    llm = CTransformers(
        model="../models/TheBloke/Llama-2-7B-Chat-GGML",
        model_type="llama",
        max_new_tokens=1024,
        temperature=0.5
    )

May need to have corresponding Chat Model LLM.

'Ask Bharat' chatbot is using simple bold formatting. Need to make it pretty, with hyperlinks to locations in the doc

Source: https://github.com/yogeshhk/Sarvadnya/tree/master/src/ask_bharat

Upgrade all repos to the latest langchain, make them running, add requirements txt wherever missing