pixegami / rag-tutorial-v2 Goto Github PK
View Code? Open in Web Editor NEWAn Improved Langchain RAG Tutorial (v2) with local LLMs, database updates, and testing.
An Improved Langchain RAG Tutorial (v2) with local LLMs, database updates, and testing.
def get_embedding_function():
embeddings = BedrockEmbeddings(
credentials_profile_name="default", region_name="us-east-1"
)
# embeddings = OllamaEmbeddings(model="nomic-embed-text")
return embeddings
Can we have similar function using Azure OPENAI
Wondering where is the text passed to this embedding? Sorry novice
Hi Jack,
First off, kudos on your excellent tutorial - it's been a real asset.
I'm eager to leverage it for a project I'm working on.
However, before I proceed, I noticed there isn't a LICENSE file attached to this repository.
Adding one would provide clarity on how your work can be used and built upon by others.
Would you mind considering this addition? It would be greatly appreciated.
Thanks again for your great work!
Best regards,
Chitoku
Haven't solved this yet but working on it.
C:\PycharmProjects\rag-tutorial-v2-main.venv\Scripts\python.exe C:/PyChar/plugins/python-ce/helpers/pycharm/_jb_pytest_runner.py --path C:\PycharmProjects\rag-tutorial-v2-main\test_rag.py
Testing started at 12:10 a.m. ...
Launching pytest with arguments C:\PycharmProjects\rag-tutorial-v2-main\test_rag.py --no-header --no-summary -q in C:\PycharmProjects\rag-tutorial-v2-main
============================= test session starts =============================
collecting ...
test_rag.py:None (test_rag.py)
test_rag.py:1: in
from query_data import query_rag
query_data.py:2: in
from langchain.vectorstores.chroma import Chroma
.venv\Lib\site-packages\langchain\vectorstores_init_.py:23: in
from langchain_core.vectorstores import VectorStore
.venv\Lib\site-packages\langchain_core\vectorstores.py:44: in
from langchain_core.retrievers import BaseRetriever
.venv\Lib\site-packages\langchain_core\retrievers.py:29: in
from langchain_core.documents import Document
.venv\Lib\site-packages\langchain_core\documents_init_.py:6: in
from langchain_core.documents.compressor import BaseDocumentCompressor
.venv\Lib\site-packages\langchain_core\documents\compressor.py:6: in
from langchain_core.callbacks import Callbacks
.venv\Lib\site-packages\langchain_core\callbacks_init_.py:22: in
from langchain_core.callbacks.manager import (
.venv\Lib\site-packages\langchain_core\callbacks\manager.py:29: in
from langsmith.run_helpers import get_run_tree_context
.venv\Lib\site-packages\langsmith\run_helpers.py:40: in
from langsmith import client as ls_client
.venv\Lib\site-packages\langsmith\client.py:52: in
from langsmith import env as ls_env
.venv\Lib\site-packages\langsmith\env_init_.py:3: in
from langsmith.env.runtime_env import (
.venv\Lib\site-packages\langsmith\env_runtime_env.py:10: in
from langsmith.utils import get_docker_compose_command
.venv\Lib\site-packages\langsmith\utils.py:31: in
from langsmith import schemas as ls_schemas
.venv\Lib\site-packages\langsmith\schemas.py:69: in
class Example(ExampleBase):
.venv\Lib\site-packages\pydantic\v1\main.py:286: in new
cls.try_update_forward_refs()
.venv\Lib\site-packages\pydantic\v1\main.py:807: in try_update_forward_refs
update_model_forward_refs(cls, cls.fields.values(), cls.config.json_encoders, localns, (NameError,))
.venv\Lib\site-packages\pydantic\v1\typing.py:554: in update_model_forward_refs
update_field_forward_refs(f, globalns=globalns, localns=localns)
.venv\Lib\site-packages\pydantic\v1\typing.py:520: in update_field_forward_refs
field.type = evaluate_forwardref(field.type_, globalns, localns or None)
.venv\Lib\site-packages\pydantic\v1\typing.py:66: in evaluate_forwardref
return cast(Any, type_)._evaluate(globalns, localns, set())
E TypeError: ForwardRef._evaluate() missing 1 required keyword-only argument: 'recursive_guard'
collected 0 items / 1 error
!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
============================== 1 error in 0.84s ===============================
Process finished with exit code 2
langchain_community
is used for the Ollama model and the embedding function, but not included in the requirements.txt
I'd like to stick to using AWS for this project rather than Ollama.
The code that invokes the model in query_data.py is
model = Ollama(model="mistral")
response_text = model.invoke(prompt)
Any idea how I'd invoke the bedrock model in a similar manner? I've tried variations of the following but no luck yet...
model_id = BedrockLLM(model_id="amazon.titan-embed-text-v1")
response_text = model_id.invoke(prompt)
Followed the example & embedding is not working (give back empty sources)
If anyone has any clue would really appreciate it!
In the video, he said something about explicitly adding the ids for the items, what does that mean and how do I do that?
HI Great video, I am trying to use OpenAI Embedding and keep running into a problem, can you help?
from openai import OpenAI
import os
client = OpenAI()
def get_embedding_function(text, model="text-embedding-3-small"):
text = text.replace("\n", " ")
return client.embeddings.create(input = [text], model=model).data[0].embedding
This is my updated code for the get_embedding_function.py
script.
The embedding need input but i cannot see it being passed to it in the script in populate_database.py
Thanks in advance ๐ Jan
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.