question model about chatpdf HOT 6 CLOSED

Yes, you can use the local gguf model, you only have to replace the llm parameter from the chain. But it would be very slow, in contrast to the gpt models. You can also use gemini models, they are both free and generate good results.

from chatpdf.

oh ... 1hour service ;)

iam not that good in pyton
is it:
#generating conversation chain
def get_conversationchain(vectorstore):
llm=ChatOpenAI(temperature=0.2)

and now ?
llm=c:\temp\XYZ.gguf ???

is it possible to define temperatur as well as top_p and top_k ?
and what is the actual max response token ?

from chatpdf.

You can use AutoModel.from_pretrained() to load the model from the path and pass the required parameters.

from chatpdf.

thx for trying to help me but ?!? ^^
that means ???

llm=AutoModel.from_pretrained(C:\temP\xyz.gguf) ???

from chatpdf.

llm=AutoModel.from_pretrained("C:\temP\xyz.gguf")
like if your model is llama
llm = AutoModelForCausalLM. from_pretrained(
"codellama-13b-instruct.Q4_K_M.gguf",
model_type='llama',
max_new_tokens = 1096,
temperature = 0.1)

from chatpdf.

i see ... its for professionals ... may i take a look here in one year ^^

get some errors . . .

from chatpdf.