Logging/Saving Settings and Instructions for Inference Jobs about h2ogpt HOT 6 CLOSED

Alot of information is stored in the save directory in history.json for every inference job. It has everything you mentioned, except not perhaps as much detail -- e.g.no input files, but just whether there are input files.

In addition, vLLM or TGI can also change its logging.

from h2ogpt.

Alot of information is stored in the save directory in history.json for every inference job. It has everything you mentioned, except not perhaps as much detail -- e.g.no input files, but just whether there are input files.

I cant seem to find history.json under /save, is there perhaps a run option that I'm missing?

from h2ogpt.

--save_dir=foo will place it in foo directory relative to startup in repo.

from h2ogpt.

--save_dir=foo will place it in foo directory relative to startup in repo.

Great! It works now.

though I'm missing the docs/chunks used in the job. I really think it needs to be added.
I have noticed that the docs being loaded when using the h2oGPT interface is different than those used when requesting the same instruction through a gradio client(i inferred this from the fact that the "num_prompt_tokens" value is higher when handling gradio client requests), it's causing my model to behave differently (worse in my case)

from h2ogpt.

Hi, the API call and history.json does contain 'save_dict['sources']` as the list of sources used.

from h2ogpt.

I see, thanks!

from h2ogpt.