Unable to load and run inference on finetuned Alpaca model about trlx HOT 4 CLOSED

@doyled-it that should be the case, yes

from trlx.

Thanks @doyled-it for opening this issue! Can confirm this is the case and will try to resolve it shortlly.

from trlx.

@maxreciprocate thanks for looking into it! I also forgot to mention that in order for it to load, I have to copy in the model's config.json into the checkpoint directory.

from trlx.

Thank you @maxreciprocate for merging that so quickly. I see that it does now in fact load when I point to the hf_model directory and give me an output that makes sense:

path = Path("path/to/best_checkpoint/hf_model")
pipe = pipeline(
    "text2text-generation",
    model=str(path),
    tokenizer="declare-lab/flan-alpaca-gpt4-xl",
)

text = pipe("tell me a joke")

print(text)

Output:

[{'generated_text': "Why don't scientists trust atoms? Because they make up everything!"}]

Just to make sure: With PR #551, trlx is now saving a trained version of the model with updated weights? And that's what I'm loading when I use the code above?

from trlx.