What're the modifications in `llava/train/transformers_replace`? about vila HOT 6 OPEN

We have manually changed some original implementations to better support grouping strategy and flash attn, and recommend every VILA user to do so.

Though our codebase should work with higher version transformer, we haven't tested throughfully thus cannot promise anything. Please use v4.36.2 for reproducement.

from vila.

I found "transformers_version": "4.38.1", in the config.json of VILA-2.7B. Which version of transformers should we use to run VILA-2.7B? By the way, is there any plan to allow loading VILA models using transformers API only without relying on the current repo?

from vila.

I noticed that you mainly manually implemented class LlamaForCausalLM(LlamaPreTrainedModel) in I noticed that you mainly manually implemented class LlamaForCausalLM(LlamaPreTrainedModel) in replace. however, in the code (either train or infer), your implemented class is not be used. What's wrong with it?
Thanks for your reply

from vila.

@Lyken17 : Could you please share some details for this ?

from vila.

please use 4.36.2 for now, we will upgrade to 4.37 in next release :)

from vila.

Also why manually replace the files?
Why not use monkeypatching, or reregister your versions with transformers using exist_ok=True?

from vila.