Fine tuning on a dataset about openlrm HOT 7 OPEN

Hi @hayoung-jeremy , I tried to train some epochs, but i don't have enough vram (always cuda-out-of-memory) to even load the model.
In this issue #2 he said he used 32 A100 to train the model for 2 days.
Except you have a good GPU it will be imposible.

About the guideline, I followed the instructions in the readme, first prepare your data, then train. aBut as i said i can´t load the model so i don't know if I did it right.

Sorry! Good Luck.

from openlrm.

Hi,

I'm not sure what you mean by add models to directory. To enable training, root_dirs should have a directory containing multiple folders e.g. uid1, uid2, uid3, etc. And uid1 should contain rgba, pose, and intrinsics.npy.

• root_dir
• • uid1
• • • rgba
• • • pose
• • • intrinsics.npy
• • uid2
• • • ....

For finetuning, plz try to use this method.

from openlrm.

Hi @ZexinHe , thank you for the advise.
I've prepared images through Objaverse Rendering as you can see below :

However, not sure how to prepare those pose and intrinsics.npy.
I'm very new to AI, so please could you specify more details for training?

Also, I have no idea how to prepare thos meta_path's json files in train-sample.yaml :

It would be greate if you can help me.
Thanks in advance:)

from openlrm.

I'm not so sure but following the code, in openlrm/datasets/base.py in line 46 expects a file path to a json with the uuids.

So i would try to create a json file with ["uid1","uid2",...]. #33

And reference by path to that json with the path

meta_path:
train: "your_path_to_json"

from openlrm.

Thank you for your kind reply, @juanfraherrero !
I'll try it!

from openlrm.

Hi @juanfraherrero , is it possible to finetune the pretrained LRM models with my custom small dataset?
I'm currently trying overfitting with my 100 pairs of high quality glb files, but it is zero-based training.
So I wonder if it is possible to use pretrained models mentioned on the README.md such as openlrm-mix-large-1.1 etc.
I don't see any guideline for finetuning, so asking your help.
Thank you in advance!

from openlrm.

Hi @juanfraherrero , thank you for your kind reply!

I've tried fine-tuning using the base model provided by OpenLRM.
If you're interested, please take a look at this

from openlrm.