How to inference custom-trained model about openlrm HOT 5 CLOSED

Hi,

It looks like the same cause to another issue here #20.
Could you plz try using a newer CUDA version such as 11.8 and keeping your torch version at least 2.1.2?
I haven't encountered such problem locally so that I'm not sure where is the root cause either. Hopefully upgrading to newer versions would help.

Plz feel free to comment if it persists.

from openlrm.

This is a problem caused by too little training data in the custom trained model, but the threshold value is large. In the openlrm/runners/infer/lrm.py file infer_mesh function, the shape of vtx and faces was 0,3 due to the threshold! Adjusting the threshold to match the value of grid_out['sigma'] solved the problem! It's been resolved, so we'll close the issue. Thank you.

from openlrm.

Hi,

Could you plz share with us how you are running inference and some context on your key not found errors?

from openlrm.

Thank you for your quick response. Below are the errors that are currently occurring
gpuadmin@gpusystem:~/ihjune/OpenLRM$ python -m openlrm.launch infer.lrm --infer $INFER_CONFIG model_name=$MODEL_NAME image_input=$IMAGE_INPUT export_video=$EXPORT_VIDEO export_mesh=$EXPORT_MESH
[2024-03-27 14:55:35,448] openlrm.models.modeling_lrm: [INFO] Using DINOv2 as the encoder
/home/gpuadmin/ihjune/OpenLRM/openlrm/models/encoders/dinov2/layers/swiglu_ffn.py:43: UserWarning: xFormers is available (SwiGLU)
warnings.warn("xFormers is available (SwiGLU)")
/home/gpuadmin/ihjune/OpenLRM/openlrm/models/encoders/dinov2/layers/attention.py:27: UserWarning: xFormers is available (Attention)
warnings.warn("xFormers is available (Attention)")
/home/gpuadmin/ihjune/OpenLRM/openlrm/models/encoders/dinov2/layers/block.py:39: UserWarning: xFormers is available (Block)
warnings.warn("xFormers is available (Block)")
Loading weights from local directory
0%| | 0/1 [00:00<?, ?it/s]/home/gpuadmin/ihjune/OpenLRM/openlrm/datasets/cam_utils.py:153: UserWarning: Using torch.cross without specifying the dim arg is deprecated.
Please either pass the dim explicitly or simply use torch.linalg.cross.
The default value of dim will change to agree with that of linalg.cross in a future release. (Triggered internally at ../aten/src/ATen/native/Cross.cpp:63.)
x_axis = torch.cross(up_world, z_axis)
0%| | 0/1 [00:27<?, ?it/s]
Traceback (most recent call last):
File "/home/gpuadmin/miniconda3/envs/lrm/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/home/gpuadmin/miniconda3/envs/lrm/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/home/gpuadmin/ihjune/OpenLRM/openlrm/launch.py", line 38, in
main()
File "/home/gpuadmin/ihjune/OpenLRM/openlrm/launch.py", line 33, in main
runner.run()
File "/home/gpuadmin/ihjune/OpenLRM/openlrm/runners/infer/base_inferrer.py", line 62, in run
self.infer()
File "/home/gpuadmin/ihjune/OpenLRM/openlrm/runners/infer/lrm.py", line 284, in infer
self.infer_single(
File "/home/gpuadmin/ihjune/OpenLRM/openlrm/runners/infer/lrm.py", line 244, in infer_single
mesh = self.infer_mesh(planes, mesh_size=mesh_size, mesh_thres=mesh_thres, dump_mesh_path=dump_mesh_path)
File "/home/gpuadmin/ihjune/OpenLRM/openlrm/runners/infer/lrm.py", line 207, in infer_mesh
vtx_colors = self.model.synthesizer.forward_points(planes, vtx_tensor)['rgb'].squeeze(0).cpu().numpy() # (0, 1)
File "/home/gpuadmin/ihjune/OpenLRM/openlrm/models/rendering/synthesizer.py", line 206, in forward_points
for k in outs[0].keys()
IndexError: list index out of range

from openlrm.

Hi @eolgaeng-i , recently I've generated checkpoint model, and tried inference on my trained model.
But the result quality of the inference ply is not very good.
Could you please check my #41 when you have time?
It would be a huge help if you could check it when it possible.
Thank you in advance!

from openlrm.