fyviezhao / m3d-vton Goto Github PK

thanks for your reply. this is great job! i have another question that what's the DRM and TFM's network? i can't find any detailed description in papers. are they using gan network? what can i refer to. i look forward to your reply, thank u.

During training, this error occurs！Do you know how to solve?

hi，thans for your great job. i am very curious about this project and how to generate depth. Is there some code to refer. hope your reply!

Hi,

Thanks for your work,

How long does it take to tune and train the end-to-end network with one GPU?

Hi,

Thanks for your work.

When I used your images with the same resolution, the output of rgbd2pcl which is .ply is not completely painted and there are some parts that are masked. Is there anything that should apply after the rgbd2pck before mesh lab?

Hi, this is absolutely amazing work! I tried to run the test demo according to all your instructions and here is the result I obtained.
May I know if these small artifacts present in the mesh are normal? If so, what might be the reason causing these artifacts?

Thanks for your interesting paper called "M3D-VTON: A Monocular-to-3D Virtual Try-On Network". I'm also involved in such a similar area and I need to know how could I extract the ground truth depth (.npy files) for 2D images. Could you please guide me in this regard?
My next question is that how you catch the camera specification from 2D images?

Hi,
How did you use gif maker of the mesh results to create gif output?

rgbd2pc could also work for lower resolution?
The .ply for my case is incomplete.

Thanks for your swift response.

File ".../utils/util.py", line 100, in depth2normal_ortho
p = torch.stack([x_cord, y_cord, depth], dim=3) # (B,H,W,3)
RuntimeError: stack expects each tensor to be equal size, but got [8, 256, 192] at entry 0 and [5, 256, 192] at entry 2

I've got this error when I used batch size 8 for DRM.

What is the batch size for DRM?

Thanks for your quick answer.

Hi,

why the store action is true in DRM? It does not include in the command line for training. However, FND and BND are dependent on this argument.

parser.add_argument('--add_gan_loss', action='store_true', help='if specified, use gan loss')

Thanks

We are getting graphic card related error.
Here are my system info

pavan@u-20:~/.../M3D-VTON$ sudo dmidecode -t 1
# dmidecode 3.2
Getting SMBIOS data from sysfs.
SMBIOS 3.0.0 present.

Handle 0x000C, DMI type 1, 27 bytes
System Information
	Manufacturer: LENOVO
	Product Name: 20JUS05X00
	Version: ThinkPad L470 W10DG
	Serial Number: PF11SM82
	UUID: 31f6b94c-2fd0-11b2-a85c-ed0533b508ea
	Wake-up Type: Power Switch
	SKU Number: LENOVO_MT_20JU_BU_Think_FM_ThinkPad L470 W10DG
	Family: ThinkPad L470 W10DG

pavan@u-20:~/.../M3D-VTON$
pavan@u-20:~/.../M3D-VTON$ inxi -G
Graphics:  Device-1: Intel Skylake GT2 [HD Graphics 520] driver: i915 v: kernel 
           Display: x11 server: X.Org 1.20.13 driver: i915 resolution: 1366x768~60Hz 
           OpenGL: renderer: Mesa Intel HD Graphics 520 (SKL GT2) v: 4.6 Mesa 21.2.6 
pavan@u-20:~/.../M3D-VTON$ 

pavan@u-20:~/.../M3D-VTON$ python3 test.py --model MTM --name MTM --dataroot mpv3d_example --datalist test_pairs --results_dir results
                  verbose: False                         
----------------- End -------------------
Traceback (most recent call last):
  File "test.py", line 25, in <module>
    opt = TestOptions().parse()  # get test options
  File "/home/pavan/Documents/aux/tmp-git/virtual-try-on/M3D-VTON/options/base_options.py", line 130, in parse
    torch.cuda.set_device(opt.gpu_ids[0])
  File "/home/pavan/.local/lib/python3.8/site-packages/torch/cuda/__init__.py", line 313, in set_device
    torch._C._cuda_setDevice(device)
  File "/home/pavan/.local/lib/python3.8/site-packages/torch/cuda/__init__.py", line 216, in _lazy_init
    torch._C._cuda_init()
RuntimeError: Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver from http://www.nvidia.com/Download/index.aspx
pavan@u-20:~/.../M3D-VTON$

is it possible to run it on my machine? I have attached my machine info also.

Possion Remeshing is no longer available in the new version

I prepared my custom test data

1. https://github.com/kaijung/M3D-VTON/blob/main/mpv3d_example/image/BAEL-K11%403%3Dperson_whole_front.png image P 320*512
2. https://github.com/kaijung/M3D-VTON/blob/main/mpv3d_example/image-parse/BAEL-K11%403%3Dperson_whole_front_label.png 2D-Human-Paring on P;
3. https://github.com/kaijung/M3D-VTON/blob/main/mpv3d_example/pose/BAEL-K11%403%3Dperson_whole_front_keypoints.json OpenPose (25 keypoints) on P;

and has ran the command

python util/data_preprocessing.py --MPV3D_root mpv3d_example 

with mpv3d_example/train_pairs.txt https://github.com/kaijung/M3D-VTON/blob/main/mpv3d_example/train_pairs.txt

and then ran the MTM command

python test.py --model MTM --name MTM --dataroot mpv3d_example --datalist test_pairs --results_dir results

and got RuntimeError

dataset [AlignedMPV3dDataset] was created
initialize network with normal
model [MTMModel] was created
loading the model from pretrained/aligned/MTM/latest_net_MTM.pth
---------- Networks initialized -------------
[Network MTM] Total number of parameters : 40.003 M
-----------------------------------------------
/usr/local/lib/python3.8/dist-packages/torch/nn/functional.py:4227: UserWarning: Default grid_sample and affine_grid behavior has changed to align_corners=False since 1.3.0. Please specify align_corners=True if the old behavior is desired. See the documentation of grid_sample for details.
  warnings.warn(
Traceback (most recent call last):
  File "test.py", line 63, in <module>
    model.test()            # run inference
  File "/content/M3D-VTON/models/base_model.py", line 110, in test
    self.forward()
  File "/content/M3D-VTON/models/MTM_model.py", line 151, in forward
    self.output = self.netMTM(self.agnostic, self.c)
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1190, in _call_impl
    return forward_call(*input, **kwargs)
  File "/content/M3D-VTON/models/networks.py", line 768, in forward
    featureA = self.extractionA(inputA) # featureA: size (batch_size,512,32,20)
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1190, in _call_impl
    return forward_call(*input, **kwargs)
  File "/content/M3D-VTON/models/networks.py", line 99, in forward
    return self.model(x)
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1190, in _call_impl
    return forward_call(*input, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/container.py", line 204, in forward
    input = module(input)
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1190, in _call_impl
    return forward_call(*input, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/conv.py", line 463, in forward
    return self._conv_forward(input, self.weight, self.bias)
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/conv.py", line 459, in _conv_forward
    return F.conv2d(input, weight, bias, self.stride,
RuntimeError: Given groups=1, weight of size [64, 29, 4, 4], expected input[1, 30, 512, 320] to have 29 channels, but got 30 channels instead
[ ]

Hello there, first of all thank you for this project/source code. I didn't face any issue regarding this. However, I could not understand the this issue related to custom data. The custom t-shirt is perfectly working on your dataset but not mine. I have create custom dataset of image but not working. Please revert whenever you get time. Thank You.

I 'm sorry to bother you again. You are so kind-hearted, thank you again.
I just happened the probelm that warp cloth saved by MTM model seems smaller when i train MTM in MPV3D Dataset with the same param configures. But the warp cloth generated by your pretrained MTM model is larger. I analysis for a long time, but i can't find the reason. Please help me! Best wish with you.

During training, this error occurs！Do you know how to solve?

After I declared the variable I got another error

Thank you for perfect repository
I am following the readme to run on custom dataset but i get this error (man_person.png is my input image)
when I run MTM model

FileNotFoundError: [Errno 2] No such file or directory: 'mpv3d_example/depth/man_person_depth.npy'

Can you help me fix this error?

Hi. I ran the repo on the test dataset. I am unable to visualize the output files (.ply) in blender/Online 3D viewer. The errors are "Invalid header ('end_header' line not found!) Invalid file" and "unable to import model, face not present" , respectively.

During training MTM model，can you tell me how to fix it?

Hi ,Zhao, Thanks for sharing your interesting work. I downloaded M3D dataset and tried to run python rgbd2pcd.py to get point clouds using depth maps of the dataset. I set flag label='gt' , but got a strange result.
Here is the front-view rgb.png and the generated rgb_back.png

And here is the mesh I got following your instructions on MeshLab.

Besides, why we use fd = -1.0 * (2.0 * fd - 1.0) rather than fd = 2.0 * fd - 1.0 to transform the fd coordinates? Can you give me some tips to fix that ? really thank.

Can you please let me know how to create custom data for testing?
I see this has many files mpv3d_example.
Also does your code hav anything to do with SMPL
I assume the end result is just the 3d model with no SMPL and with poses
Correct?