fyviezhao / m3d-vton Goto Github PK
View Code? Open in Web Editor NEWOfficial code for ICCV2021 paper "M3D-VTON: A Monocular-to-3D Virtual Try-on Network"
Official code for ICCV2021 paper "M3D-VTON: A Monocular-to-3D Virtual Try-on Network"
thanks for your reply. this is great job! i have another question that what's the DRM and TFM's network? i can't find any detailed description in papers. are they using gan network? what can i refer to. i look forward to your reply, thank u.
hi,thans for your great job. i am very curious about this project and how to generate depth. Is there some code to refer. hope your reply!
Hi,
Thanks for your work,
How long does it take to tune and train the end-to-end network with one GPU?
Hi,
Thanks for your work.
When I used your images with the same resolution, the output of rgbd2pcl which is .ply is not completely painted and there are some parts that are masked. Is there anything that should apply after the rgbd2pck before mesh lab?
Thanks for your interesting paper called "M3D-VTON: A Monocular-to-3D Virtual Try-On Network". I'm also involved in such a similar area and I need to know how could I extract the ground truth depth (.npy files) for 2D images. Could you please guide me in this regard?
My next question is that how you catch the camera specification from 2D images?
Hi,
How did you use gif maker of the mesh results to create gif output?
rgbd2pc could also work for lower resolution?
The .ply for my case is incomplete.
Thanks for your swift response.
File ".../utils/util.py", line 100, in depth2normal_ortho
p = torch.stack([x_cord, y_cord, depth], dim=3) # (B,H,W,3)
RuntimeError: stack expects each tensor to be equal size, but got [8, 256, 192] at entry 0 and [5, 256, 192] at entry 2
I've got this error when I used batch size 8 for DRM.
What is the batch size for DRM?
Thanks for your quick answer.
Hi,
why the store action is true in DRM? It does not include in the command line for training. However, FND and BND are dependent on this argument.
parser.add_argument('--add_gan_loss', action='store_true', help='if specified, use gan loss')
Thanks
We are getting graphic card related error.
Here are my system info
pavan@u-20:~/.../M3D-VTON$ sudo dmidecode -t 1
# dmidecode 3.2
Getting SMBIOS data from sysfs.
SMBIOS 3.0.0 present.
Handle 0x000C, DMI type 1, 27 bytes
System Information
Manufacturer: LENOVO
Product Name: 20JUS05X00
Version: ThinkPad L470 W10DG
Serial Number: PF11SM82
UUID: 31f6b94c-2fd0-11b2-a85c-ed0533b508ea
Wake-up Type: Power Switch
SKU Number: LENOVO_MT_20JU_BU_Think_FM_ThinkPad L470 W10DG
Family: ThinkPad L470 W10DG
pavan@u-20:~/.../M3D-VTON$
pavan@u-20:~/.../M3D-VTON$ inxi -G
Graphics: Device-1: Intel Skylake GT2 [HD Graphics 520] driver: i915 v: kernel
Display: x11 server: X.Org 1.20.13 driver: i915 resolution: 1366x768~60Hz
OpenGL: renderer: Mesa Intel HD Graphics 520 (SKL GT2) v: 4.6 Mesa 21.2.6
pavan@u-20:~/.../M3D-VTON$
pavan@u-20:~/.../M3D-VTON$ python3 test.py --model MTM --name MTM --dataroot mpv3d_example --datalist test_pairs --results_dir results
verbose: False
----------------- End -------------------
Traceback (most recent call last):
File "test.py", line 25, in <module>
opt = TestOptions().parse() # get test options
File "/home/pavan/Documents/aux/tmp-git/virtual-try-on/M3D-VTON/options/base_options.py", line 130, in parse
torch.cuda.set_device(opt.gpu_ids[0])
File "/home/pavan/.local/lib/python3.8/site-packages/torch/cuda/__init__.py", line 313, in set_device
torch._C._cuda_setDevice(device)
File "/home/pavan/.local/lib/python3.8/site-packages/torch/cuda/__init__.py", line 216, in _lazy_init
torch._C._cuda_init()
RuntimeError: Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver from http://www.nvidia.com/Download/index.aspx
pavan@u-20:~/.../M3D-VTON$
is it possible to run it on my machine? I have attached my machine info also.
Possion Remeshing is no longer available in the new version
I prepared my custom test data
and has ran the command
python util/data_preprocessing.py --MPV3D_root mpv3d_example
with mpv3d_example/train_pairs.txt https://github.com/kaijung/M3D-VTON/blob/main/mpv3d_example/train_pairs.txt
and then ran the MTM command
python test.py --model MTM --name MTM --dataroot mpv3d_example --datalist test_pairs --results_dir results
and got RuntimeError
dataset [AlignedMPV3dDataset] was created
initialize network with normal
model [MTMModel] was created
loading the model from pretrained/aligned/MTM/latest_net_MTM.pth
---------- Networks initialized -------------
[Network MTM] Total number of parameters : 40.003 M
-----------------------------------------------
/usr/local/lib/python3.8/dist-packages/torch/nn/functional.py:4227: UserWarning: Default grid_sample and affine_grid behavior has changed to align_corners=False since 1.3.0. Please specify align_corners=True if the old behavior is desired. See the documentation of grid_sample for details.
warnings.warn(
Traceback (most recent call last):
File "test.py", line 63, in <module>
model.test() # run inference
File "/content/M3D-VTON/models/base_model.py", line 110, in test
self.forward()
File "/content/M3D-VTON/models/MTM_model.py", line 151, in forward
self.output = self.netMTM(self.agnostic, self.c)
File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1190, in _call_impl
return forward_call(*input, **kwargs)
File "/content/M3D-VTON/models/networks.py", line 768, in forward
featureA = self.extractionA(inputA) # featureA: size (batch_size,512,32,20)
File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1190, in _call_impl
return forward_call(*input, **kwargs)
File "/content/M3D-VTON/models/networks.py", line 99, in forward
return self.model(x)
File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1190, in _call_impl
return forward_call(*input, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/container.py", line 204, in forward
input = module(input)
File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1190, in _call_impl
return forward_call(*input, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/conv.py", line 463, in forward
return self._conv_forward(input, self.weight, self.bias)
File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/conv.py", line 459, in _conv_forward
return F.conv2d(input, weight, bias, self.stride,
RuntimeError: Given groups=1, weight of size [64, 29, 4, 4], expected input[1, 30, 512, 320] to have 29 channels, but got 30 channels instead
[ ]
Hello there, first of all thank you for this project/source code. I didn't face any issue regarding this. However, I could not understand the this issue related to custom data. The custom t-shirt is perfectly working on your dataset but not mine. I have create custom dataset of image but not working. Please revert whenever you get time. Thank You.
I 'm sorry to bother you again. You are so kind-hearted, thank you again.
I just happened the probelm that warp cloth saved by MTM model seems smaller when i train MTM in MPV3D Dataset with the same param configures. But the warp cloth generated by your pretrained MTM model is larger. I analysis for a long time, but i can't find the reason. Please help me! Best wish with you.
Thank you for perfect repository
I am following the readme to run on custom dataset but i get this error (man_person.png is my input image)
when I run MTM model
FileNotFoundError: [Errno 2] No such file or directory: 'mpv3d_example/depth/man_person_depth.npy'
Can you help me fix this error?
Hi. I ran the repo on the test dataset. I am unable to visualize the output files (.ply) in blender/Online 3D viewer. The errors are "Invalid header ('end_header' line not found!) Invalid file" and "unable to import model, face not present" , respectively.
Hi ,Zhao, Thanks for sharing your interesting work. I downloaded M3D dataset and tried to run python rgbd2pcd.py
to get point clouds using depth maps of the dataset. I set flag label='gt' , but got a strange result.
Here is the front-view rgb.png and the generated rgb_back.png
And here is the mesh I got following your instructions on MeshLab.
Besides, why we use fd = -1.0 * (2.0 * fd - 1.0) rather than fd = 2.0 * fd - 1.0 to transform the fd coordinates? Can you give me some tips to fix that ? really thank.
Can you please let me know how to create custom data for testing?
I see this has many files mpv3d_example.
Also does your code hav anything to do with SMPL
I assume the end result is just the 3d model with no SMPL and with poses
Correct?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.