lmb-freiburg / mv3d Goto Github PK

If I just run:

python3 nobg_nodm.py

in the root working directory, I get the follow error:

Traceback (most recent call last):
  File "nobg_nodm.py", line 5, in <module>
    import utils.realtime_renderer as rtr
  File "/home/michael/mv3d/utils/realtime_renderer.py", line 1, in <module>
    from renderer import *
ModuleNotFoundError: No module named 'renderer'

If I move everything out of utils into the root working directory and change:

import utils.realtime_renderer as rtr

from utils.tf_utils import *

to:

import realtime_renderer as rtr

from tf_utils import *

it makes it past this error.

I run the nobg_nodm.py , it has something wrong:
Traceback (most recent call last):
File "nobg_nodm.py", line 191, in
net.test(0)
File "nobg_nodm.py", line 136, in test
log_value(self.writer, total_loss, 'test_loss', global_iter)
AttributeError: mv3d instance has no attribute 'writer'
I run in test mode and I downloaded the data and snapshots.What's the problem?

This commit did not fix this issue for me. Adding:

self.writer = None

to __init__ in mv3d makes it past this error.

When the Renderer is initialized with depth set to False, dm_unit is never set in renderView even though it is always returned.

From what I could find on the Panda3D forums, the obj2egg utility does not support textures/.mtl files. So, for example, converting the ShapeNet object ShapeNetCore/02958343/114b662c64caba81bb07f8c2248e54bc (a "NYPD Highway Patrol Dodge Charger") to a BAM file using:

obj2egg model_normalized.obj model_normalized.egg
egg2bam model_normalized.egg model_normalized.bam

and then running the following code:

from PIL import Image
from renderer import Renderer
from panda3d.core import VBase4

renderer = Renderer(False, False)
renderer.alight.setColor(VBase4(1, 1, 1, 1))
renderer.models = renderer.loader.loadModel(["model_normalized.bam"])
renderer.selectModel(0)

lights = [[3.5, 0, 60, 13]]

rad = 2.0
el = 15
az = 60
blur = 0
blending = 0

(im, dm) = renderer.renderView(
    [rad, el, az], lights, blur, blending, default_bg_setting=0
)

image = Image.fromarray(im)
image.show()

produces this as output:

@treeform's forked obj2egg tool found here seems to add support for textures, but has not been updated in almost 10 years. Can you share the exact steps you were doing to convert the objects?

Hi, after reading the paper, I have some questions

1. about the label

The paper used three parameters, azimuth, elevation and distance to express the labels. I assume rad is the distance. when we use both sin and cos to express the angle, is this way to make the angle unique ? why can't we express angle as angle = angle / 360 to make it between [0-1] or use sin() or cos() to express the angle ?

the code is in https://github.com/lmb-freiburg/mv3d/blob/master/utils/tf_utils.py#L159

2. the loss function

https://github.com/lmb-freiburg/mv3d/blob/master/bg_nodm.py#L92

        gt_cm = tf.slice(self.images2, [0, 0, 0, 0], [64, 128, 128, 3])
        gt_sm = tf.slice(self.images2, [0, 0, 0, 3], [64, 128, 128, 1])
        sm = gt_sm
        gt_sm = gt_sm * 0.75
        pr_cm = tf.slice(self.gen, [0, 0, 0, 0], [64, 128, 128, 3])
        pr_sm = tf.slice(self.gen, [0, 0, 0, 3], [64, 128, 128, 1])

        self.loss = euclidean_loss(tf.mul(gt_cm, sm), tf.mul(pr_cm, sm)) +\
            0.1 * euclidean_loss(gt_sm, pr_sm)

in the first euclidean_loss(tf.mul(gt_cm, sm), tf.mul(pr_cm, sm)), gt_cm is the ground truth in color whereas sm is the ground truth in depth, pr_cm is the predication in color , In my understanding it should be like euclidean_loss(tf.mul(pr_cm, sm), tf.mul(pr_cm, sm))

in the second euclidean_loss(gt_sm, pr_sm), gt_sm is the 0.75 * ground truth in color, pr_sm is the predicted depth, why these two are used in loss?

3. about the model

in the convolution layer, the channel are increased from 16,32,64,128,256, my question is why the lower layer has less channels (the lower layer describes more detail feature ) while the higher layers has more channels (the higher layers has more semantic information)

Thanks

Hi Team,
Can you please share the trained model for generating chairs also using mv3d and tensorflow?
need it for a project, really appreciate it, thanks

Hello, I am a novice in 3D learning. I want to know how to get training data. Thank you.

When I run:

python3 nobg_nodm.py

in test mode, I get the following error:

loading test set...
done.
WARNING:tensorflow:From nobg_nodm.py:86 in buildModel.: scalar_summary (from tensorflow.python.ops.logging_ops) is deprecated and will be removed after 2016-11-30.
Instructions for updating:
Please switch to tf.summary.scalar. Note that tf.summary.scalar uses the node name instead of the tag. This means that TensorFlow will automatically de-duplicate summary names based on the scope they are created in. Also, passing a tensor or list of tags to a scalar summary op is no longer supported.
loading snapshots/nobg_nodm/snapshot435000-435000...
done.
Traceback (most recent call last):
  File "nobg_nodm.py", line 186, in <module>
    net.test(0)
  File "nobg_nodm.py", line 120, in test
    self.labels: batch_labels})
  File "/home/michael/.pyenv/versions/3.6.6/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 766, in run
    run_metadata_ptr)
  File "/home/michael/.pyenv/versions/3.6.6/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 943, in _run
    % (np_val.shape, subfeed_t.name, str(subfeed_t.get_shape())))
ValueError: Cannot feed value of shape (0,) for Tensor 'input_images:0', which has shape '(64, 128, 128, 3)'

Seems like it's probably related to this. I'll see if I can figure it out.

ValueError: Dimension 1 in both shapes must be equal, but are 4096 and 64. Shapes are [64,4096] and [64,64].
From merging shape 0 with other shapes. for 'concat/concat_dim' (op: 'Pack') with input shapes: [64,4096], [64,64].

is this because of tensorflow version, I cant install tensorflow= 0.12 on windows (which is requirement for this code) , but using tensorflow 1.14.0 currently

can anyone please help me on this

Hi, I'm attempting to re-implement the code from the mv3d work as part of a different project. I am using the script utils/render_test_set.py (with modifications for locating the file paths) to render examples, but while the depth maps appear to render correctly, the rgb images render as simple black and white silhouettes. Any insight on why this may be the case? We are using OpenGL, the default rendering mode for Panda3D.

One of the obj models:

Rendered output:

Hi, your work is impressive. When I read the paper, I found inconsistent fc layer output size ( 1024 in the paper vs 4096 in the code). So which one do you choose finally.

百度base找不到，pip也一直下不下来，这是个啥库呀？

还有这个renderer.py中的第88行，render.setLight(plnp)也报错，说没有render

Im facing this issue, please help t resolve the same