linhungshi / gcnetwork Goto Github PK

In the paper, the 3.1 talked about the Unary features, they used eight residual blocks. But in the code, the NUM_RES = 5, only loop 6 times . Is this mean that the code used six residual blocks?

Hi, I found that latest version of Keras does not have the Conv3DTranspose module needed in the Network architecture. Do you use an older version of Keras ?

I thought the cost volume is constructed by left cost features and the corresponding right cost features with deviation d. But I wonder that for Middlebury dataset, the disparity range for each image is different and some is as large as 400, so I wonder how can this be solved.

Hi, I run the test commend "python test.py -data driving_RGB_cleanpass/ -pspath driving_RGB_cleanpass/a-out/ -bs 1 " and I get the problem as follow.
Use shared weight for first stage
Traceback (most recent call last):
File "test.py", line 60, in
Predict()
File "test.py", line 42, in Predict
model = createGCNetwork(hp, tp, True)
File "src/gcnetwork.py", line 167, in createGCNetwork
disp_map = LearnReg(cv, num_filters, ksize, ds_stride, resnet, padding, highway_func, num_down_conv)
File "src/gcnetwork.py", line 114, in LearnReg
deconv = createDeconv3D(up_convs, filters, ksize, ds_stride, padding)
File "src/gcnetwork.py", line 38, in createDeconv3D
deconv = Conv3DTranspose(filters, ksize, stride, padding) (input)
File "/home/lucifer16/venv/local/lib/python2.7/site-packages/keras/engine/base_layer.py", line 457, in call
output = self.call(inputs, **kwargs)
File "src/conv3dTranspose.py", line 237, in call
self.padding)
TypeError: deconv_length() takes at least 5 arguments (4 given)

The problem seems occurs in the line 237 of "src/conv3dTranspose.py".
The code is "out_depth = conv_utils.deconv_length(depth, stride_h, kernel_h, self.padding)". It passes 4 arguments.

But the source code in Keras github about the "conv_utils.deconv_length" is as follows. And it passes 5arguments.

def deconv_length(dim_size, stride_size, kernel_size, padding,
output_padding, dilation=1):
"""Determines output length of a transposed convolution given input length.
# Arguments
dim_size: Integer, the input length.
stride_size: Integer, the stride along the dimension of dim_size.
kernel_size: Integer, the kernel size along the dimension of dim_size.
padding: One of "same", "valid", "full".
output_padding: Integer, amount of padding along the output dimension, Can be set to None in
which case the output length is inferred.
dilation: dilation rate, integer.
link : https://github.com/keras-team/keras/blob/master/keras/utils/conv_utils.py

I use the "Keras==2. 2. 4". Shall I use a older version or any other suggestions?

Your prompt response will be most appreciated.

Hi Lin, thanks for your work for implementing the GCNetwork.

I try to run test.py to test the driving dataset. With the original image size the memory will run out and show ResourceExhaustedError. So I downsample the images size by two (480 * 270), however the following error will occur:

InvalidArgumentError (see above for traceback): Incompatible shapes: [1,12,18,30,64] vs. [1,12,17,30,64]
[[Node: add_17/add = Add[T=DT_FLOAT, _device="/job:localhost/replica:0/task:0/gpu:0"](activation_32/Relu, activation_28/Relu)]]
[[Node: model_2/lambda_4/Squeeze/_969 = _Recvclient_terminated=false, recv_device="/job:localhost/replica:0/task:0/cpu:0", send_device="/job:localhost/replica:0/task:0/gpu:0", send_device_incarnation=1, tensor_name="edge_6294_model_2/lambda_4/Squeeze", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/cpu:0"]]

It is related to the downsamples in the 3D convolutions? Hope you can help.

Hi,
Congrats for your nice work. I am trying to run the training script but I get the following error:
"TypeError: Value passed to parameter 'paddings' has DataType float32 not in list of allowed values: int32, int64"
in gcnetwork.py, when calling
cv = Lambda(getCostVolume, arguments = {'max_d':d/2}, output_shape = (d/2, None, None, num_filters * 2))(unifeature)
Any suggestion?
Thanks,
Julian

文章中说用prescaling解决多模问题，在代码中没看到。是我看漏了吗？

def createFeature(input, layers):
res = layers0
tensor = res
for layer in layers[1:]:
if isinstance(layer, Add):
tensor = layer([tensor, res])
res = tensor
else:
tensor = layer(tensor)
return tensor

Why doesn't this need to be modified like this? Sorry, I am a new novice and I hope to get your reply.

def createFeature(input, layers):
res = layers0
tensor = res
for layer in layers[1:]:
for lay in layer:
if isinstance(lay, Add):
tensor = lay([tensor, res])
res = tensor
else:
tensor = lay(tensor)
return tensor

thanks for your work! i download it and want to test it performance. but after the command. i found no picture on my result folder.
this is my command,
"python test.py -data ./pic"
(the 'pic ' is the folder where i put my test pictures in 'left' and 'right',such as"./pic/left/0l.png"and "./pic/right/0r.png")

the content on the 'test_params.json' are:
"{
"pspath": "./res",
"batch_size": 1,
"w_path": "./model/pretrained_model_weight.hdf5",
"max_q_size": 3,
"verbose": 1
}
"
the content on the 'util_param.json' are:
"{
"crop_width": 128,
"crop_height": 96,
"new_max": 1,
"new_min": -1,
"old_max": 256,
"old_min": 0,
"val_ratio": 0.1,
"file_extension": "png",
"seed": 1234,
"fraction": 1
}
"
thank you for your help!

Hi,Is it possible to get disparity of both left and right image instead of one disparity ?
In case it doesn't can anyone suggest me a way to get both disparities from output disparity
thanks in advance

Dear Lin, have you got the error when test.py
The test commmad is: python test.py -data data/test -pspath prediction
The results are:
File "test.py", line 67, in
Predict()
File "test.py", line 47, in Predict
model = createGCNetwork(hp, tp, True)
File "src\gcnetwork.py", line 134, in createFeature
tensor = layer(tensor)
.......
ValueError: Variable batchnormalization_1_running_mean/biased already exists, disallowed. Did you mean to set reuse=True in VarScope? Originally defined at:

hope for your reply , Thanks.

In test.py, it seems that you have made a careless mistake:
pred = _predictFromArrays_(model, images[left], images[1], images[2], bs), while the function only has 4 arguments. def _predictFromArrays_(model, left, right, bs)

In order to make final layer (soft argmin) don't limits the output value.
I use linear combinations to predict. (my result is good with your pre-trained model in softargmin)

my hyperparams:
{
"max_disp": 64,
"base_num_filters": 32,
"first_kernel_size": 5,
"kernel_size": 3,
"num_res": 8,
"num_down_conv": 4,
"resnet": 1,
"output": "linear",
"act_func": "relu",
"h_act_func": "sigmoid",
"ds_stride": 2,
"padding": "same",
"data_format": "channels_last"
}

but the result is very strange. I print out the shape and min max

Shape: (1, 320, 320)
Min: -37094.2 Max 4468.41
Shape After squeeze: (320, 320)

It seems the output range of GCNetwork is wrong....

Any suggestion?
Thanks,
Fang

Issue Description:
createGCNetwork throws an TypeError

my clone version(the newest):
684ad7f

my cmd:
python3 test.py -data data/test_256 -pspath result
(the 'test_256' is the folder where i put my test pictures in 'left' and 'right',such as"./data/test/left/left_0.png"and "./data/test/right/right_0.png")

note:

1. I find that you disable the "test_params.json" in parseArguments
2. my test image is 256 x 256 3 channel color image

Error Output:

Use shared weight for first stage
Traceback (most recent call last):
File "test.py", line 62, in
Predict()
File "test.py", line 44, in Predict
model = createGCNetwork(hp, tp, True)
File "src/gcnetwork.py", line 166, in createGCNetwork
cv = Lambda(getCostVolume, arguments = {'max_d':d/2}, output_shape = (d/2, None, None, num_filters * 2))(unifeatures)
File "/usr/local/lib/python3.5/dist-packages/keras/engine/topology.py", line 602, in call
output = self.call(inputs, **kwargs)
File "/usr/local/lib/python3.5/dist-packages/keras/layers/core.py", line 650, in call
return self.function(inputs, **arguments)
File "src/gcnetwork.py", line 52, in getCostVolume
right_tensor = K.spatial_2d_padding(right_tensor, padding=((0, 0), (max_d, 0)))
File "/usr/local/lib/python3.5/dist-packages/keras/backend/tensorflow_backend.py", line 2020, in spatial_2d_padding
return tf.pad(x, pattern)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/array_ops.py", line 1679, in pad
return gen_array_ops._pad(tensor, paddings, name=name)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/gen_array_ops.py", line 1989, in _pad
name=name)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/op_def_library.py", line 589, in apply_op
param_name=input_name)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/op_def_library.py", line 60, in _SatisfiesTypeConstraint
", ".join(dtypes.as_dtype(x).name for x in allowed_list)))
TypeError: Value passed to parameter 'paddings' has DataType float32 not in list of allowed values: int32, int64

how can I solve it to finish the predict flow? thanks for any suggestion~~~

Hi,
I've download your code and would like to do testing first.
When running test.py, I got the error
"ValueError: The channel dimension of the inputs should be defined. Found None."

I run it by the argument list: -data ./data -pspath ./output -wpath ./model/shared_cost_weight.hdf5
I found
cv = Lambda(getCostVolume, arguments = {'max_d':int(d/2)}, output_shape = (int(d/2), None, None, num_filters * 2))(unifeatures)
takes several arguments but the function tooks only the max_d instead of all the parameters.
how could I solve this, thanks!

Could you tell me that can this implementation achieves the result authors shown in GC-NET paper?

README.md says - "Model performance can't acheive that in the original paper."

Can you please share what is the current performance you the model achieves?

Thanks!

Hi,
I've been trying to use your re-implementation, although, I stumbled across a couple errors.
First, I got the "'module' object has no attribute 'conv3d_transpose'", I fixed it copying your function to the tensorflow_backend library (I assumed you have done the same?) Maybe could be an idea to include a comment about this in the readme (to people relatively new to keras)?

Second, when trying to load your pretrained model, I got the error:
TypeError: Output tensors to a Model must be Keras tensors. Found: <keras.engine.training.Model object at 0x7fbcef09f450>

I fixed it by commenting line 28 in the main.py as it seems this line loads a model based on an already loaded model?
Finally, I wanted to ask which architecture you used for your pre-trained model and if it is supposed be used 'out-of-the-box'? Since it contains 107 layers.

Best,