janivanecky / depth-estimation Goto Github PK

Hi,

the script of train_augment2.py and train_augment1.py need to add

import matplotlib

to create the data.

Thank you for share this perfect job very much!

In your thesis.pdf

Does it means that output = net.blobs['depth-refine'].data is the real absolute scale?
I want to know how to get the true depth scale in meter.

Hello, I am new to Caffe. I was wondering if you had a script to read the trained caffe model and apply it on a test image. I am not sure how to get an image output after reading the caffemodel file.

Any help is appreciated.
Thank you.

Hi Janivanecky
I am not able to download the trained caffe model from the following link.
https://dl.dropboxusercontent.com/u/65091385/model_norm_abs_100k.caffemodel
Can you please provide an alternate link ?

Regards
Ajay

Question A) In your script in /dataset/test/crop.py. On lines 39-49 https://github.com/janivanecky/Depth-Estimation/blob/master/dataset/test/crop.py#L42 you have some logic that seems to be dividing the depth array by 65535.0. What is that value from? Is that the max depth value that the kinect sensor can get?

Question B) Also, are the ground truth depth maps for the gradient network scaled from [0, 255] or from [0, 10]?

Question C) If the ground truth depths for the gradient network (that are of shape 75, 54), are scaled from [0, 255], are you simply converting them to [0, 10] inside the loss function? Specifically the ScaleInvariantMeanSquaredError loss function inside /source/global_context_network/eval_depth.py. https://github.com/janivanecky/Depth-Estimation/blob/master/eval_depth.py#L44. I don't really understand what is being passed to the LogDepth() function because if you divide by 10.0 then multiply by 10.0 isn't that the same thing as multiplying by 1?

Thanks!

Hi,

I am trying to do inference of the trained model and get the depth of one of my images from KITTI dataset.

I am not quite sure how to do it. I have tried creating firstly lmdb file with only my image and the running the get_depth script but it does not work.

I have been able to run process_test.sh that calculates the depth of the nyu_depth_v2_dataset but I am not sure how to do it on new images.

Can you explain in detail please how could I obtain depth map of own image?

thanks

Hello, I am interested in your doctoral thesis. It is also interested in capturing depth information from a single image. I am a deep learning for beginners. I would like to refer to your modified Caffe source code. I wonder if it would be convenient for you to share it with me.
My email address is “[email protected]”
Thank you.

Hi @janivanecky
I have read your thesis, but I still do not grasp too much about training the net.
To train the first component, is the command like this: ./build/tools/caffe train --solver=solver.prototxt ?
If so, where should I add below lines?
solver = caffe.get_solver('solver.prototxt')
solver.net.copy_from('bvlc_alexnet.caffemodel')

Thanks for the help!
Yingjun

Hi Jan, your thesis is great, I need to have a quick try with your trained model, but it just has a .caffemodel file, and which one is the corresponding .prototxt file for this trained model?

Command windows output:

process_raw
590

 1

basement_0001a
0

 0

Found 0 depth, 0 rgb images, and 0 accel dumps.

filecount:1
filecount to process:1
Error using process_raw (line 29)
Reference to non-existent field 'rawRgbFilename'.

I check everything 10 times and cannot make it work, its like basement_0001a contain nothing but its contain all the data needed when I check from the file folder....

Thanks for your help

Can I use my data set while executing test_depth.py, if yes how can i pass my images and i'm getting below mentioned result
TOP 0 for AbsRelDiff

TOP 0 for SqrRelDiff

TOP 0 for RMSE

TOP 0 for RMSELog

TOP 0 for SIMSE

TOP 0 for Log10

TOP 0 for MVN

TOP 0 for Threshold 1.25

TOP 0 for Threshold 1.25^2

TOP 0 for Threshold 1.25^3

And also two directors created but no output in the folders

Hello,
Thanks for sharing your work on depth estimation.
I am wondering whether you have performed any experiments on the kitti data set.
I particular I am interested to know if you could provide your model trained on kitti for a fair comprison in my research work.

many thanks

Hi @janivanecky
With your guide, I can run the code now. Thanks! However, there is a warning:
/usr/lib/python2.7/dist-packages/scipy/ndimage/interpolation.py:535: RuntimeWarning: invalid value encountered in true_divide
zoom = (numpy.array(input.shape) - 1) / zoom_div

Does it matter?

By the way, why do we need ground truth depth map as an input? You know the goal is using the model to estimate depth map without providing ground truth depth map.

Thanks,
Yingjun

Hello,

It seems that it is a matlab file.
Do I need matlab environment to run process_raw.m?
Looking forward to your reply.

Hello ,

When I run poccess_raw.m, I got into several errors:

1)Error in get_synched_frames (line 20)
files = regexp(ls(sceneDir), '(\s+|\n)', 'split');
And the error shows:The 'STRING' input must be either a char row vector, a cell array of char row vectors, or a string array.

2)When I used cellstr to change it into a cell array. I got a second error:
Error using sort
Input argument must be a cell array of character vectors.

Error in get_synched_frames (line 24)
files = sort(files);

I am not really familiar with matlab, so can you give me some hints or advice about that?
By the way, I am using the matlab 2018, can it be a problem of the matlab version ? Which version did you use and Did you meet these errors too?

Hi @janivanecky,
I'm getting an error while running train.py from global_context_network folder. Error is
Check failed: registry.count(type) == 1 (0 vs. 1) Unknown layer type: data (known types: AbsVal, Accuracy,...

I've created .lmdb with a python script where original images are rgb and gt is grayscale. I'm not sure if these correct at all.

Can you guide me on these initial points,

1. How to create dataset (images dimensions, rbg or grayscall)? If you can share some sample images for original and gt that would be great.
   2 How to generate .lmdb file?

Thank you.

@janivanecky

Hi. Thanks for your great work. I have a question regarding the data processing. Why do you crop each image into the resolution of 561 x 427 in the process_row.m?

Hi, I am confused about the prototxt file named model_norm_abs_100k.prototxt, which is used for testing by runing the test_depth.py, where is this file? Can you help me? Best regards!

Hi @janivanecky ,

Thanks for your work.

I try to use your code to get the train images from your code to NYU_Depth_V2 dataset.

but there are some miss folders in the raw dataset comparing to the results from get_train_scenes.m

such as: 'book_store0000' which is in the results of get_train_scenes and have no corresponding folder in the raw dataset. I download the split-file of raw datasets and the one-only file of the raw datasets, this folder doesn't exist in both datasets.

Do you have any ideas to solve this problem?

or just ignore this folder is ok?

I am looking forward to your response.

Hi @janivanecky

1. Running test_depth.py will need eval_depth.py, but no this file in the project.
2. Beside the above, it still needs model_norm_abs_100k.prototxt, no this file. So I changed the name of net_deploy.prototxt to model_norm_abs_100k.prototxt. Not sure if this is OK.

Thanks,
Yingjun