yzcjtr / geonet Goto Github PK

HI:
Thank you for your work！
I read your code, but I didn't find out how to get a real-world depth image.
Looking forward to your reply.
Thank you！

Hi All,
I am trying to use this code for depth estimation in biomedical imaging. I seem to have run into a strange error.
I am running this code on a google cloud instance of a p100 gpu with CUDA 9.0 and python 3.5 installed.
The model starts training, then after a couple thousand iterations, it throws this error. I processed my biomedical imaging data the same way the KITTI data was processed.

Thanks!
Daniel Borders

Unfortunately I didn't save the terminal output from the GeoNet error, but when I tried to run with SfM learner it threw the same error. I have attached that terminal output below. If it would help, I can re-run GeoNet and post the output.
terminalOutput .txt

Thanks for your impressive work. I have some questions. I tested the pre-trained depth model on the camera pose estimation task and I got 0.0087 ATE with 0.0053 std, which is much better than the pre-trained pose model. Moreover, I checked the trajectory on sequence 9 of KITTI and the result is better as well. Why it was not relevant to be reported?

Hi, @yzcjtr , great work!

I can reproduce the depth results as reported in your paper, but the pose results is not very well, so let me double check the training setting: the pose task is trained on kitti odometry dataset, and the sequence length is 5. Do you use pre-trained dispnet or just train from scratch? Is there anything I am missing?

I am also wondering why your pose results can be so well, even outperform other papers with more constraints (like https://arxiv.org/abs/1802.05522 with 3D constraints), is there any insight you can share? What's the key components for pose task here? Thanks!

Hello, I would like to generate a video like the one shared in your google drive.
https://drive.google.com/drive/folders/1Avh8ApcV1mjBTZTo_sNWPIWMl8DedSF2

Does any of your code allows me to do so? dump processed image sequence?

@yzcjtr
hello ,i want to ask one more question ,when i test optical flow in testing process. As your constructions:
python kitti_eval/generate_multiview_extension.py --dataset_dir=/path/to/data_scene_flow_multiview/ --calib_dir=/path/to/data_scene_flow_calib/ --dump_root=/path/to/formatted/testdata/ --cam_id=02 --seq_length=3
where can i download the data_scene_flow_calib files --calib_dir=/path/to/data_scene_flow_calib
i search it in raw_data_KITTI ,KITTI_flow_2015_dataset and multi-view_extension dataset ,but i get nothing,can you provide me the download link?or do i misunderstanding your constructions?

thank you very much!

For
python geonet_main.py --mode=test_flow --dataset_dir=/path/to/formatted/testdata/ --init_ckpt_file=/path/to/trained/model/ --flownet_type=direct --batch_size=1 --output_dir=/path/to/save/predictions/

Where is the pre-trained model? I know in Google Drive, there are a few pre-trained models. I tried all of them, none of them seem work, all complain with following error:

data loss: not an sstable (bad magic number): perhaps your file is in a different file format and you need to use a different restore operator?

If I decide to train from scratch, I need to use the following ( I think), but which data I need to use?

python geonet_main.py --mode=train_flow --dataset_dir=/path/to/formatted/data/ --checkpoint_dir=/path/to/save/ckpts/ --learning_rate=0.0002 --seq_length=3 --flownet_type=direct --max_steps=400000

Thank you!!

For training, the following command is required to enter:

python geonet_main.py --mode=train_rigid --dataset_dir=/path/to/formatted/data/ --checkpoint_dir=/path/to/save/ckpts/ --learning_rate=0.0002 --seq_length=3 --batch_size=4 --max_steps=350000

However, there is one parameter for indicating dataset but we have download depth and pose datasets. What should we enter here?

Also in residual training case, there should be 2 checkpoints as an input. However, it seems that 1 input is required. Can you explain this point, please?

Thanks in advance.

Hi, I read your paper recently and found your code here. It's really a fantastic work. I have a question about the part of calculating flow consistency.
`self.fwd_flow_diff_pyramid = [tf.abs(self.bwd2fwd_flow_pyramid[s] + self.fwd_full_flow_pyramid[s]) for s in range(opt.num_scales)]

self.bwd_flow_diff_pyramid = [tf.abs(self.fwd2bwd_flow_pyramid[s] + self.bwd_full_flow_pyramid[s]) for s in range(opt.num_scales)]`
Does this mean to compute the full flow difference by forward-backward consistency check? Then, why the operation is '+' to calculate the difference, instead of '-' ?

can you tell me the format of the downloadable datasets.I download the datasets from the offical website.But when I preprocess the data,it is wrong.

Hi, @yzcjtr , great work!
I have some questions.
How do I get the average displacement of pixels between two frames in an optical flow network? How can i testing optical network validate on own data set? Does the data set have to be formatted before testing?
thank you very much!

@yinzhichao：
什么时候上传源代码，很期待！

Hi,
I have a question. I only have one NVIDIA 1080Ti(11GB) GPU. Is a NVIDIA 1080Ti(11GB) GPU enough to train the whole network(flow+depth+pose)?
Best,
Zhai

Hey,

Great work! when i download your pretrained models(geonet_dirflownet), and testing the Optical Flow.
But here's the mistake:
Unsuccessful TensorSliceReader constructor: Failed to find any matching files for ./geonet_dirflownet/
How do I solve this?

Thank you for your time and patience.

when I run geonet_main.py in ubuntu 16.04LTS, it output as below, how shoule I do?

from ._conv import register_converters as _register_converters
Traceback (most recent call last):
File "geonet_main.py", line 166, in
tf.app.run()
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/platform/app.py", line 125, in run
_sys.exit(main(argv))
File "geonet_main.py", line 147, in main
opt.num_source = opt.seq_length - 1
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/platform/flags.py", line 88, in setattr
return self.dict['__wrapped'].setattr(name, value)
File "/usr/local/lib/python2.7/dist-packages/absl/flags/_flagvalues.py", line 496, in setattr
return self._set_unknown_flag(name, value)
File "/usr/local/lib/python2.7/dist-packages/absl/flags/_flagvalues.py", line 374, in _set_unknown_flag
raise _exceptions.UnrecognizedFlagError(name, value)
absl.flags._exceptions.UnrecognizedFlagError: Unknown command line flag 'num_source'

Hi, I have read your GeoNet paper and code. But I have a question about geometric consistency.
In build_rigid_flow_warping and data_loader part:
fwd_rigid_flow = compute_rigid_flow(tf.squeeze(self.pred_depth[s][:bs], axis=3),
self.pred_poses[:,i,:], self.intrinsics[:,s,:,:], False)
bwd_rigid_flow = compute_rigid_flow(tf.squeeze(self.pred_depth[s][bs*(i+1):bs*(i+2)], axis=3),
self.pred_poses[:,i,:], self.intrinsics[:,s,:,:], True)
In the unpack_image_sequence for data_loader, assuming the center image is the target frame, means if a sequence has 2 batches for each have 3 frames A:[a1, a2, a3] and B:[b1, b2, b3], the a2 and b2 are selected target frames. Then the standards for the fwd_rigid_flow and bwd_rigid_flow are a2 and b2, but they are different, how to check the geometric consistency?

hello,
Can you provide a requirement.txt file of your project?

data/prepare_train_data.py 的94行是不是写错了？dump_example函数的参数没有传进去吧？
Parallel(n_jobs=args.num_threads)(delayed(dump_example)(n) for n in range(data_loader.num_train)

I just checked the loss of GeoNet on training, and its seems that it has a high variance. How did you selected your best model for evaluation on depth and camera motion tasks?.

hello ,I have some doubts.
Have you tried to use more than one images as input to predict depth or what's the advantages and significance of singe-view depth prediction? From some discussion, there is no true depth for a single image.
one more,I think that single view depth has more problems generalizing to previously unseen types of images. For example ,a model trained on outdoor driving sequences is unlikely to work well on indoor scenes ?
what do you think of them ?

Thank you for this wonderful work! I managed to run the pose estimation on the Kitti data set with good results.
First, I was wondering what the 5 poses on each frame mean (Is it possible to do a prediction based on 2 frames?)
Second, would it be possible to run on different data sets? If so what is needed to change in order to do so.
Lastly, I would like to know if there is a place where I could modify the field of view used by the camera.

Thank you again!

how can i get the leftImg8bit_sequence_trainvaltest.zip which is 324G,i need to contact to the team of cityscapes lab?my silly question

Hi, could I ask if you tried to use seq_length=2 to train the depth & pose network, which is the minimal case for SfM? Does it work well? Thank you.

I am running Geonet on Tensorflow 1.1, CUDA 8.0 and Ubuntu 16.04.

After training the model using command given in train_rigid mode for pose tasks, I am running camera pose test with command given in the Testing - Camera Pose section. The first command

python geonet_main.py --mode=test_pose --dataset_dir=/path/to/kitti/odom/dataset/ --init_ckpt_file=/path/to/trained/model/ --batch_size=1 --seq_length=5 --pose_test_seq=9 --output_dir=/path/to/save/predictions/

returns the following invalid argument error

Traceback (most recent call last):
File "geonet_main.py", line 166, in
tf.app.run()
File "/home/ubuntu/anaconda3/envs/py27/lib/python2.7/site-packages/tensorflow/python/platform/app.py", line 48, in run
_sys.exit(main(_sys.argv[:1] + flags_passthrough))
File "geonet_main.py", line 161, in main
test_pose(opt)
File "/home/ubuntu/GeoNet/geonet_test_pose.py", line 42, in test_pose
saver.restore(sess, opt.init_ckpt_file)
File "/home/ubuntu/anaconda3/envs/py27/lib/python2.7/site-packages/tensorflow/python/training/saver.py", line 1457, in restore
{self.saver_def.filename_tensor_name: save_path})
File "/home/ubuntu/anaconda3/envs/py27/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 778, in run
run_metadata_ptr)
File "/home/ubuntu/anaconda3/envs/py27/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 982, in _run
feed_dict_string, options, run_metadata)
File "/home/ubuntu/anaconda3/envs/py27/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1032, in _do_run
target_list, options, run_metadata)
File "/home/ubuntu/anaconda3/envs/py27/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1052, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Assign requires shapes of both tensors to match. lhs shape= [1,1,256,24] rhs shape= [1,1,256,12]
[[Node: save/Assign_29 = Assign[T=DT_FLOAT, _class=["loc:@pose_net/Conv_7/weights"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/gpu:0"](pose_net/Conv_7/weights, save/RestoreV2_29/_3)]]
[[Node: save/RestoreV2_4/_48 = _SendT=DT_FLOAT, client_terminated=false, recv_device="/job:localhost/replica:0/task:0/gpu:0", send_device="/job:localhost/replica:0/task:0/cpu:0", send_device_incarnation=1, tensor_name="edge_141_save/RestoreV2_4", _device="/job:localhost/replica:0/task:0/cpu:0"]]

Caused by op u'save/Assign_29', defined at:
File "geonet_main.py", line 166, in
tf.app.run()
File "/home/ubuntu/anaconda3/envs/py27/lib/python2.7/site-packages/tensorflow/python/platform/app.py", line 48, in run
_sys.exit(main(_sys.argv[:1] + flags_passthrough))
File "geonet_main.py", line 161, in main
test_pose(opt)
File "/home/ubuntu/GeoNet/geonet_test_pose.py", line 26, in test_pose
saver = tf.train.Saver([var for var in tf.model_variables()])
File "/home/ubuntu/anaconda3/envs/py27/lib/python2.7/site-packages/tensorflow/python/training/saver.py", line 1056, in init
self.build()
File "/home/ubuntu/anaconda3/envs/py27/lib/python2.7/site-packages/tensorflow/python/training/saver.py", line 1086, in build
restore_sequentially=self._restore_sequentially)
File "/home/ubuntu/anaconda3/envs/py27/lib/python2.7/site-packages/tensorflow/python/training/saver.py", line 691, in build
restore_sequentially, reshape)
File "/home/ubuntu/anaconda3/envs/py27/lib/python2.7/site-packages/tensorflow/python/training/saver.py", line 419, in _AddRestoreOps
assign_ops.append(saveable.restore(tensors, shapes))
File "/home/ubuntu/anaconda3/envs/py27/lib/python2.7/site-packages/tensorflow/python/training/saver.py", line 155, in restore
self.op.get_shape().is_fully_defined())
File "/home/ubuntu/anaconda3/envs/py27/lib/python2.7/site-packages/tensorflow/python/ops/state_ops.py", line 270, in assign
validate_shape=validate_shape)
File "/home/ubuntu/anaconda3/envs/py27/lib/python2.7/site-packages/tensorflow/python/ops/gen_state_ops.py", line 47, in assign
use_locking=use_locking, name=name)
File "/home/ubuntu/anaconda3/envs/py27/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py", line 768, in apply_op
op_def=op_def)
File "/home/ubuntu/anaconda3/envs/py27/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 2336, in create_op
original_op=self._default_original_op, op_def=op_def)
File "/home/ubuntu/anaconda3/envs/py27/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1228, in init
self._traceback = _extract_stack()

InvalidArgumentError (see above for traceback): Assign requires shapes of both tensors to match. lhs shape= [1,1,256,24] rhs shape= [1,1,256,12]
[[Node: save/Assign_29 = Assign[T=DT_FLOAT, _class=["loc:@pose_net/Conv_7/weights"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/gpu:0"](pose_net/Conv_7/weights, save/RestoreV2_29/_3)]]
[[Node: save/RestoreV2_4/_48 = _SendT=DT_FLOAT, client_terminated=false, recv_device="/job:localhost/replica:0/task:0/gpu:0", send_device="/job:localhost/replica:0/task:0/cpu:0", send_device_incarnation=1, tensor_name="edge_141_save/RestoreV2_4", _device="/job:localhost/replica:0/task:0/cpu:0"]]

Any idea how to fix this error? Any help would be appreciated.

Thanks

Hi,
Im running the pose-test mode with the pretrained weights
the output is a txt file per frame with 8 numbers per row (time, x,y,z,qx,qy,qz,qw)
but when I want to evaluate results - KITTI poses are per sequence and each row is a 3X4 projection matrix
meaning, the kitti_eval can't work

Am I missing something? I followed the instuction in the GitHub (test + evaluate)

I can not open the Google Drive (https://drive.google.com/open?id=1VSGpdMrQ3dFKdher_2RteDfz7F0g57ZH) when I wanna download the pretained model and predictions. Is there any other websites can I download?

Thanks for the wonderful work. I just tried it out and the results are amazing. I can totally reproduce the results reported in your paper.

But may you also provide the pre-trained pose model of sequence length 3? Because if I want to resume training of train_rigid, I need two pre-trained models both in sequence length 3, otherwise the fully connected layer parameter dimension doesn't match. So if you could also provide that posenet, that would be very helpful. Thank you very much.

In the tutorial, you mention that

Note that for replicating depth and pose results, the --seq_length is suggested to be 3 and 5 respectively.

However in your source code, depth and pose estimation are trained together. How can you set different seq_length for the whole pipeline? I'm confused for a while.

Hi,
In your paper, you said training the depth and pose network takes around 30 epochs. Training the residual flow network takes around 200 epochs. I have prepared the data for training using prepare_train_data.py file. How many iterations are 30 epochs and 200 epochs? I saw the examples in Readme are 350000 iterations and 400000 iterations.
Best,
Zhai

Hi,
In your paper, I saw the visualization results of depth map. However, when I run the eval_depth.py. I only obtain the error of evaluation and can not obtain the visualization results of depth map.
On flow task, I run the eval_flow.py. I can obtain the flow color map. So, how to visualize the depth map?
Best,
Mingliang

Hi, thank you for publishing the GeoNet code.
I have a minor question about the input of ResFlowNet.
In the code, the input size of the flownet is (4*b) x h x w x 12, where 'b' stands for the batch size and 12 is the channel (tgt, src, wrp, flow, err - stacked) size.
The number of batch size is multiplied by 4, because there are forward & backward directions I think.
Is there a specific reason why you didn't concatenate only in the channel-wise direction?
Isn't it enough with the number of 21 channels for the input (ie, b x h x w x 21) of flownet?

21channels: I_t, I_{t-1}, I_{t+1}, W_{t-1}, W_{t+1}, F_{t,t-1}, F_{t,t+1}, E_{t-1}, E_{t+1}
, where, I: images (3ch), W: warped images (3ch), F: flows (2ch), E: similarity errors (1ch)

Hi,
I would like to ask about the training strategy of geonet.

1. To reproduce the result of depth, I use the kitti_raw_eigen split to train the depth and pose networks, 30 epoch.
2. To reproduce the result of the residual flow, I first use the kitti_raw_stereo split to train the depth and pose networks, 30 epoch, and then I use the kitti_raw_stereo split to train the residual flow, about 200 epoch.

Is the whole process like this? or

1. Reproduce the result of depth, use kitti_raw_eigen split to train depth and pose networks, 30 epoch
2. To reproduce the result of the residual flow, I first use the kitti_raw_eigen split to train the depth and pose networks, 30 epoch, and then use kitti_raw_stereo split to train the residual flow, about 200 epoch.

I used depth's training strategy to reproduce the results of depth. However, training the residual flow is a bit worse than the paper results. Can you help me to see which one is wrong?

1:I get the KITTI_odometry_grondtruth_poses data from tinghuiz/SfMLearner which only consists of 10 txt files,but there are 21 sequences in KITTI odometry dataset ,do i get the wrong groundtruth pose data?
2:in the process of training flow,when the iteration is 15200,the problem is :

Iteration: [ 14700] | Time: 0.3787s/iter | Loss: 1.504
2018-07-01 17:51:40.590857: W tensorflow/core/framework/op_kernel.cc:1192] Invalid argument: Expected image (JPEG, PNG, or GIF), got empty file
[[Node: DecodeJpeg = DecodeJpegacceptable_fraction=1, channels=0, dct_method="", fancy_upscaling=true, ratio=1, try_recover_truncated=false, _device="/job:localhost/replica:0/task:0/device:CPU:0"]]
Iteration: [ 14800] | Time: 0.3839s/iter | Loss: 1.462
Iteration: [ 14900] | Time: 0.3801s/iter | Loss: 1.380
Iteration: [ 15000] | Time: 0.3811s/iter | Loss: 1.538
Iteration: [ 15100] | Time: 0.4323s/iter | Loss: 1.279
Iteration: [ 15200] | Time: 0.3785s/iter | Loss: 1.250

2018-07-01 17:55:13.338006: W tensorflow/core/framework/op_kernel.cc:1192] Out of range: RandomShuffleQueue '_2_shuffle_batch/random_shuffle_queue' is closed and has insufficient elements (requested 4, current size 0)
[[Node: shuffle_batch = QueueDequeueManyV2[component_types=[DT_UINT8, DT_UINT8, DT_FLOAT], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/device:CPU:0"](shuffle_batch/random_shuffle_queue, shuffle_batch/n)]]
2018-07-01 17:55:13.338261: W tensorflow/core/framework/op_kernel.cc:1192] Out of range: RandomShuffleQueue '_2_shuffle_batch/random_shuffle_queue' is closed and has insufficient elements (requested 4, current size 0)
[[Node: shuffle_batch = QueueDequeueManyV2[component_types=[DT_UINT8, DT_UINT8, DT_FLOAT], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/device:CPU:0"](shuffle_batch/random_shuffle_queue, shuffle_batch/n)]]
2018-07-01 17:55:13.338448: W tensorflow/core/framework/op_kernel.cc:1192] Out of range: RandomShuffleQueue '_2_shuffle_batch/random_shuffle_queue' is closed and has insufficient elements (requested 4, current size 0)
[[Node: shuffle_batch = QueueDequeueManyV2[component_types=[DT_UINT8, DT_UINT8, DT_FLOAT], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/device:CPU:0"](shuffle_batch/random_shuffle_queue, shuffle_batch/n)]]
Traceback (most recent call last):
File "geonet_main.py", line 166, in
tf.app.run()
File "/home/ubuntu/.local/lib/python2.7/site-packages/tensorflow/python/platform/app.py", line 48, in run
_sys.exit(main(_sys.argv[:1] + flags_passthrough))
File "geonet_main.py", line 157, in main
train()
File "geonet_main.py", line 143, in train
saver.save(sess, os.path.join(opt.checkpoint_dir, 'model'), global_step=step)
File "/usr/lib/python2.7/contextlib.py", line 35, in exit
self.gen.throw(type, value, traceback)
File "/home/ubuntu/.local/lib/python2.7/site-packages/tensorflow/python/training/supervisor.py", line 964, in managed_session
self.stop(close_summary_writer=close_summary_writer)
File "/home/ubuntu/.local/lib/python2.7/site-packages/tensorflow/python/training/supervisor.py", line 792, in stop
stop_grace_period_secs=self._stop_grace_secs)
File "/home/ubuntu/.local/lib/python2.7/site-packages/tensorflow/python/training/coordinator.py", line 389, in join
six.reraise(*self._exc_info_to_raise)
File "/home/ubuntu/.local/lib/python2.7/site-packages/tensorflow/python/training/queue_runner_impl.py", line 238, in _run
enqueue_callable()
File "/home/ubuntu/.local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1231, in _single_operation_run
target_list_as_strings, status, None)
File "/home/ubuntu/.local/lib/python2.7/site-packages/tensorflow/python/framework/errors_impl.py", line 473, in exit
c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.InvalidArgumentError: Expected image (JPEG, PNG, or GIF), got empty file
[[Node: DecodeJpeg = DecodeJpegacceptable_fraction=1, channels=0, dct_method="", fancy_upscaling=true, ratio=1, try_recover_truncated=false, _device="/job:localhost/replica:0/task:0/device:CPU:0"]]

where prepare_training_flow:--dataset_dir=raw_data_KITTI/ --dataset_name=kitti_raw_stereo(if I set it as kitti_raw_eigen,there is nothing wrong)
and train_flow:--dataset_dir=resulting/formatted/data_flow/ -flownet_type=direct

i don't know why

Hey,

Great work! However, I have encountered an problem of how to find the accumulated error of predicted positions. The ATE used in the program seems only considers the relatively error in each snippets. I want to find out the accumulated deviation in position between ground truth position and the predicted position from first images in the dataset. How can I achieve this?

Thank you for your time and patience.

Hi, i am trying to reproduce your results with the pretrained model you provided. And I find that, the posenet has very good translation error but large error in rotation. Is there something wrong?

Hi. I am trying to train the first stage from scratch for the depth prediction. I used the preprocess code (kitti_raw_eigen, seq=3) and train code(batch_size=4, steps=350k. lr=0.0002) suggested in README.md but could not get similar result in paper. The model I used for evaluation is in the 345000th step. I am not sure is that the best model to use. Though I can use the pretrained model to get the result, I am still curious about how to train from scratch and is there anything we need to pay attention to during the training? Thanks.

Hi,

First of all, thanks a lot for sharing your code. I am eager to learn details about your impressive work!

I tried to reproduce the results in the GeoNet paper by running the depth/pose evaluation using the pretrained models. As described in the README, I downloaded "depthnet/dirflownet/posenet/resflownet" from the provided GoogleDrive links. Then I ran the following simple shell script to run evaluation, e.g.,:

DATASET_DIR=~/Data/KITTI
INIT_CHECKPOINT_DIR=~/Workspace/GeoNet/models/geonet_depthnet/model.index
OUTPUT_DIR=./Testing_log/depth
python geonet_main.py --mode=test_depth --dataset_dir=$DATASET_DIR --init_ckpt_file=$INIT_CHECKPOINT_DIR --batch_size=1 --depth_test_split=eigen --output_dir=$OUTPUT_DIR

Then I got the following error:

Tensor name "pose_net/Conv/BatchNorm/beta" not found in checkpoint files ~/Workspace/GeoNet/models/geonet_posenet/model.index

I think either I am missing something simple or there's some mismatch in the uploaded python code and the pretrained model. (I got the same error for POSENET evaluation...) I guess this issue could be something any novice like me would suffer! Could you kindly help me on this?

Best regards,
Jungwook

I just preparing the data followed by your code prepare_trin_data.py

But, dump_root path rose the error while executing that code as like this.

python data/prepare_train_data.py --dataset_dir=dataset/raw_data_downloader/ --dataset_name=kitti_raw_eigen --dump_root=dataset/dump/ --seq_length=3 --img_height=128 --img_width=416 --num_threads=8 --remove_static

I attached the TrackBack error log below.

jhm@vclab:~/GeoNet-master/GeoNet-master$ python data/prepare_train_data.py --dataset_dir=dataset/raw_data_downloader/ --dataset_name=kitti_raw_eigen --dump_root=dataset/dump/ --seq_length=3 --img_height=128 --img_width=416 --num_threads=8 --remove_static
Progress 0/45016....
Traceback (most recent call last):
  File "data/prepare_train_data.py", line 110, in <module>
    main()
  File "data/prepare_train_data.py", line 92, in main
    Parallel(n_jobs=args.num_threads)(delayed(dump_example)(n) for n in range(data_loader.num_train))
  File "/home/jhm/miniconda2/lib/python2.7/site-packages/joblib/parallel.py", line 934, in __call__
    self.retrieve()
  File "/home/jhm/miniconda2/lib/python2.7/site-packages/joblib/parallel.py", line 862, in retrieve
    raise exception.unwrap(this_report)
joblib.my_exceptions.JoblibAttributeError: JoblibAttributeError
___________________________________________________________________________
...........................................................................
/home/jhm/GeoNet-master/GeoNet-master/data/prepare_train_data.py in <module>()
    105                     if np.random.random() < 0.1:
    106                         vf.write('%s %s\n' % (s, frame))
    107                     else:
    108                         tf.write('%s %s\n' % (s, frame))
    109
--> 110 main()
    111

...........................................................................
/home/jhm/GeoNet-master/GeoNet-master/data/prepare_train_data.py in main()
     87         data_loader = cityscapes_loader(args.dataset_dir,
     88                                         img_height=args.img_height,
     89                                         img_width=args.img_width,
     90                                         seq_length=args.seq_length)
     91
---> 92     Parallel(n_jobs=args.num_threads)(delayed(dump_example)(n) for n in range(data_loader.num_train))
     93
     94     # Split into train/val
     95     np.random.seed(8964)
     96     subfolders = os.listdir(args.dump_root)

...........................................................................
/home/jhm/miniconda2/lib/python2.7/site-packages/joblib/parallel.py in __call__(self=Parallel(n_jobs=8), iterable=<generator object <genexpr>>)
    929                 # No need to wait for async callbacks to trigger to
    930                 # consumption.
    931                 self._iterating = False
    932
    933             with self._backend.retrieval_context():
--> 934                 self.retrieve()
        self.retrieve = <bound method Parallel.retrieve of Parallel(n_jobs=8)>
    935             # Make sure that we get a last message telling us we are done
    936             elapsed_time = time.time() - self._start_time
    937             self._print('Done %3i out of %3i | elapsed: %s finished',
    938                         (len(self._output), len(self._output),

---------------------------------------------------------------------------
Joblib worker traceback:
---------------------------------------------------------------------------
AttributeError                                     Wed Feb 13 22:07:02 2019
PID: 27037                   Python 2.7.15: /home/jhm/miniconda2/bin/python
...........................................................................
/home/jhm/miniconda2/lib/python2.7/site-packages/joblib/parallel.py in __call__(self=<joblib.parallel.BatchedCalls object>)
    220     def __call__(self):
    221         # Set the default nested backend to self._backend but do not set the
    222         # change the default number of processes to -1
    223         with parallel_backend(self._backend, n_jobs=self._n_jobs):
    224             return [func(*args, **kwargs)
--> 225                     for func, args, kwargs in self.items]
        func = <function dump_example>
        args = (1,)
        kwargs = {}
        self.items = [(<function dump_example>, (1,), {})]
    226
    227     def __len__(self):
    228         return self._size
    229

...........................................................................
/home/jhm/miniconda2/lib/python2.7/site-packages/joblib/externals/loky/backend/popen_loky_posix.py in dump_example(n=1)
     37     intrinsics = example['intrinsics']
     38     fx = intrinsics[0, 0]
     39     fy = intrinsics[1, 1]
     40     cx = intrinsics[0, 2]
     41     cy = intrinsics[1, 2]
---> 42     dump_dir = os.path.join(args.dump_root, example['folder_name'])
     43
     44     try:
     45         os.makedirs(dump_dir)
     46     except OSError:

AttributeError: 'Namespace' object has no attribute 'dump_root'

How can I prepare the dump_root file path?
Thanks

hello ,in Figure 5 of your paper,GeoNet shows clear advantages in occluded ,texture ambiguous regions,and even in shaded dim area.
but i can't see the comparison of GeoNet Predictions and other methods like DirFlowNetsS,can you show it in the figure ,and where is the difference between GeoNet Error and DirFlowNetsS Error? i can't see the advantages of GeoNet.can you show the accurate data and show more pictures (not just chose the best 4 input)

cool man

Hi, thanks a lot for your promising work, but I am strongly confused about the data preparation for three different tasks, where you said we should to run 'prepare_train_data.py' to get training data with three types -- e.g., kitti_raw_stereo, kitti_raw_eigen and kitti_odom. Using different type, we could get three types training data that are totally different. However, the 'dump_root' is the same, so which type should I choose to train the first stage? Or I just need to choose one of them randomly? Thanks a lot!

Hi, is there a PyTorch implementation of the paper instead of TensorFlow? Thanks!

Hi, @yzcjtr ,

As the deadline of ECCV had passed, would you release the code of GeoNet?

THX!

Sorry,I am the beginner of tensorflow. When I prepared data ,and then I open terminal.

$ python data/prepare_train_data.py --dataset_dir=/home/panc/dataset/ --dataset_name=kitti_odom --dump_root=/home/panc/formatted/data/ --seq_length=5 --img_height=128 --img_width=416 --num_threads=16 --remove_static

The first step was successful
BUT when I trained the dataset , an error occurs. I have no idea to solve it .

$ python geonet_main.py --mode=train_rigid --dataset_dir=/home/panc/formatted/data/ --checkpoint_dir=/home/panc/save/ckpts/ --learning_rate=0.0002 --seq_length=5 --batch_size=4 --max_steps=350000

x_steps=350000
{'add_dispnet': True,
'add_flownet': False,
'add_posenet': True,
'alpha_recon_image': 0.85,
'batch_size': 4,
'checkpoint_dir': '/home/panc/save/ckpts/',
'dataset_dir': '/home/panc/formatted/data/',
'depth_test_split': 'eigen',
'disp_smooth_weight': 0.5,
'dispnet_encoder': 'resnet50',
'flow_consistency_alpha': 3.0,
'flow_consistency_beta': 0.05,
'flow_consistency_weight': 0.2,
'flow_smooth_weight': 0.2,
'flow_warp_weight': 1.0,
'flownet_type': 'residual',
'img_height': 128,
'img_width': 416,
'init_ckpt_file': None,
'learning_rate': 0.0002,
'max_steps': 350000,
'max_to_keep': 20,
'mode': 'train_rigid',
'num_scales': 4,
'num_source': 4,
'num_threads': 32,
'output_dir': None,
'pose_test_seq': 9,
'rigid_warp_weight': 1.0,
'save_ckpt_freq': 5000,
'scale_normalize': False,
'seq_length': 5}
Traceback (most recent call last):
File "geonet_main.py", line 166, in
tf.app.run()
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/platform/app.py", line 48, in run
_sys.exit(main(_sys.argv[:1] + flags_passthrough))
File "geonet_main.py", line 157, in main
train()
File "geonet_main.py", line 72, in train
tgt_image, src_image_stack, intrinsics = loader.load_train_batch()
File "/home/panc/GeoNet/data_loader.py", line 21, in load_train_batch
file_list['image_file_list'], shuffle=False)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/input.py", line 217, in string_input_producer
raise ValueError(not_null_err)
ValueError: string_input_producer requires a non-null input tensor

谢谢, 如果有好的建议的话，中文回复我就可以了

Hi, @yzcjtr , great work!
I have some questions.
How can i testing optical network validate on own data set? Does the data set have to be formatted before testing?
thank you very much!

Great code! Your depth results have much clearer boundaries, compared with that of Tinghui Zhou.

Could you share some insight of persevering edges, especially when the unsupervised loss is used?

Hi, thanks for sharing codes!
I have a question for joint learning depth and flow networks.
Are there some positive effects when training the tasks all together?

I wonder why you didn't train them all at the last phase.