This codebase implements the system described in the paper:
Unsupervised Learning of Depth and Ego-Motion from Video link
See the project webpage for more details.
TF code: https://github.com/tinghuiz/SfMLearner
In order to train the model using the provided code, the data needs to be formatted in a certain manner.
For KiTTI, first download the dataset using this script provided on the official website, and then run the following command
python data/prepare_train_data.py /path/to/KITTI_raw --dataset-format kitti_raw --static-frames ./data/static_frames.txt --dump-root /path/to/KITTI_formatted --height 128 --width 416 --num-threads 8This script generates only training data.
Remove '2011_09_26_drive_0067' sequence because there is no data at kitti server.
python data/prepare_train_data.py /path/to/KITTI_raw --dataset-format kitti_odom --static-frames ./data/static_frames.txt --dump-root /path/to/KITTI_formatted --height 128 --width 416 --num-threads 8Once the data are formatted following the above instructions, you should be able to train the model by running the following command
python3 train.py experiments/sfm_learner_v1.ymlpython3 train.py experiments/sfm_learner_v1_odom.ymlIf you finish training models using above scripts, you should be able to evaluate your trained model.
You can obtain the single-view depth predictions on the KITTI eigen test split formatted properly for evaluation by running.
You could download pretrained model from here
python evaluate.py experiments/sfm_learner_v1_eval.ymlYou can obtain the 5-snipped odometry predictions on the KITTI odometry dataset. This scripts use kitti raw dataset directly.
python evaluate.py experiments/sfm_learner_v1_odom_eval.yml --mode odom| abs_rel | sq_rel | rms | log_rms | a1 | a2 | a3 |
|---|---|---|---|---|---|---|
| 0.1779 | 1.3594 | 6.2696 | 0.2570 | 0.7390 | 0.9075 | 0.9647 |
You could download pretrained model from here
# For kitti formatted dataset
python inference.py experiments/sfm_learner_v1_test.yml
# For a image
python inference.py experiments/sfm_learner_v1_test.yml --img_path /path/to/img --save 1 --width 416 --height 128# Create predicted trajectory
python inference.py experiments/sfm_learner_v1_odom_test.yml --mode odom
# Visualize trajectories
python inference.py experiments/sfm_learner_v1_odom_test.yml --mode odom --gt_file ./kitti_eval/pose_data/ground_truth/10_full.txt --pred_file ./test.txt
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent