Waymo Detection and Tracking
Requirements
- Linux
- Python >= 3.6
- PyTorch >= 1.4
- CUDA >= 10.0
- CMake >= 3.13.2
Installation Steps
- Sign up for waymo and accept the terms of use
- install GCloud command line tools and use the same credentials for Waymo dataset
- install Conda
- create conda env from requirement.yml to install dependences
- install MinkowskiEngine
- follow the steps mentioned in:
https://github.com/NVIDIA/MinkowskiEngine
- follow the steps mentioned in:
- install open_waymo_dataset library to extract data
pip install waymo-open-dataset-tf-2-3-0
- build cuda layers of dcn and iou3d_nms
bash setup.sh
- run the downloader to download data splits
python3 download_tfrecords.py --split 'training' --root_path './data/Waymo'
- install Waymo dataset evaluation kit. Follow the steps mentioned in https://github.com/waymo-research/waymo-open-dataset/blob/master/docs/quick_start.md
- optional: sparse convolutions are implemented using MinkowskiEngine and spconv library. we adopted MinkowskiEngine due to its efficiency and spped. in case you are interested in running spconv. you need to follow these steps
sudo apt-get install libboost-all-dev git clone https://github.com/traveller59/spconv.git --recursive cd spconv && git checkout 7342772 python setup.py bdist_wheel cd ./dist && pip install *
- then apply the following fixes:
- fix build error in spconv pytorch/extension-script#6
- solve cuda compiler error traveller59/spconv#211
- fix cudnn version check while building spconv pytorch/pytorch#40965
- then apply the following fixes:
Data Preparation
- preprocess data, extract annotaions and point clouds for every frame into piclke file
# train set CUDA_VISIBLE_DEVICES=-1 python3 waymo_dataset/waymo_converter.py --tfrecord_path 'data/Waymo/tfrecord_training/segment-*.tfrecord' --root_path './data/Waymo/train/'``` # validation set CUDA_VISIBLE_DEVICES=-1 python3 waymo_dataset/waymo_converter.py --tfrecord_path './data/Waymo/tfrecord_validation/segment-*.tfrecord' --root_path './data/Waymo/val/' # testing set CUDA_VISIBLE_DEVICES=-1 python3 waymo_dataset/waymo_converter.py --tfrecord_path 'data/Waymo/tfrecord_validation/segment-*.tfrecord' --root_path './data/Waymo/test/'
- create info files
# One Sweep Infos python waymo_dataset/create_data.py waymo_data_prep --root_path=data/Waymo --split train --nsweeps=1 python waymo_dataset/create_data.py waymo_data_prep --root_path=data/Waymo --split val --nsweeps=1 python waymo_dataset/create_data.py waymo_data_prep --root_path=data/Waymo --split test --nsweeps=1 # Two Sweep Infos python waymo_dataset/create_data.py waymo_data_prep --root_path=data/Waymo --split train --nsweeps=2 python waymo_dataset/create_data.py waymo_data_prep --root_path=data/Waymo --split val --nsweeps=2 python waymo_dataset/create_data.py waymo_data_prep --root_path=data/Waymo --split test --nsweeps=2
Folder Structure
.
├── cmd_waymo_eval_kit.py # script to run Waymo evaluation kit !
├── conf # configurations for the detector
├── conf_tracking # configurations for the tracker
├── data
├── download_tfrecords.py
├── outputs
├── runs
├── setup.sh
├── test.py
├── tracking
├── tracking_baseline.py # inference script for the baseline tracker
├── tracking_gnn.py # inference script for the GNN Tracker
├── train.py # training script for the detector
├── train_tracker.py # training script for the tracker
├── trainer_utils.py
└── viz_predictions.py
Training and Evaluation commands
Detector
All configurations for the Detector is saved in ./conf
folder inside *.yml
files.
Train
python3 ./train.py
Evaluate
- adjust
checkpoint
in./conf/config.yaml
to refer the trained model checkpoint - run the prediction
python ./test.py
- generate Ground Truth data for the validation set
python waymo_dataset/waymo_common.py --info_path data/Waymo/infos_val_01sweeps_filter_zero_gt.pkl --result_path data/Waymo/ --gt
- evaluate predicitons (Detections) !
before running the evaluation cmd, open
python cmd_waymo_eval_kit.py evaluate_detections
cmd_waymo_eval_kit.py
and insert the correct links the prediction file and ground truth file generated from the model and the current used validation dataset.
Tracker
Train
All configurations for the Detector is saved in ./conf_tracking
folder inside *.yml
files.
python ./train_tracker.py
Evaluate
-
Tracking baseline: this is the baseline tracker which uses velocity cues and greedy matching to assign new detections to the current tracks
python tracking_baseline.py --work_dir=./output_tracking --checkpoint=./ckpts/epoch_36/prediction.pkl --info_path=./data/Waymo/infos_val_02sweeps_filter_zero_gt.pkl
provide the arguments for the current checkpoint resulted from training and dataset info file generated when preparing the data.
-
Tracking GNN
- adjust
checkpoint
in./conf/config.yaml
to refer the trained model checkpoint
python3 tracking_gnn.py --work_dir=./output_tracking --checkpoint=./outputs/2021-04-05/04-37-43/epoch_50.pt --prediction_results=./ckpts/epoch_36/prediction.pkl --info_path=./data/Waymo/infos_val_02sweeps_filter_zero_gt.pkl
provide the arguments for the current checkpoint resulted from training and dataset info file generated when preparing the data.
- adjust
-
evaluate predicitons (Tracking) !
python cmd_waymo_eval_kit.py evaluate_tracking
before running the evaluation cmd, open
cmd_waymo_eval_kit.py
and insert the correct links the prediction file and ground truth file generated from the model and the current used validation dataset.
Launch Tensorboard
python3 -m tensorboard.main --logdir work_dirs/ --port=6006