This repository is the official open-source implementation of the paper MBNSF - 3DV'2024 (oral):

Multi-Body Neural Scene Flow
Kavisha Vidanapathirana, Shin-Fang Ch'ng, Xueqian Li, Simon Lucey
International Conference on 3D Vision 2024 arXiv

This repository contains the code for:

• Evaluation of 3D scene flow and 4D trajectory estimation (via forward-Euler integration) on the Argoverse dataset using MBNSF (Ours) and NSFP - NeurIPS'2021 (Tab. 1 and 2 in the paper).
• Our implementation of NSFP++ - ECCV'2022 (Tab. 4 in the paper).
• Our implementation of NTP - CVPR'2022 and MBNSF (ours) for direct 4D trajectory prediction (Tab. 2 in the paper).

MBNSF is a regularizor for incorporating multi-body rigidity into neural scene flow in a manner that (1) doesn't require explicit estimaiton of SE(3) parameters for each rigid body, (2) is robust to the inherent noise in real-world LiDAR datasets, (3) maintains the fully unsupervised nature, and (4) maintains the estimation of continuous flow fields.

conda create --name mbnsf python=3.9.4
conda activate mbnsf

conda install pytorch==1.12.1 torchvision==0.13.1 torchaudio==0.12.1 -c pytorch
# Make sure the pytorch cuda version matches your output of 'nvcc --version'

conda install -c fvcore -c iopath -c conda-forge fvcore iopath
conda install -c bottler nvidiacub
conda install pytorch3d -c pytorch3d

pip install open3d

python -c "import torch ; import pytorch3d ; import open3d ; print(torch.cuda.is_available())"

This repository provides evaluation scripts for the Argoverse dataset.

This section re-creates the results of Tab. 1 and Tab. 2 in our paper on the Argoverse dataset.

cd scene_flow_estimation/

python nsfp.py --dataset_path </path/to/data>

python mbnsf.py --dataset_path </path/to/data>

cd trajectory_estimation/

python nsfp_fe.py --exp_name nsfp_fe_test --dataset_path </path/to/data>
python compute_traj_metrics.py --exp_name nsfp_fe_test --dataset_path </path/to/data>

python mbnsf_fe.py --exp_name mbnsf_fe_test --dataset_path </path/to/data>
python compute_traj_metrics.py --exp_name mbnsf_fe_test --dataset_path </path/to/data>

python ntp.py --exp_name ntp_test --dataset_path </path/to/data>
python compute_traj_metrics.py --exp_name ntp_test --dataset_path </path/to/data>

python mbnt.py --exp_name mbnt_test --dataset_path </path/to/data>
python compute_traj_metrics.py --exp_name mbnt_test --dataset_path </path/to/data>

If you find this work useful in your research, please cite:

@article{vidanapathirana2023mbnsf,
  title={Multi-Body Neural Scene Flow},
  author={Vidanapathirana, Kavisha and Chng, Shin-Fang and Li, Xueqian and Lucey, Simon},
  journal={arXiv preprint arXiv:2310.10301},
  year={2023}
}

Functions from 3rd party have been acknowledged at the respective function definitions or readme files.

This project was mainly inspired by NSFP and SpectralMatching.