xuyangbai / ppf-foldnet Goto Github PK

PyTorch reimplementation for "PPF-FoldNet: Unsupervised Learning of Rotation Invariant 3D Local Descriptors" https://arxiv.org/abs/1808.10322

Python 91.69% Shell 8.31%

ppfnet ppf-foldnet point-cloud pytorch registration

ppf-foldnet's Introduction

PyTorch PPF-FoldNet

This repo is the unofficial implementation for PPF-FoldNet(https://arxiv.org/abs/1808.10322v1) in pytorch.

Project Structure

models/: dictionary that saves the model of PPF-FoldNet. PPF-FoldNet is an Auto-Encoder for point pair feature of a local patch. The input is a batch of point cloud fragments [bs(num_patches), num_points_per_patch, 4], output of the Encoder is the descriptor for each local patch in these point cloud, [bs(num_patches), 1, 512], where 512 is the default codeword length.
1. models_conv1d.py: PPF-FoldNet model using conv1d layers.
2. models_linear.py: PPF-FoldNet model using linear layers. Theoretically, nn.Conv1d and nn.Linear should be same when (kernel_size=1, stride=1, padding=0, dilation=1). You can try misc/linear_conv1d.py for the experiment.
input_preparation.py: used before training, including:
1. read point cloud and voxel downsample.
2. choose reference point(or interest point) from the point cloud.
3. collect neighboring points near each reference point.
4. build local patch for each reference point and their neighbor.
5. save the local patch as numpy array for later use.
6. I also write a function for preparing the ppf input on the fly.
dataset.py: define the Dataset, read from the files generated from input prepration stage or on-the-fly.
dataloader.py: define the dataloader.
loss.py: define the Chamfer Loss. (Earth Move Distance Loss is worth trying.)
trainer.py: the trainer class, handle the training process including snapshot.
train.py: the entrance file, every time I start training, this file will be copied to the snapshot dictionary.
geometric_registration/: dictionary for evaluating the model through the task of geometric registration
1. gt_result/: the ground truth information provided by 3DMatch benchmark.
2. preparation.py: calculate the descriptor for each interest point provided by 3DMatch Benchmark. (need to calculate the ppf representation for each interest point first)
3. evaluate_ppfnet.py: using the evaluation metric proposed in PPF-FoldNet paper to evaluate the performance of the descriptor.
  1. get point cloud from .ply file and get the interest point coordinate from .keypts.bin file.
  2. use the descriptor generated by preparation.py to register each pair of point clouds fragment and save the result in pred_result/
  3. after register each pair of fragment, we can get the final recall of the descriptors.
4. utils.py
misc/

Data

rgbd_fragments/: fragments of training set.
intermediate-files-real/: dictionary that saves the keypoints coordinates for each scene.
fragments/: fragments of test set.

Prepara the date

Use script/download.sh to download all the training set from 3DMatch, and script/fuse_fragments_3DMatch.py to fuse rgbd frames into fragments.

And the intermediate file is downloaded from this link

Train the model

python train.py

All the configuration is in the train.py. When you start training, the train.py and model.py will be saved in the snapshot/ folder and the tensorboard file is saved in tensorboard/.

Evaluate the model

See Geometric Registration for detail

Performance

Model	Average Recall
PPF-FoldNet	69.3%
Origin Paper	71.8%
3DMatch	57.3%

The model with best performance is in folder pretrained/

ppf-foldnet's People

Contributors

Stargazers

Watchers

Forkers

hikari513 nnu-gisa sergioragostinho flamehaze1115 buxiangqiming-666 aosheng1996 dzy130120 shengyuh megayeye samsartor mengwentan lannio devriesmatt listindexoutofrange jiawenzhao xrr-233 powrluis jiaotongshebei132

ppf-foldnet's Issues

一个小问题关于rgbd_fragments/scene_list_train.txt

文件夹里没有这个./data/3DMatch/rgbd_fragments/scene_list_train.txt" 文件
Traceback (most recent call last):
File "input_preparation.py", line 172, in
with open("./data/3DMatch/rgbd_fragments/scene_list_train.txt") as f:
FileNotFoundError: [Errno 2] No such file or directory: './data/3DMatch/rgbd_fragments/scene_list_train.txt'

scene_list_{split}.txt

Thanks for the wonderful code. But the training could not start without scene_list_{split}.txt, could you provide it?

Hello, what is the data input to the network during the test model phase? Original point or ppfs?

How to feed my own ply file into the pretrained model?

Hello! thanks for sharing your codes. I have a question that I don't know which api or function in your codes to use to feed my own .ply file into the pretrained model in the pretrained directory. I want to test the model to have a more clear understanding about the Encode and Decode process. Hope you can get me some tips, thanks a lot.

how to get gt.log

hello
Rencently i try to read evaluate_ppfnet.py, however i can't find how to generate the corresponding gt.log. Does this file need to download or run other .py files to get?

converge issue

Thanks for the excellent work!
In my training, the loss in the first epoch is about 10000, and after 20 epochs, the loss decreases to ~1000. Should we add more epochs？

Unsupported image format in Prepare the data

Hello
When I prepared the train data using script/fuse_fragments_3DMatch.py to fuse rgbd fragments obtained an error in console:

Start making fragments
2 scenes
  Start scene rgbd-scenes-v2-scene_05
  1 sequences
    Start seq-01
[ScalableTSDFVolume::Integrate] Unsupported image format.
[ScalableTSDFVolume::Integrate] Unsupported image format.
[ScalableTSDFVolume::Integrate] Unsupported image format.
[ScalableTSDFVolume::Integrate] Unsupported image format.
...
[ScalableTSDFVolume::Integrate] Unsupported image format.
Write PLY failed: point cloud has 0 points.
    Finished seq-01
  Finished scene rgbd-scenes-v2-scene_06
Finished making fragments

And I don't find the .ply files with the points clouds

Training leads to crash randomly

When I run your code, my computer crashed and rebooted at line trainer.py/self.evaluate(0). I suspected it might relate to the conflict of some packages. Could you provide a complete list of your package versions? Such as PyTorch, tensorboardX, ect.? And your cuda and nvidia driver versions?

missing 'utils' in 'fuse_fragments_3DMatch.py'

Hi, I run this code with windows 10, vscode and the 'utils' could not be found, could you provide this module please?
Thank you!

How to get keypoints in intermediate files?

Thanks for your implementation of PPF-FoldNet.

I am curious about how to generate the keypoints in intermediate files. Are these keypoints generated by your algorithms or provided by 3DMatch?

Best,
Yoking

Why the number is 1 in your code?

Hi, thanks for code!
I have a question: why the number of patch is 1 in your code? In the original paper the number of patch es 2048 or 5k, but in training.py you defined in the líne 21 like 1?
Thanks!