chrischoy / deepglobalregistration Goto Github PK

Is that possible to use deep global registration without MinkowskiEngine? I want to test deep global registration but cannot install MinkowskiEngine successfully. It seems that MinkowskiEngine has a bug because of GCC, I must update Cuda version to 11.0 to avoid the problem, but I don't have sudo, and cannot update cuda.

In this section
5.2. Multi-way Registration
Multi-way registration for RGB-D scans proceeds via multiple stages. First, the pipeline estimates the camera pose via off-the-shelf odometry and integrates multiple 3D scans to reduce noise and generate accurate 3D fragments of a scene. Next, a pairwise registration algorithm roughly aligns all fragments, followed by multi-way registration [6] which optimizes fragment poses with robust pose graph optimization.
I want to know in the fisrt stage, if you use the groud truth of these datasets ?
thanks

Hi,
as always, thanks for sharing.
I just want to point out that running the script to download the training dataset (3dMatch) is not working because the wget command returns the 404 error.
Am I correct by assuming that the threedmatch.tgz file is the same as the one linked in the script of FCGF? (I've just checked that link and it's working).
Thanks again and have a nice day,
Marco

Can you please verify which Version does the code work? I am trying multiple version of pytorch. and MinkowskiEngine. They are out-dated and cannot work using the current command

Hello,

Thanks for sharing the code. The network runs into an infinity error every time I train the network. I use the same parameters and the same dataset as the initial code. Could you please say how I can prevent it?
Thanks in advance.

Regards,
Vaishali Nimilan

for CUDA 10.2 (Ubuntu 20.04.2LTS)

g++-7 --version` is at least 7.4.0
open3d=0.10.0
MinkowskiEngine=0.4.3

• cd MinkowskiEngine

• export CXX=g++-7

• python setup.py
  pytorch=1.8.1 torchvision cudatoolkit=10.2

other libraries as in requirements.txt

Thank you for sharing your great work.

Will it be possible to share dockerfile for this project?

The current version of Minkowski Engine is not supported and I am having trouble making DGR and MK Engine work together.

Any help will be appreciated.

Hello Christopher,
I'm trying to evaluate your model on the synthetic ModelNet40 dataset. I wrote my own dataloader and trained your model with the default settings. But after a few steps, the loss becomes infinite. Can you give some suggestions about how to train your model on ModelNet40 dataset? (I've changed and tried different voxel sizes choose from 0.01 to 0.05, but did not work.)

A piece of logs during training can be found below. It seems the returned 'ws' variables from weighted_procrustes are all lower than the threshold (10 in your code), which means all input items have less than 10 valid correspondences pairs.

12/27 19:11:19 => loading weights for inlier model './FCGF_pretrained_3dmatch.pth'
12/27 19:11:19 => Loaded base model weights from './FCGF_pretrained_3dmatch.pth'
12/27 19:11:19 Inlier weight not found in './FCGF_pretrained_3dmatch.pth'
/home/mmvc/anaconda3/envs/XL_py3_cuda10/lib/python3.7/site-packages/torch/optim/lr_scheduler.py:449: UserWarning: To get the last learning rate computed by the scheduler, please use get_last_lr().
"please use get_last_lr().", UserWarning)
12/27 19:11:19 Epoch: 1, LR: [0.1]
num valid_mask, weights.mean(), weights.max 6141 0.49184879660606384 0.8573861122131348
num_valid, ws 8 tensor([379.1353, 379.4299, 379.1117, 378.9486, 366.3976, 379.6558, 379.5316,
379.6907])
rot_error, trans_error, loss 1.0711798667907715 0.260419100522995 1.3315989971160889
12/27 19:11:23 Train Epoch: 1 [0/1230], Current Loss: 2.011e+00, Correspondence acc: 2.116e-02 , Precision: 0.0156, Recall: 0.0154, F1: 0.0155, TPR: 0.0154, TNR: 0.9790, BAcc: 0.4972 RTE: 2.604e-01, RRE: 6.137e+01, Succ rate: 2.500000e-01 Avg num valid: 8.000000e+00 Data time: 2.2289, Train time: 1.5519, NN search time: 1.772e-02, Total time: 3.7808
num valid_mask, weights.mean(), weights.max 6141 0.4818011224269867 0.8922387957572937
num_valid, ws 8 tensor([370.2599, 365.9893, 370.7393, 370.6736, 370.6388, 370.6003, 370.6378,
370.5956])
rot_error, trans_error, loss 0.7573173642158508 0.19806239008903503 0.9553797245025635
num valid_mask, weights.mean(), weights.max 6143 0.45103055238723755 0.9931040406227112
num_valid, ws 8 tensor([345.9790, 343.8760, 344.7417, 344.6624, 344.8690, 345.0934, 357.2584,
344.6322])
rot_error, trans_error, loss 0.7590370774269104 0.26072144508361816 1.0197584629058838
num valid_mask, weights.mean(), weights.max 6144 0.42308372259140015 0.9964439272880554
num_valid, ws 8 tensor([327.4283, 324.3058, 324.6727, 324.4388, 324.5334, 324.9588, 324.6275,
324.4608])
rot_error, trans_error, loss 1.3009799718856812 0.21060435473918915 1.5115842819213867
num valid_mask, weights.mean(), weights.max 6144 0.38447126746177673 0.9924339056015015
num_valid, ws 8 tensor([309.3771, 293.2216, 293.0517, 294.7009, 293.1519, 292.7238, 292.6430,
293.3217])
rot_error, trans_error, loss 0.544593334197998 0.2057737112045288 0.7503671646118164
num valid_mask, weights.mean(), weights.max 6143 0.34405946731567383 0.8312152028083801
num_valid, ws 8 tensor([265.8239, 263.1904, 265.2418, 266.2042, 263.8054, 263.6702, 263.0773,
262.8550])
rot_error, trans_error, loss 1.1339480876922607 0.24252642691135406 1.3764744997024536
num valid_mask, weights.mean(), weights.max 6144 0.2863656282424927 0.9999984502792358
num_valid, ws 8 tensor([222.1238, 218.1181, 215.6836, 226.8676, 217.5520, 217.8743, 220.7595,
220.4514])
rot_error, trans_error, loss 1.0707015991210938 0.27141687273979187 1.3421183824539185
num valid_mask, weights.mean(), weights.max 6144 0.21661357581615448 0.34146520495414734
num_valid, ws 8 tensor([173.3432, 162.5075, 164.1453, 168.3337, 164.7490, 168.9405, 158.8151,
170.0394])
rot_error, trans_error, loss 0.8970396518707275 0.30696189403533936 1.2040014266967773
num valid_mask, weights.mean(), weights.max 6138 0.22650477290153503 0.9914805889129639
num_valid, ws 8 tensor([171.5588, 173.3483, 179.2511, 173.2488, 172.3625, 172.5608, 172.9400,
176.2171])
rot_error, trans_error, loss 1.0415538549423218 0.34417128562927246 1.3857251405715942
num valid_mask, weights.mean(), weights.max 6116 0.17247292399406433 0.35712677240371704
num_valid, ws 8 tensor([132.9017, 130.7651, 128.1572, 133.2996, 131.4898, 134.0894, 132.3888,
135.9132])
rot_error, trans_error, loss 0.9503939151763916 0.22698664665222168 1.1773805618286133
num valid_mask, weights.mean(), weights.max 6064 0.1356555074453354 0.7415195107460022
num_valid, ws 8 tensor([104.9554, 104.2914, 102.3165, 103.3207, 103.8977, 103.7889, 103.3343,
105.0705])
rot_error, trans_error, loss 0.5307180881500244 0.16665557026863098 0.697373628616333
num valid_mask, weights.mean(), weights.max 6048 0.12273039668798447 0.5408609509468079
num_valid, ws 8 tensor([92.7914, 91.0353, 90.9816, 92.6906, 98.6065, 94.2565, 96.6829, 94.6045])
rot_error, trans_error, loss 0.7709248661994934 0.1336054652929306 0.9045303463935852
num valid_mask, weights.mean(), weights.max 5288 0.07958836853504181 0.17408770322799683
num_valid, ws 8 tensor([59.2375, 49.9505, 57.0326, 52.8035, 60.2806, 60.2326, 60.3720, 57.5639])
rot_error, trans_error, loss 1.1103489398956299 0.25814875960350037 1.3684978485107422
num valid_mask, weights.mean(), weights.max 3978 0.06213882565498352 0.18988150358200073
num_valid, ws 8 tensor([32.8604, 31.9178, 39.7851, 39.5970, 37.5519, 40.9863, 38.9679, 40.8111])
rot_error, trans_error, loss 1.3329100608825684 0.35916927456855774 1.6920795440673828
num valid_mask, weights.mean(), weights.max 3824 0.05560467392206192 0.16689369082450867
num_valid, ws 8 tensor([34.9690, 44.5694, 32.6451, 32.3816, 34.3029, 31.2937, 31.9045, 32.9062])
rot_error, trans_error, loss 1.1255593299865723 0.261192262172699 1.386751413345337
num valid_mask, weights.mean(), weights.max 2160 0.04612640663981438 0.19192813336849213
num_valid, ws 8 tensor([21.0208, 19.3035, 14.4086, 24.3450, 20.2616, 22.0708, 29.0271, 16.9002])
rot_error, trans_error, loss 0.9262062311172485 0.3681674003601074 1.2943737506866455
num valid_mask, weights.mean(), weights.max 4355 0.05530725419521332 0.10722033679485321
num_valid, ws 8 tensor([34.5013, 41.0727, 37.0651, 35.8570, 38.5333, 39.4196, 36.3496, 38.5415])
rot_error, trans_error, loss 0.977398157119751 0.1942673623561859 1.1716655492782593
num valid_mask, weights.mean(), weights.max 1208 0.03593476861715317 0.09210973232984543
num_valid, ws 5 tensor([11.2543, 8.6302, 11.9625, 9.4134, 14.0098, 13.0315, 11.4335, 5.5197])
rot_error, trans_error, loss 1.2372373342514038 0.310983270406723 1.81670343875885
num valid_mask, weights.mean(), weights.max 2057 0.040355708450078964 0.10537022352218628
num_valid, ws 8 tensor([17.9894, 16.3948, 18.6110, 18.1689, 11.5592, 15.3969, 13.3000, 15.5626])
rot_error, trans_error, loss 0.769038736820221 0.1769949048757553 0.9460336565971375
num valid_mask, weights.mean(), weights.max 1714 0.036536455154418945 0.0762948989868164
num_valid, ws 7 tensor([17.7795, 12.5434, 10.8226, 14.2316, 14.7396, 10.7642, 7.0441, 12.2532])
rot_error, trans_error, loss 0.6872734427452087 0.21808893978595734 0.9117898941040039
num valid_mask, weights.mean(), weights.max 572 0.028709784150123596 0.08586414158344269
num_valid, ws 0 tensor([2.6134, 7.0487, 4.7059, 3.5765, 4.7559, 5.8939, 6.2377, 0.6551])
rot_error, trans_error, loss 1.3239400386810303 0.4019933044910431 nan
12/27 19:12:02 Loss is infinite, abort
num valid_mask, weights.mean(), weights.max 680 0.03068387135863304 0.09823115170001984
num_valid, ws 0 tensor([4.6802, 3.1837, 6.2774, 6.7470, 5.4355, 6.1506, 3.3652, 6.2467])
rot_error, trans_error, loss 1.7035499811172485 0.46357956528663635 nan
12/27 19:12:04 Loss is infinite, abort
num valid_mask, weights.mean(), weights.max 1567 0.03716740012168884 0.09059126675128937
num_valid, ws 5 tensor([18.5755, 16.7832, 11.3389, 9.6333, 11.9852, 9.1712, 5.9884, 11.1698])
rot_error, trans_error, loss 0.9750125408172607 0.32420605421066284 1.2242933511734009
num valid_mask, weights.mean(), weights.max 1137 0.03218415006995201 0.09962588548660278
num_valid, ws 3 tensor([12.0679, 8.3144, 9.6743, 9.2133, 3.5999, 8.8996, 11.8518, 10.5215])
rot_error, trans_error, loss 0.5337458252906799 0.19239680469036102 0.6585435271263123
num valid_mask, weights.mean(), weights.max 966 0.029521774500608444 0.10332392156124115
num_valid, ws 0 tensor([8.4000, 8.9959, 6.1997, 8.6553, 9.4742, 7.0023, 8.5738, 4.6189])
rot_error, trans_error, loss 1.0506246089935303 0.3039046823978424 nan
12/27 19:12:10 Loss is infinite, abort
num valid_mask, weights.mean(), weights.max 1074 0.02992144785821438 0.10137491673231125
num_valid, ws 2 tensor([12.6404, 8.2678, 3.2425, 12.0476, 6.9744, 5.5812, 9.3145, 8.7726])
rot_error, trans_error, loss 1.611760139465332 0.4179706275463104 1.2622950077056885
num valid_mask, weights.mean(), weights.max 1543 0.034905172884464264 0.09911085665225983
num_valid, ws 5 tensor([10.5303, 9.5016, 17.0358, 9.5487, 14.0452, 16.5256, 8.4574, 10.9382])
rot_error, trans_error, loss 1.512508749961853 0.3802349269390106 2.0282459259033203
num valid_mask, weights.mean(), weights.max 1506 0.03240210562944412 0.1912095695734024
num_valid, ws 4 tensor([10.4049, 8.4145, 12.4113, 8.7133, 10.1371, 16.9338, 7.0592, 9.6483])
rot_error, trans_error, loss 0.7196844816207886 0.161781445145607 0.6110647916793823
num valid_mask, weights.mean(), weights.max 831 0.02881685458123684 0.10011401772499084
num_valid, ws 0 tensor([7.1511, 2.9459, 9.2325, 5.5985, 2.2618, 4.2966, 4.5117, 9.9505])
rot_error, trans_error, loss 1.474617600440979 0.3669683635234833 nan
12/27 19:12:18 Loss is infinite, abort
num valid_mask, weights.mean(), weights.max 449 0.026938147842884064 0.09409534186124802
num_valid, ws 0 tensor([5.2363, 2.1232, 1.9240, 1.4735, 2.0403, 1.9763, 6.4409, 3.1939])
rot_error, trans_error, loss 1.2611165046691895 0.29535406827926636 nan
12/27 19:12:20 Loss is infinite, abort

hello chrischoy, Thank you for great work.

When I tried to run KITTI test script below,

python -m scripts.test_kitti --kitti_dir /path/to/kitti/ --weights /path/to/dgr_kitti.pth

I got error like below

Traceback (most recent call last):
  File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/Documents/MEGA/github/DeepGlobalRegistration/scripts/test_kitti.py", line 21, in <module>
    from scripts.test_3dmatch import DGRWrapper, FPFHWrapper, FCGFWrapper, rte_rre
ImportError: cannot import name 'DGRWrapper' from 'scripts.test_3dmatch' (/Documents/MEGA/github/DeepGlobalRegistration/scripts/test_3dmatch.py)

DGRWrapper, FPFHWrapper, FCGFWrapper looks not exist in this repository.

How can i fix this test script?

By the way, scripts.test_3dmatch looks working fine.

Traceback (most recent call last):
File "train.py", line 79, in
main(config)
File "train.py", line 57, in main
trainer.train()
File "/home/lthach/DeepGlobalRegistration-master (1)/core/trainer.py", line 135, in train
self._train_epoch(epoch)
File "/home/lthach/DeepGlobalRegistration-master (1)/core/trainer.py", line 266, in _train_epoch
loss.backward()
File "/home/lthach/anaconda3/envs/dgr/lib/python3.7/site-packages/torch/tensor.py", line 245, in backward
torch.autograd.backward(self, gradient, retain_graph, create_graph, inputs=inputs)
File "/home/lthach/anaconda3/envs/dgr/lib/python3.7/site-packages/torch/autograd/init.py", line 147, in backward
allow_unreachable=True, accumulate_grad=True) # allow_unreachable flag
File "/home/lthach/anaconda3/envs/dgr/lib/python3.7/site-packages/torch/autograd/function.py", line 89, in apply
return self._forward_cls.backward(self, *args) # type: ignore
File "/home/lthach/anaconda3/envs/dgr/lib/python3.7/site-packages/MinkowskiEngine/MinkowskiConvolution.py", line 111, in backward
coordinate_manager._manager,
RuntimeError: CUBLAS_STATUS_EXECUTION_FAILED at /tmp/pip-install-z25sch0o/minkowskiengine_101b60a34a8c4b73a1c33ae9cb85e161/src/math_functions_gpu.cu:46

Hi, when i try to train the DGR model , i meet this problem , do you know how to solve it ? Thanks!

First of all, thank you @chrischoy for this great work!

I have a question on the data augmentation used on the KITTI dataset: did you apply random rotations during training? from the code it seems that the random rotation is always set to False for both training and testing.

很烦，MinkowskiEngine版本不匹配，torch版本不匹配。一度绝望。
最后成功的版本
RTX3090,cuda11.4 ,torch 1.9.0+cu111,
然后是无尽的变量名修改，相比于前面的过程，显得轻松而愉快，毕竟改改代码比配环境可以见到进步，而不用无休止的conda create -n env_name pyhton==3.x -y pip insatll XXX
希望能帮到大家

Hi, @chrischoy thanks for your excellent job! Would your mind release the code for multiway registration?

Hi!
Thank you for your interesting work.
When I tested the demo, I got the error about 'feat_model'. How to get this “feat_model”？
In addition, according to the current version of MinkowskiEngine, it may be possible to replace "has_bias" with "bias".
Thanks.

Best wishes.

log:
python demo.py
=> loading checkpoint 'ResUNetBN2C-feat32-3dmatch-v0.05.pth'
=> Setting voxel size to 0.05
Traceback (most recent call last):
File "/home/houyongkuo/Documents/DeepGlobalRegistration/core/deep_global_registration.py", line 94, in init
FCGFModel = load_model(network_config['feat_model'])
KeyError: 'feat_model'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "demo.py", line 40, in
dgr = DeepGlobalRegistration(config)
File "/home/houyongkuo/Documents/DeepGlobalRegistration/core/deep_global_registration.py", line 104, in init
self.fcgf_model = FCGFModel(num_feats,
File "/home/houyongkuo/Documents/DeepGlobalRegistration/model/resunet.py", line 518, in init
self.conv4_tr = conv_tr(
File "/home/houyongkuo/Documents/DeepGlobalRegistration/model/residual_block.py", line 72, in conv_tr
return ME.MinkowskiConvolutionTranspose(
TypeError: init() got an unexpected keyword argument 'has_bias'

Hi,

Thanks for sharing the code.

I'm trying to train DGR on my own dataset.
So I made a dataloader which returns the same format as other loaders in this repo.
For example, I printed what getitem returns right before its returning line like this

print('u0: {}, u1: {}, c0: {}, c1: {}, f0: {}, f1: {}, m: {}, trans: {}'.format(unique_xyz0_th.shape, unique_xyz1_th.shape, coords0.shape, coords1.shape, feats0.shape, feats1.shape, len(matches), trans.shape))

        return (unique_xyz0_th.float(),
                unique_xyz1_th.float(), coords0.int(), coords1.int(), feats0.float(),
                feats1.float(), matches, trans, extra_package)

and this is its example output

u0: torch.Size([279, 3]), u1: torch.Size([281, 3]), c0: torch.Size([279, 3]), c1: torch.Size([281, 3]), f0: torch.Size([279, 1]), f1: torch.Size([281, 1]), m: 46745, trans: (4, 4)
u0: torch.Size([900, 3]), u1: torch.Size([859, 3]), c0: torch.Size([900, 3]), c1: torch.Size([859, 3]), f0: torch.Size([900, 1]), f1: torch.Size([859, 1]), m: 4696, trans: (4, 4)
u0: torch.Size([1159, 3]), u1: torch.Size([1153, 3]), c0: torch.Size([1159, 3]), c1: torch.Size([1153, 3]), f0: torch.Size([1159, 1]), f1: torch.Size([1153, 1]), m: 298974, trans: (4, 4)
u0: torch.Size([2092, 3]), u1: torch.Size([2048, 3]), c0: torch.Size([2092, 3]), c1: torch.Size([2048, 3]), f0: torch.Size([2092, 1]), f1: torch.Size([2048, 1]), m: 587866, trans: (4, 4)

These look similar to what 3DMatch dataset returns.

So I run the training code but it complains with

Traceback (most recent call last):
  File "train.py", line 76, in <module>
    main(config)
  File "train.py", line 55, in main
    trainer.train()
  File "dgr/core/trainer.py", line 135, in train
    self._train_epoch(epoch)
  File "dgr/core/trainer.py", line 237, in _train_epoch
    weights=weights)
  File "dgr/core/trainer.py", line 591, in weighted_procrustes
    X=xyz0[pred_pair[:, 0]].to(self.device),
IndexError: index 588 is out of bounds for dimension 0 with size 588

To see what it means, I also printed xyz0, xyz1, and pred_pair in core/trainer.py like this

  def weighted_procrustes(self, xyz0s, xyz1s, pred_pairs, weights):
    decomposed_weights = self.decompose_by_length(weights, pred_pairs)
    RT = []
    ws = []

    for xyz0, xyz1, pred_pair, w in zip(xyz0s, xyz1s, pred_pairs, decomposed_weights):
      xyz0.requires_grad = False
      xyz1.requires_grad = False
      ws.append(w.sum().item())
      print('in trainer, xyz0: {}, xyz1: {}, pred_pair: {}'.format(xyz0.shape, xyz1.shape, pred_pair))
      predT = GlobalRegistration.weighted_procrustes(
          X=xyz0[pred_pair[:, 0]].to(self.device),
          Y=xyz1[pred_pair[:, 1]].to(self.device),
          w=w,
          eps=np.finfo(np.float32).eps)
      RT.append(predT)

and this is what I got

in trainer, xyz0: torch.Size([1201, 3]), xyz1: torch.Size([1178, 3]), pred_pair: tensor([[  0,  23],
        [  1,   5],
        [  2,   5],
        ...,
        [585, 531],
        [586, 532],
        [587, 533]])
in trainer, xyz0: torch.Size([588, 3]), xyz1: torch.Size([569, 3]), pred_pair: tensor([[   0,  998],
        [   1,  948],
        [   2,   14],
        ...,
        [1188, 1167],
        [1189, 1166],
        [1190, 1072]])

For me, it seems like somehow the pred_pair is swapped since the first pred_pair has indices up to 587 which is the size of the xyz0 in the second.

I verified that I can run the training code of 3DMatch for a while.
Do you have an idea of why this error is happening?

Best,

How many graphics cards do you use for training and how long is the training time?
Thank you!

Hi Chris @chrischoy

Thanks for sharing this interesting work. I wonder have you tried the Context Normalization in the outlier filtering module, which is commonly used in 2D outlier rejection methods? And will using some pre-filtering strategies like mutual check or ratio test give better performance to your network?

Best,
Xuyang.

Thanks your work !!!
I would like to know how to achieve the reconstruction rendering effect like supplementmaterial and showing video.

My reconstruction is at a poor rendering level.

Thanks your work !!!

Hi chrischoy: Thanks your work,I want know where is the supplement?
I want to know the detail of the 3.3. Weighted Procrustes for SE(3),thanks!

The CPU of my computer is 12 Intel(R) Core(TM) i7-8700 CPU @ 3.20GHz. When i install MinkowskiEngine, all the memory of the CPU is occupied, and then the computer crashes.

I followed the command below to install MinkowskiEngine from MinkowskiEngine's github.

conda install openblas-devel -c anaconda

# Install MinkowskiEngine
pip install -U git+https://github.com/NVIDIA/MinkowskiEngine -v --no-deps --install-option="--blas_include_dirs=${CONDA_PREFIX}/include" --install-option="--blas=openblas"

Thanks for sharing your code.
I assume that the input of registration method only includes the input and output point cloud without other additional information. However, in dataloader/kitti_loader.py, the matches = get_matching_indices(pcd0, pcd1, trans, matching_search_voxel_size) uses the ground-truth transformation to get matched indices and this matches are used in
WeightedProcrustesTrainer._valid_epoch function in core/trainer.py (input_dict['correspondences']). That means without ground truth pose, the matched indices cannot be acquired and the whole algorithm cannot predict transformation? Maybe I got something wrong, but I'm very confused about this operation in dataloader. Hope for your explanation, thank you. The relative code are shown as follows:

in kitti_loader.py

# Voxelization
    xyz0_th = torch.from_numpy(xyz0)
    xyz1_th = torch.from_numpy(xyz1)

    sel0 = ME.utils.sparse_quantize(xyz0_th / self.voxel_size, return_index=True)
    sel1 = ME.utils.sparse_quantize(xyz1_th / self.voxel_size, return_index=True)

    # Make point clouds using voxelized points
    pcd0 = make_open3d_point_cloud(xyz0[sel0])
    pcd1 = make_open3d_point_cloud(xyz1[sel1])

    # Get matches
    matches = get_matching_indices(pcd0, pcd1, trans, matching_search_voxel_size)  # trans is ground truth transformation

in trainer.py

reg_coords, reg_feats, pred_pairs, is_correct, feat_time, nn_time = self.generate_inlier_input(
          xyz0=input_dict['pcd0'],
          xyz1=input_dict['pcd1'],
          iC0=input_dict['sinput0_C'],
          iC1=input_dict['sinput1_C'],
          iF0=input_dict['sinput0_F'],
          iF1=input_dict['sinput1_F'],
          len_batch=input_dict['len_batch'],
          pos_pairs=input_dict['correspondences']) # relies on matches which need ground truth transformation

When i run the ./train.sh , i meet this wrong, anything to solve it? Thank u!

Dear @chrischoy ,thank you for sharing your code. I have FileNotFoundError when testing 3DMatch.

I have downlaoded 3DMatch dataset through download_3dmatch.sh and run relevant script test_3dmatch.sh. However, it cannot find a file with the suffix of gt.log (Line 173 in threedmatch_loader.py).

    if scene_id is not None:
      subset_names = [subset_names[scene_id]]
    for sname in subset_names:
      traj_file = os.path.join(self.root, sname + '-evaluation/gt.log')  # cannot be found
      assert os.path.exists(traj_file)
      traj = read_trajectory(traj_file)
      for ctraj in traj:
        i = ctraj.metadata[0]
        j = ctraj.metadata[1]
        T_gt = ctraj.pose
        self.files.append((sname, i, j, T_gt))

I will appreciate for your reply.

Hi, thank your for your sharing.
I have one question, when train on the 3DMatch, the ground truth relative pose is generated randomly, are the original frames with same pose?

-------------------- threedmatch_loader.py, line 64~ line 72 -----------------------
if self.random_rotation:
T0 = sample_random_trans(xyz0, self.randg, self.rotation_range)
T1 = sample_random_trans(xyz1, self.randg, self.rotation_range)
trans = T1 @ np.linalg.inv(T0)

  xyz0 = self.apply_transform(xyz0, T0)
  xyz1 = self.apply_transform(xyz1, T1)
else:
  trans = np.identity(4)

Hi,

Thank you for the interesting work.

In table 3 of the paper, you compare DGR and FCGF in terms of speed and accuracy. I wonder how DGR can be quicker then FCGF. As far as I understood DRG pipeline includes evaluation of FCGF as one of the steps, so the time should be higher for the DGR. Maybe these times stand for something else. Can you please clarify that? Also, it looks like DGR performed worse than FCGF in the first place. Isn't it better just to use FCGF followed by Ransac instead?

Thank you for your answer.

Does this method do registration or matching for dynamic objects as well?

How can I create GT trans parameters of my own ply files?

Hello together,

I was trying to run DGR with mke 0.5.1 and had to do some adaptions.

FROM pytorch/pytorch:1.7.0-cuda11.0-cudnn8-devel

WORKDIR /app

RUN apt update 
RUN apt-get install -y apt-utils 
RUN apt-get install -y\
      build-essential \
      ca-certificates \
      wget \
      git \
      vim \
      libopenblas-dev \
      libssl-dev \
      libgl1-mesa-glx \
      libusb-1.0-0 \
      locate \
      python3.8-dev \
      curl \
      unzip \
      unrar

RUN export MAX_JOBS=1

#git checkout v0.5.1; \
COPY ./MinkowskiEngine /app/MinkowskiEngine 
RUN export CXX=c++; export CUDA_HOME=/usr/local/cuda-11.0; \
      cd /app/MinkowskiEngine/; \
      python setup.py install --blas=openblas --force_cuda

COPY ./DeepGlobalRegistration/requirements.txt /tmp/DeepGlobalRegistration/requirements.txt 
RUN pip install -r /tmp/DeepGlobalRegistration/requirements.txt

CMD [ "/bin/bash", "" ]

diff --git a/core/trainer.py b/core/trainer.py
index b8ae434..87bd834 100644
--- a/core/trainer.py
+++ b/core/trainer.py
@@ -215,7 +215,7 @@ class WeightedProcrustesTrainer:
         # Inlier prediction with 6D ConvNet
         inlier_timer.tic()
         reg_sinput = ME.SparseTensor(reg_feats.contiguous(),
-                                     coords=reg_coords.int()).to(self.device)
+                                     coordinates=reg_coords.int(), device="cuda")
         reg_soutput = self.inlier_model(reg_sinput)
         inlier_timer.toc()
 
@@ -396,7 +396,7 @@ class WeightedProcrustesTrainer:
 
       inlier_timer.tic()
       reg_sinput = ME.SparseTensor(reg_feats.contiguous(),
-                                   coords=reg_coords.int()).to(self.device)
+                                   coordinates=reg_coords.int(), device="cuda")
       reg_soutput = self.inlier_model(reg_sinput)
       inlier_timer.toc()
 
@@ -630,10 +630,10 @@ class WeightedProcrustesTrainer:
   def generate_inlier_input(self, xyz0, xyz1, iC0, iC1, iF0, iF1, len_batch, pos_pairs):
     # pairs consist of (xyz1 index, xyz0 index)
     stime = time.time()
-    sinput0 = ME.SparseTensor(iF0, coords=iC0).to(self.device)
+    sinput0 = ME.SparseTensor(iF0, coordinates=iC0, device="cuda")
     oF0 = self.feat_model(sinput0).F
 
-    sinput1 = ME.SparseTensor(iF1, coords=iC1).to(self.device)
+    sinput1 = ME.SparseTensor(iF1, coordinates=iC1, device="cuda")
     oF1 = self.feat_model(sinput1).F
     feat_time = time.time() - stime
 
diff --git a/dataloader/threedmatch_loader.py b/dataloader/threedmatch_loader.py
index 99ba346..d45ad0d 100644
--- a/dataloader/threedmatch_loader.py
+++ b/dataloader/threedmatch_loader.py
@@ -75,8 +75,8 @@ class IndoorPairDataset(PairDataset):
     xyz0_th = torch.from_numpy(xyz0)
     xyz1_th = torch.from_numpy(xyz1)
 
-    sel0 = ME.utils.sparse_quantize(xyz0_th / self.voxel_size, return_index=True)
-    sel1 = ME.utils.sparse_quantize(xyz1_th / self.voxel_size, return_index=True)
+    _, sel0 = ME.utils.sparse_quantize(xyz0_th / self.voxel_size, return_index=True)
+    _, sel1 = ME.utils.sparse_quantize(xyz1_th / self.voxel_size, return_index=True)
 
     # Make point clouds using voxelized points
     pcd0 = make_open3d_point_cloud(xyz0[sel0])
diff --git a/model/pyramidnet.py b/model/pyramidnet.py
index 8a0b9aa..0a643e3 100644
--- a/model/pyramidnet.py
+++ b/model/pyramidnet.py
@@ -15,7 +15,7 @@ from model.residual_block import get_block, conv, conv_tr, conv_norm_non
 class PyramidModule(ME.MinkowskiNetwork):
   NONLINEARITY = 'ELU'
   NORM_TYPE = 'BN'
-  REGION_TYPE = ME.RegionType.HYPERCUBE
+  REGION_TYPE = ME.RegionType.HYPER_CUBE
 
   def __init__(self,
                inc,
@@ -93,7 +93,7 @@ class PyramidNet(ME.MinkowskiNetwork):
   DEPTHS = [1, 1, 1, 1]
   # None        b1, b2, b3, btr3, btr2
   #               1  2  3 -3 -2 -1
-  REGION_TYPE = ME.RegionType.HYPERCUBE
+  REGION_TYPE = ME.RegionType.HYPER_CUBE
 
   # To use the model, must call initialize_coords before forward pass.
   # Once data is processed, call clear to reset the model before calling initialize_coords
diff --git a/model/residual_block.py b/model/residual_block.py
index f933be5..02985cb 100644
--- a/model/residual_block.py
+++ b/model/residual_block.py
@@ -17,12 +17,12 @@ def conv(in_channels,
          kernel_size=3,
          stride=1,
          dilation=1,
-         has_bias=False,
+         bias=False,
          region_type=0,
          dimension=3):
   if not isinstance(region_type, ME.RegionType):
     if region_type == 0:
-      region_type = ME.RegionType.HYPERCUBE
+      region_type = ME.RegionType.HYPER_CUBE
     elif region_type == 1:
       region_type = ME.RegionType.HYPERCROSS
     else:
@@ -49,8 +49,8 @@ def conv_tr(in_channels,
             kernel_size,
             stride=1,
             dilation=1,
-            has_bias=False,
-            region_type=ME.RegionType.HYPERCUBE,
+            bias=False,
+            region_type=ME.RegionType.HYPER_CUBE,
             dimension=-1):
   assert dimension > 0, 'Dimension must be a positive integer'
   kernel_generator = ME.KernelGenerator(
@@ -75,7 +75,7 @@ def conv_tr(in_channels,
       kernel_size=kernel_size,
       stride=stride,
       dilation=dilation,
-      has_bias=has_bias,
+      bias=bias,
       kernel_generator=kernel_generator,
       dimension=dimension)
 
@@ -174,7 +174,7 @@ def conv_norm_non(inc,
                   stride,
                   dimension,
                   bn_momentum=0.05,
-                  region_type=ME.RegionType.HYPERCUBE,
+                  region_type=ME.RegionType.HYPER_CUBE,
                   norm_type='BN',
                   nonlinearity='ELU'):
   return nn.Sequential(
@@ -184,7 +184,7 @@ def conv_norm_non(inc,
           kernel_size=kernel_size,
           stride=stride,
           dilation=1,
-          has_bias=False,
+          bias=False,
           region_type=region_type,
           dimension=dimension),
       get_norm(norm_type, outc, bn_momentum=bn_momentum, dimension=dimension),
diff --git a/model/resunet.py b/model/resunet.py
index 831517f..136104a 100644
--- a/model/resunet.py
+++ b/model/resunet.py
@@ -18,7 +18,7 @@ class ResUNet(ME.MinkowskiNetwork):
   BLOCK_NORM_TYPE = 'BN'
   CHANNELS = [None, 32, 64, 128]
   TR_CHANNELS = [None, 32, 64, 64]
-  REGION_TYPE = ME.RegionType.HYPERCUBE
+  REGION_TYPE = ME.RegionType.HYPER_CUBE
 
   # To use the model, must call initialize_coords before forward pass.
   # Once data is processed, call clear to reset the model before calling initialize_coords
@@ -43,7 +43,7 @@ class ResUNet(ME.MinkowskiNetwork):
         kernel_size=conv1_kernel_size,
         stride=1,
         dilation=1,
-        has_bias=False,
+        bias=False,
         dimension=D)
     self.norm1 = get_norm(NORM_TYPE, CHANNELS[1], bn_momentum=bn_momentum, dimension=D)
 
@@ -61,7 +61,7 @@ class ResUNet(ME.MinkowskiNetwork):
         kernel_size=3,
         stride=2,
         dilation=1,
-        has_bias=False,
+        bias=False,
         dimension=D)
     self.norm2 = get_norm(NORM_TYPE, CHANNELS[2], bn_momentum=bn_momentum, dimension=D)
 
@@ -79,7 +79,7 @@ class ResUNet(ME.MinkowskiNetwork):
         kernel_size=3,
         stride=2,
         dilation=1,
-        has_bias=False,
+        bias=False,
         dimension=D)
     self.norm3 = get_norm(NORM_TYPE, CHANNELS[3], bn_momentum=bn_momentum, dimension=D)
 
@@ -97,7 +97,7 @@ class ResUNet(ME.MinkowskiNetwork):
         kernel_size=3,
         stride=2,
         dilation=1,
-        has_bias=False,
+        bias=False,
         dimension=D)
     self.norm3_tr = get_norm(
         NORM_TYPE, TR_CHANNELS[3], bn_momentum=bn_momentum, dimension=D)
@@ -116,7 +116,7 @@ class ResUNet(ME.MinkowskiNetwork):
         kernel_size=3,
         stride=2,
         dilation=1,
-        has_bias=False,
+        bias=False,
         dimension=D)
     self.norm2_tr = get_norm(
         NORM_TYPE, TR_CHANNELS[2], bn_momentum=bn_momentum, dimension=D)
@@ -135,7 +135,7 @@ class ResUNet(ME.MinkowskiNetwork):
         kernel_size=1,
         stride=1,
         dilation=1,
-        has_bias=False,
+        bias=False,
         dimension=D)
 
     # self.block1_tr = BasicBlockBN(TR_CHANNELS[1], TR_CHANNELS[1], bn_momentum=bn_momentum, D=D)
@@ -146,7 +146,7 @@ class ResUNet(ME.MinkowskiNetwork):
         kernel_size=1,
         stride=1,
         dilation=1,
-        has_bias=True,
+        bias=True,
         dimension=D)
 
   def forward(self, x):
@@ -185,8 +185,8 @@ class ResUNet(ME.MinkowskiNetwork):
     if self.normalize_feature:
       return ME.SparseTensor(
           out.F / (torch.norm(out.F, p=2, dim=1, keepdim=True) + 1e-8),
-          coords_key=out.coords_key,
-          coords_manager=out.coords_man)
+          coordinate_map_key=out.coords_key,
+          coordinate_manager=out.coords_man)
     else:
       return out
 
@@ -202,7 +202,7 @@ class ResUNetBNF(ResUNet):
 
 
 class ResUNetBNFX(ResUNetBNF):
-  REGION_TYPE = ME.RegionType.HYPERCROSS
+  REGION_TYPE = ME.RegionType.HYPER_CROSS
 
 
 class ResUNetSP(ME.MinkowskiNetwork):
@@ -213,7 +213,7 @@ class ResUNetSP(ME.MinkowskiNetwork):
   # None        b1, b2, b3, btr3, btr2
   #               1  2  3 -3 -2 -1
   DEPTHS = [None, 1, 1, 1, 1, 1, None]
-  REGION_TYPE = ME.RegionType.HYPERCUBE
+  REGION_TYPE = ME.RegionType.HYPER_CUBE
 
   # To use the model, must call initialize_coords before forward pass.
   # Once data is processed, call clear to reset the model before calling initialize_coords
@@ -238,7 +238,7 @@ class ResUNetSP(ME.MinkowskiNetwork):
         kernel_size=conv1_kernel_size,
         stride=1,
         dilation=1,
-        has_bias=False,
+        bias=False,
         region_type=REGION_TYPE,
         dimension=D)
     self.norm1 = get_norm(NORM_TYPE, CHANNELS[1], bn_momentum=bn_momentum, dimension=D)
@@ -260,7 +260,7 @@ class ResUNetSP(ME.MinkowskiNetwork):
         kernel_size=1,
         stride=1,
         dilation=1,
-        has_bias=False,
+        bias=False,
         dimension=D)
     self.norm2 = get_norm(NORM_TYPE, CHANNELS[2], bn_momentum=bn_momentum, dimension=D)
 
@@ -281,7 +281,7 @@ class ResUNetSP(ME.MinkowskiNetwork):
         kernel_size=1,
         stride=1,
         dilation=1,
-        has_bias=False,
+        bias=False,
         dimension=D)
     self.norm3 = get_norm(NORM_TYPE, CHANNELS[3], bn_momentum=bn_momentum, dimension=D)
 
@@ -302,7 +302,7 @@ class ResUNetSP(ME.MinkowskiNetwork):
         kernel_size=1,
         stride=1,
         dilation=1,
-        has_bias=False,
+        bias=False,
         dimension=D)
     self.norm3_tr = get_norm(
         NORM_TYPE, TR_CHANNELS[3], bn_momentum=bn_momentum, dimension=D)
@@ -324,7 +324,7 @@ class ResUNetSP(ME.MinkowskiNetwork):
         kernel_size=1,
         stride=1,
         dilation=1,
-        has_bias=False,
+        bias=False,
         region_type=REGION_TYPE,
         dimension=D)
     self.norm2_tr = get_norm(
@@ -346,7 +346,7 @@ class ResUNetSP(ME.MinkowskiNetwork):
         kernel_size=1,
         stride=1,
         dilation=1,
-        has_bias=False,
+        bias=False,
         region_type=REGION_TYPE,
         dimension=D)
 
@@ -358,7 +358,7 @@ class ResUNetSP(ME.MinkowskiNetwork):
         kernel_size=1,
         stride=1,
         dilation=1,
-        has_bias=True,
+        bias=True,
         dimension=D)
 
   def forward(self, x):
@@ -402,14 +402,14 @@ class ResUNetSP(ME.MinkowskiNetwork):
     if self.normalize_feature:
       return ME.SparseTensor(
           out.F / (torch.norm(out.F, p=2, dim=1, keepdim=True) + 1e-8),
-          coords_key=out.coords_key,
-          coords_manager=out.coords_man)
+          coordinate_map_key=out.coords_key,
+          coordinate_manager=out.coords_man)
     else:
       return out
 
 
 class ResUNetBNSPC(ResUNetSP):
-  REGION_TYPE = ME.RegionType.HYPERCROSS
+  REGION_TYPE = ME.RegionType.HYPER_CROSS
 
 
 class ResUNetINBNSPC(ResUNetBNSPC):
@@ -421,7 +421,7 @@ class ResUNet2(ME.MinkowskiNetwork):
   BLOCK_NORM_TYPE = 'BN'
   CHANNELS = [None, 32, 64, 128, 256]
   TR_CHANNELS = [None, 32, 64, 64, 128]
-  REGION_TYPE = ME.RegionType.HYPERCUBE
+  REGION_TYPE = ME.RegionType.HYPER_CUBE
 
   # To use the model, must call initialize_coords before forward pass.
   # Once data is processed, call clear to reset the model before calling initialize_coords
@@ -445,7 +445,7 @@ class ResUNet2(ME.MinkowskiNetwork):
         kernel_size=conv1_kernel_size,
         stride=1,
         dilation=1,
-        has_bias=False,
+        bias=False,
         region_type=REGION_TYPE,
         dimension=D)
     self.norm1 = get_norm(NORM_TYPE, CHANNELS[1], bn_momentum=bn_momentum, dimension=D)
@@ -464,7 +464,7 @@ class ResUNet2(ME.MinkowskiNetwork):
         kernel_size=3,
         stride=2,
         dilation=1,
-        has_bias=False,
+        bias=False,
         region_type=REGION_TYPE,
         dimension=D)
     self.norm2 = get_norm(NORM_TYPE, CHANNELS[2], bn_momentum=bn_momentum, dimension=D)
@@ -483,7 +483,7 @@ class ResUNet2(ME.MinkowskiNetwork):
         kernel_size=3,
         stride=2,
         dilation=1,
-        has_bias=False,
+        bias=False,
         region_type=REGION_TYPE,
         dimension=D)
     self.norm3 = get_norm(NORM_TYPE, CHANNELS[3], bn_momentum=bn_momentum, dimension=D)
@@ -502,7 +502,7 @@ class ResUNet2(ME.MinkowskiNetwork):
         kernel_size=3,
         stride=2,
         dilation=1,
-        has_bias=False,
+        bias=False,
         region_type=REGION_TYPE,
         dimension=D)
     self.norm4 = get_norm(NORM_TYPE, CHANNELS[4], bn_momentum=bn_momentum, dimension=D)
@@ -521,7 +521,7 @@ class ResUNet2(ME.MinkowskiNetwork):
         kernel_size=3,
         stride=2,
         dilation=1,
-        has_bias=False,
+        bias=False,
         region_type=REGION_TYPE,
         dimension=D)
     self.norm4_tr = get_norm(
@@ -541,7 +541,7 @@ class ResUNet2(ME.MinkowskiNetwork):
         kernel_size=3,
         stride=2,
         dilation=1,
-        has_bias=False,
+        bias=False,
         region_type=REGION_TYPE,
         dimension=D)
     self.norm3_tr = get_norm(
@@ -561,7 +561,7 @@ class ResUNet2(ME.MinkowskiNetwork):
         kernel_size=3,
         stride=2,
         dilation=1,
-        has_bias=False,
+        bias=False,
         region_type=REGION_TYPE,
         dimension=D)
     self.norm2_tr = get_norm(
@@ -581,7 +581,7 @@ class ResUNet2(ME.MinkowskiNetwork):
         kernel_size=1,
         stride=1,
         dilation=1,
-        has_bias=False,
+        bias=False,
         dimension=D)
 
     # self.block1_tr = BasicBlockBN(TR_CHANNELS[1], TR_CHANNELS[1], bn_momentum=bn_momentum, D=D)
@@ -592,7 +592,7 @@ class ResUNet2(ME.MinkowskiNetwork):
         kernel_size=1,
         stride=1,
         dilation=1,
-        has_bias=True,
+        bias=True,
         dimension=D)
 
   def forward(self, x):
@@ -643,8 +643,8 @@ class ResUNet2(ME.MinkowskiNetwork):
     if self.normalize_feature:
       return ME.SparseTensor(
           out.F / (torch.norm(out.F, p=2, dim=1, keepdim=True) + 1e-8),
-          coords_key=out.coords_key,
-          coords_manager=out.coords_man)
+          coordinate_map_key=out.coordinate_map_key,
+          coordinate_manager=out.coordinate_manager)
     else:
       return out
 
@@ -666,7 +666,7 @@ class ResUNetBN2C(ResUNet2):
 
 
 class ResUNetBN2CX(ResUNetBN2C):
-  REGION_TYPE = ME.RegionType.HYPERCROSS
+  REGION_TYPE = ME.RegionType.HYPER_CROSS
 
 
 class ResUNetBN2D(ResUNet2):
@@ -688,7 +688,7 @@ class ResUNetBN2F(ResUNet2):
 
 
 class ResUNetBN2FX(ResUNetBN2F):
-  REGION_TYPE = ME.RegionType.HYPERCROSS
+  REGION_TYPE = ME.RegionType.HYPER_CROSS
 
 
 class ResUNet2v2(ME.MinkowskiNetwork):
@@ -699,7 +699,7 @@ class ResUNet2v2(ME.MinkowskiNetwork):
   # None        b1, b2, b3, b4, btr4, btr3, btr2
   #               1  2  3  4,-4,-3,-2,-1
   DEPTHS = [None, 1, 1, 1, 1, 1, 1, 1, None]
-  REGION_TYPE = ME.RegionType.HYPERCUBE
+  REGION_TYPE = ME.RegionType.HYPER_CUBE
 
   # To use the model, must call initialize_coords before forward pass.
   # Once data is processed, call clear to reset the model before calling initialize_coords
@@ -724,7 +724,7 @@ class ResUNet2v2(ME.MinkowskiNetwork):
         kernel_size=conv1_kernel_size,
         stride=1,
         dilation=1,
-        has_bias=False,
+        bias=False,
         dimension=D)
     self.norm1 = get_norm(NORM_TYPE, CHANNELS[1], bn_momentum=bn_momentum, dimension=D)
 
@@ -743,7 +743,7 @@ class ResUNet2v2(ME.MinkowskiNetwork):
         kernel_size=3,
         stride=2,
         dilation=1,
-        has_bias=False,
+        bias=False,
         dimension=D)
     self.norm2 = get_norm(NORM_TYPE, CHANNELS[2], bn_momentum=bn_momentum, dimension=D)
 
@@ -762,7 +762,7 @@ class ResUNet2v2(ME.MinkowskiNetwork):
         kernel_size=3,
         stride=2,
         dilation=1,
-        has_bias=False,
+        bias=False,
         dimension=D)
     self.norm3 = get_norm(NORM_TYPE, CHANNELS[3], bn_momentum=bn_momentum, dimension=D)
 
@@ -781,7 +781,7 @@ class ResUNet2v2(ME.MinkowskiNetwork):
         kernel_size=3,
         stride=2,
         dilation=1,
-        has_bias=False,
+        bias=False,
         dimension=D)
     self.norm4 = get_norm(NORM_TYPE, CHANNELS[4], bn_momentum=bn_momentum, dimension=D)
 
@@ -800,7 +800,7 @@ class ResUNet2v2(ME.MinkowskiNetwork):
         kernel_size=3,
         stride=2,
         dilation=1,
-        has_bias=False,
+        bias=False,
         dimension=D)
     self.norm4_tr = get_norm(
         NORM_TYPE, TR_CHANNELS[4], bn_momentum=bn_momentum, dimension=D)
@@ -820,7 +820,7 @@ class ResUNet2v2(ME.MinkowskiNetwork):
         kernel_size=3,
         stride=2,
         dilation=1,
-        has_bias=False,
+        bias=False,
         dimension=D)
     self.norm3_tr = get_norm(
         NORM_TYPE, TR_CHANNELS[3], bn_momentum=bn_momentum, dimension=D)
@@ -840,7 +840,7 @@ class ResUNet2v2(ME.MinkowskiNetwork):
         kernel_size=3,
         stride=2,
         dilation=1,
-        has_bias=False,
+        bias=False,
         dimension=D)
     self.norm2_tr = get_norm(
         NORM_TYPE, TR_CHANNELS[2], bn_momentum=bn_momentum, dimension=D)
@@ -860,7 +860,7 @@ class ResUNet2v2(ME.MinkowskiNetwork):
         kernel_size=1,
         stride=1,
         dilation=1,
-        has_bias=False,
+        bias=False,
         dimension=D)
 
     # self.block1_tr = BasicBlockBN(TR_CHANNELS[1], TR_CHANNELS[1], bn_momentum=bn_momentum, dimension=D)
@@ -871,7 +871,7 @@ class ResUNet2v2(ME.MinkowskiNetwork):
         kernel_size=1,
         stride=1,
         dilation=1,
-        has_bias=True,
+        bias=True,
         dimension=D)
     self.weight_initialization()
 
@@ -932,8 +932,8 @@ class ResUNet2v2(ME.MinkowskiNetwork):
     if self.normalize_feature:
       return ME.SparseTensor(
           out.F / (torch.norm(out.F, p=2, dim=1, keepdim=True) + 1e-8),
-          coords_key=out.coords_key,
-          coords_manager=out.coords_man)
+          coordinate_map_key=out.coords_key,
+          coordinate_manager=out.coords_man)
     else:
       return out
 
@@ -977,7 +977,7 @@ class ResUNet2SP(ME.MinkowskiNetwork):
   BLOCK_NORM_TYPE = 'BN'
   CHANNELS = [None, 32, 64, 128, 256]
   TR_CHANNELS = [None, 32, 64, 64, 128]
-  REGION_TYPE = ME.RegionType.HYPERCUBE
+  REGION_TYPE = ME.RegionType.HYPER_CUBE
 
   # To use the model, must call initialize_coords before forward pass.
   # Once data is processed, call clear to reset the model before calling initialize_coords
@@ -1001,8 +1001,8 @@ class ResUNet2SP(ME.MinkowskiNetwork):
         kernel_size=conv1_kernel_size,
         stride=1,
         dilation=1,
-        has_bias=False,
-        region_type=ME.RegionType.HYPERCUBE,
+        bias=False,
+        region_type=ME.RegionType.HYPER_CUBE,
         dimension=D)
     self.norm1 = get_norm(NORM_TYPE, CHANNELS[1], bn_momentum=bn_momentum, dimension=D)
 
@@ -1021,7 +1021,7 @@ class ResUNet2SP(ME.MinkowskiNetwork):
         kernel_size=3,
         stride=1,
         dilation=1,
-        has_bias=False,
+        bias=False,
         region_type=REGION_TYPE,
         dimension=D)
     self.norm2 = get_norm(NORM_TYPE, CHANNELS[2], bn_momentum=bn_momentum, dimension=D)
@@ -1041,7 +1041,7 @@ class ResUNet2SP(ME.MinkowskiNetwork):
         kernel_size=3,
         stride=1,
         dilation=1,
-        has_bias=False,
+        bias=False,
         region_type=REGION_TYPE,
         dimension=D)
     self.norm3 = get_norm(NORM_TYPE, CHANNELS[3], bn_momentum=bn_momentum, dimension=D)
@@ -1061,8 +1061,8 @@ class ResUNet2SP(ME.MinkowskiNetwork):
         kernel_size=3,
         stride=1,
         dilation=1,
-        has_bias=False,
-        region_type=ME.RegionType.HYPERCUBE,
+        bias=False,
+        region_type=ME.RegionType.HYPER_CUBE,
         dimension=D)
     self.norm4 = get_norm(NORM_TYPE, CHANNELS[4], bn_momentum=bn_momentum, dimension=D)
 
@@ -1071,7 +1071,7 @@ class ResUNet2SP(ME.MinkowskiNetwork):
         CHANNELS[4],
         CHANNELS[4],
         bn_momentum=bn_momentum,
-        region_type=ME.RegionType.HYPERCUBE,
+        region_type=ME.RegionType.HYPER_CUBE,
         dimension=D)
 
     self.conv4_tr = conv_tr(
@@ -1080,8 +1080,8 @@ class ResUNet2SP(ME.MinkowskiNetwork):
         kernel_size=3,
         stride=2,
         dilation=1,
-        has_bias=False,
-        region_type=ME.RegionType.HYPERCUBE,
+        bias=False,
+        region_type=ME.RegionType.HYPER_CUBE,
         dimension=D)
     self.norm4_tr = get_norm(
         NORM_TYPE, TR_CHANNELS[4], bn_momentum=bn_momentum, dimension=D)
@@ -1100,7 +1100,7 @@ class ResUNet2SP(ME.MinkowskiNetwork):
         kernel_size=3,
         stride=2,
         dilation=1,
-        has_bias=False,
+        bias=False,
         region_type=REGION_TYPE,
         dimension=D)
     self.norm3_tr = get_norm(
@@ -1120,7 +1120,7 @@ class ResUNet2SP(ME.MinkowskiNetwork):
         kernel_size=3,
         stride=2,
         dilation=1,
-        has_bias=False,
+        bias=False,
         region_type=REGION_TYPE,
         dimension=D)
     self.norm2_tr = get_norm(
@@ -1140,7 +1140,7 @@ class ResUNet2SP(ME.MinkowskiNetwork):
         kernel_size=1,
         stride=1,
         dilation=1,
-        has_bias=False,
+        bias=False,
         dimension=D)
 
     # self.block1_tr = BasicBlockBN(TR_CHANNELS[1], TR_CHANNELS[1], bn_momentum=bn_momentum, D=D)
@@ -1151,7 +1151,7 @@ class ResUNet2SP(ME.MinkowskiNetwork):
         kernel_size=1,
         stride=1,
         dilation=1,
-        has_bias=True,
+        bias=True,
         dimension=D)
 
   def forward(self, x):
@@ -1205,8 +1205,8 @@ class ResUNet2SP(ME.MinkowskiNetwork):
     if self.normalize_feature:
       return ME.SparseTensor(
           out.F / (torch.norm(out.F, p=2, dim=1, keepdim=True) + 1e-8),
-          coords_key=out.coords_key,
-          coords_manager=out.coords_man)
+          coordinate_map_key=out.coords_key,
+          coordinate_manager=out.coords_man)
     else:
       return out
 
@@ -1218,4 +1218,4 @@ class ResUNetBN2SPC(ResUNet2SP):
 
 
 class ResUNetBN2SPCX(ResUNetBN2SPC):
-  REGION_TYPE = ME.RegionType.HYPERCROSS
+  REGION_TYPE = ME.RegionType.HYPER_CROSS
diff --git a/scripts/train_3dmatch.sh b/scripts/train_3dmatch.sh
index 668a7ca..71a9cf0 100755
--- a/scripts/train_3dmatch.sh
+++ b/scripts/train_3dmatch.sh
@@ -64,8 +64,8 @@ python train.py \
        $MISC_ARGS 2>&1 | tee -a $LOG
 
 # Test
-python -m scripts.test_3dmatch \
-       $MISC_ARGS \
-       --threed_match_dir ${THREED_MATCH_DIR} \
-       --weights ${OUT_DIR}/best_val_checkpoint.pth \
-       2>&1 | tee -a $LOG
+#python -m scripts.test_3dmatch \
+#      $MISC_ARGS \
+#      --threed_match_dir ${THREED_MATCH_DIR} \
+#      --weights ${OUT_DIR}/best_val_checkpoint.pth \
+#      2>&1 | tee -a $LOG
root@3f73bd4901b9:/app/DeepGlobalRegistration# echo "" | tee -a $LOG

root@3f73bd4901b9:/app/DeepGlobalRegistration# nvidia-smi | tee -a $LOG
Tue Mar  2 12:10:42 2021       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 450.102.04   Driver Version: 450.102.04   CUDA Version: 11.0     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  GeForce RTX 2070    Off  | 00000000:08:00.0 Off |                  N/A |
| 29%   31C    P8    14W / 175W |    552MiB /  7981MiB |      5%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
+-----------------------------------------------------------------------------+

Currently i am running into hardware bottlenecks but beside that it seems to work.
Possibly its helpful for someone

Is there any way to reduce the hardware requirements of fcgf & dgr ?

When I run demo.py, I get this error. My environment configuration is, torch1.9.0, torchvision1.10.0, MinkowskiEngine0.5.4, RTX3090Ti.
Does anyone have a similar issue or have any idea to solve it?
Thank you very much!

Hi, Thank you for the interesting work.
Recently, I want to train the work in my dataset, but i don't know the key elements of the dataset for training your excellent work.
Please help me,thanks!

thanks your work !
I want to know how to define the the absolute trajectory error (ATE) in section 5.2. Multi-way Registration and how to calculate it?
Is it translation error between the rigid tranformation first fragment to the last fragment ?
Ground-truth Trajectory in http://redwood-data.org/indoor/dataset.html

Hi, @chrischoy and @theNded , thanks for sharing this code!

It would be helpful if you can recommend/share me with some implementations on the RANSAC, Go-ICP methods?

thanks your work.
In this section
5.2. Multi-way Registration
Multi-way registration for RGB-D scans proceeds via multiple stages. First, the pipeline estimates the camera pose via off-the-shelf odometry and integrates multiple 3D scans to reduce noise and generate accurate 3D fragments of a scene. Next, a pairwise registration algorithm roughly aligns all fragments, followed by multi-way registration [6] which optimizes fragment poses with robust pose graph optimization.
I want to know in the fisrt stage, in order to generate accurate 3D fragments
of a scene and make sure that each fragment contains enough geometric features, how to set the n_frames_per_fragment of open3d ?
thanks

Hi,

Thanks for sharing the code. I'm trying to fine-tune DGR on another dataset. So I modify the dataloader which produces same format data as the data provided by threedmatch_loader. But I found that the output of the ME.SparseTensor has different shape with the input tensor.

To be specific the error lies in this line.

Here is the shape of the tensors, I notice that there are a lot of lines are the same in the tensor iC0. The replicate line issue also happens in the threedmatch data, but the output and the input is same. Do you have any clue about how to solve this issue? Thanks in advance.

Best,

I followed your "Section 5.2. Multiway Registration" details and I implemented DGR as the pairwise registration module in Open3D multiway registration example. When I tested on the Indoor LIDAR dataset (e.g. bedroom) it is taking forever for pairwise registration due to 220 fragments. Also, for many examples in the dataset, when I use the first few (6 nos.) fragments, the registration is bad.

How long did it take for your implementation?

Could you please share the multiway registration module that you used for your paper?

when i run demo.py, i had the issue as following:
/media/turing/C236399636398D03/hyx/yes/envs/dgr37/lib/python3.7/site-packages/MinkowskiEngine-0.5.0-py3.7-linux-x86_64.egg/MinkowskiEngine/init.py:210: UserWarning: The MinkowskiEngine was compiled with CPU_ONLY flag. If you want to compile with CUDA support, make sure torch.cuda.is_available() is True when you install MinkowskiEngine.
"If you want to compile with CUDA support, make sure torch.cuda.is_available() is True when you install MinkowskiEngine.",
=> loading checkpoint 'ResUNetBN2C-feat32-kitti-v0.3.pth'
=> Setting voxel size to 0.3
=> loading finished
Traceback (most recent call last):
File "demo.py", line 41, in
T01 = dgr.register(pcd0, pcd1)
File "/home/turing/hyx/DeepGlobalRegistration-MEv0.5/core/deep_global_registration.py", line 241, in register
xyz0, coords0, feats0 = self.preprocess(xyz0)
File "/home/turing/hyx/DeepGlobalRegistration-MEv0.5/core/deep_global_registration.py", line 151, in preprocess
xyz = torch.from_numpy(xyz[sel])
IndexError: shape mismatch: indexing arrays could not be broadcast together with shapes (197,3) (197,)

can anybody helps me?

Hello,

Thank you for your great work.
In file https://github.com/chrischoy/DeepGlobalRegistration/blob/master/dataloader/kitti_loader.py
from line 176 to 182.

I don't see you scale the ground truth translation. Is it a bug?

Thanks for sharing the source code of this paper.

I am exploring the dataset and I found a weird behaviour related to the minimum distance between poses in the KITTI Odometry dataset.

I observed that the translation vector norm is below the MIN_DIST threshold (following the paper, I used a threshold of 10m) for most of the samples. That means that most of the samples have relative distance below (but close to) the minimum distance of 10m.

To verify this behaviour one can simply inspect the value of pdist[cur_time, next_time] after

I believe the problem is in line

next_time = next_time[0] + curr_time

Can anyone else observe this behaviour?

could you please upload them again?@chrischoy

Hello.

I've tried to load 3DMatch pre-trained model from the demo section using train_3dmatch.py. Script failed. When I tried to debug the crux of the problem. I ended up that the script tried to load *.pth file and failed because the dictionary of weights doesn't have the field inlier_feature_type in network config here.

Is there any way to work around this issue?

When I'm running demo.py,RuntimeError: unexpected EOF, expected 23767609 more bytes. The file might be corrupted.

OR

What should I do to register my two pairs of point clouds?

THANK YOU VERY MUCH!

First off, I'd like to just thank you for making this code base openly available. It is very quick to set up and easy to work with. I was able to reproduce the results on KITTI with little effort. Thank you!

My question is rather specific about the rotation estimation, and I am aware it probably doesn't matter too much given the option to run the refinement on the global reg. result. However, I am curious about the mechanism behind it.

Specifically, while training the network, I noticed that when training DGR on KITTI, it seems like the RRE can never go below approximately 2.5 degrees, even after multiple epochs.

Moreover, I am unable to reduce the error even when overfitting a minibatch of just four KITTI samples. I am able to get the RTE as low as 5cm, but the rotation error never drops below 2.56deg, even after 100+ iterations on the same batch. Reducing the translation weight doesn't help either. It seems every sample gets stuck at 0.0447 rad of rotation error and simply can't go lower.

This never gets any lower even with 10s of iterations:

08/28 20:32:14 Rot error: tensor([0.0447, 0.0447, 0.0447, 0.0447], grad_fn=<AcosBackward>)

Have you encountered this when working on Deep Global Registration? It's not a major issue but I am thinking about whether this could be due to how the model is structured (backpropagating error through the correspondences), or due to some other reason.

Thank you,
Andrei

P.S. Here is an easy way to support overfitting in trainer.py:

First, one can modify get_data:

    def get_data(self, iterator):
        while True:
            if self.overfit:
                if self._cached_data is None:
                    self._cached_data = iterator.next()
                else:
                    print("Returning the same old batch!")

                return self._cached_data
            return iterator.next()

And just set self.overfit = True in the constructor, pass it via the config, etc.

Hello!

Thanks for sharing this excellent work, by the way, which software of tool did you use to obtain the visualization result between 4:21~4:40 in your full CVPR oral presentation video? That would be so much helpful.

Best.

Hi, I would like to ask you about the tests you presented within the paper. In particular, I was trying to reproduce the results you obtained on 3DMatch using FGR. I am currently using the open3d library and I have inserted into deep_global_registration.py the following code:

if self.safeguard_method == 'feat_fast':
distance_threshold = voxel_size * 2.0
T = registration_fast_based_on_fcgf_feature_matching(pcd0, pcd1,
feats0.cpu().numpy(),
feats1.cpu().numpy(),
distance_threshold)

where

def registration_fast_based_on_fcgf_feature_matching(pcd0, pcd1, feats0, feats1,
distance_threshold):
assert feats0.shape[1] == feats1.shape[1]

source_feat = o3d.registration.Feature()
source_feat.resize(feats0.shape[1], len(feats0))
source_feat.data = feats0.astype('d').transpose()

target_feat = o3d.registration.Feature()
target_feat.resize(feats1.shape[1], len(feats1))
target_feat.data = feats1.astype('d').transpose()

result = o3d.registration.registration_fast_based_on_feature_matching(
pcd0, pcd1, source_feat, target_feat,
o3d.registration.FastGlobalRegistrationOption(maximum_correspondence_distance=distance_threshold))

return result.transformation

Which, as you can see, is very similar to the code you already provided (except for Fast instead of RANSAC).

Then, inside the register function of deep_global_registration I call:
pcd0 = make_open3d_point_cloud(xyz0)
pcd1 = make_open3d_point_cloud(xyz1)
T = self.safeguard_registration(pcd0,
pcd1,
corres_idx0,
corres_idx1,
feats0,
feats1,
self.voxel_size,
num_iterations=self.config.safeguard_num_iterations)

Everything else is untouched with respect with your code.
Unfortunately I am having very poor results: the overall accuracy for instance is around 5%.
The version of 3DMatch should be ok, in fact I see the same results as yours with DGR and even greater results when runnin RANSAC based on correspondences instead of features matching.
Do you have any idea about what could be the issue? Can you tell me which implementation of FAST did you use or which parameters?

Thank you very much for your time,
Marco