Hello @Lilac-Lee ,

I'm running the test.py on threedmatch dataset with all your default settings in the code, but got the following error, can you please help me with this?

`(env_lk_revisited) mingli@tarsserver4:~/env_lk_revisited/PointNetLK_Revisited$ python test.py

init_trans shape is (627, 6)

Traceback (most recent call last):

File "test.py", line 154, in

main(ARGS)

File "test.py", line 110, in main

test(args, testset, dptnetlk)

File "test.py", line 104, in test

dptnetlk.test_one_epoch(model, testloader, args.device, 'test', args.data_type, args.vis)

File "/home/mingli/env_lk_revisited/PointNetLK_Revisited/trainer.py", line 122, in
test_one_epoch
_ = model.AnalyticalPointNetLK.do_forward(ptnetlk, voxel_features_p0, voxel_coords_p0,

File "/home/mingli/env_lk_revisited/PointNetLK_Revisited/model.py", line 189, in
do_forward
r = net(q0, q1, mode, maxiter=maxiter, xtol=xtol, voxel_coords_diff=voxel_coords_diff, data_type=data_type, num_random_points=num_random_points)

File "/home/mingli/env_lk_revisited/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)

File "/home/mingli/env_lk_revisited/PointNetLK_Revisited/model.py", line 210, in
forward
r, g, itr = self.iclk_new(g0, p0, p1, maxiter, xtol, mode, voxel_coords_diff=voxel_coords_diff, data_type=data_type, num_random_points=num_random_points)

File "/home/mingli/env_lk_revisited/PointNetLK_Revisited/model.py", line 296, in
iclk_new
J = self.Cal_Jac(Mask_fn, A_fn, Ax_fn, BN_fn, max_idx,
File "/home/mingli/env_lk_revisited/PointNetLK_Revisited/model.py", line 260, in
Cal_Jac
J = torch.einsum('ij,jk->ik', J_, warp_condition).unsqueeze(0) # 1 X K X 6

File "/home/mingli/env_lk_revisited/lib/python3.8/site-packages/torch/functional.py", line 327, in
einsum
return _VF.einsum(equation, operands) # type: ignore[attr-defined]

RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu! (when checking argument for argument mat2 in method wrapper__bmm)
`

Thank you in advance.

Hi,

First of all, thank you for your great paper and codes, and also the trained model you provided.

I trained the model on ModelNet40 using the default settings in your code twice. Then I found the performance on the evaluation set of my best model is much worse than the results in Table 1 in the paper.

> ********************mean********************
> rot_MSE: 39.06458769078561, rot_RMSE: 6.250167013031381, rot_MAE: 1.0076209857277225, trans_MSE: 0.004946976434439421, trans_RMSE: 0.07033474743366241, trans_MAE: 0.009108548983931541
> ********************median********************
> rot_MSE: 8.32185003866275e-12, rot_RMSE: 2.884761695298721e-06, rot_MAE: 2.884761695298721e-06, trans_MSE: 3.552713678800501e-15, trans_RMSE: 5.960464477539063e-08, trans_MAE: 5.960464477539063e-08

The test was performed on the second half categories with 1000 points sampled. I also test your provided trained model with the same setting and got the same performance as in the paper:

********************mean********************
rot_MSE: 25.622303942241103, rot_RMSE: 5.061847878220078, rot_MAE: 0.9258663473932836, trans_MSE: 0.0031295332591980696, trans_RMSE: 0.05594222992658615, trans_MAE: 0.008093184791505337
********************median********************
rot_MSE: 4.499777552562683e-12, rot_RMSE: 2.121267911547875e-06, rot_MAE: 2.121267911547875e-06, trans_MSE: 1.9984014443252818e-15, trans_RMSE: 4.470348358154297e-08, trans_MAE: 4.470348358154297e-08

I guess this proves that adding point sampling during testing doesn't harm the performance?

Therefore, I would like to know more details about how you train and test over ModelNet40:

1. Is the best model selected by total test loss or test transformation loss?
2. In the paper you mentioned that

We also partitioned 20% of the training set for evaluation.
Does it mean that you used 20% training + the whole test set for validation during training and testing? Or did you only use the 20% training set for training and testing?

3. Are there any additional settings for testing on ModelNet40? I see the code is for 3DMatch in default.

Thanks again.

Hi Authors,

Thanks for your great work!
During training, I want to utilize other geometric features (e.g. normal) except the coordinates (x, y, z). This is, the input shape will become (B, N, 6) instead of (B, N, 3). After doing this extension, there is some problems about the calculation of the jacobi matrix. I still only calculate gradient with respect to x, y, and z: A1 = self.mlp1[0].weight[:, :3, :] . Does it make sense?

Thanks!

In data_utils.py, as find_voxel_overlaps(p0, p1, voxel) used in class ThreeDMatch_Testing , the additional prior knowledge has been added into test dataloader because the overlap relationship of voxels is acquired in registrated point cloud pairs (p0_pre, p1_pre). Noted that the perturbation (random transform) are carried out after this operation, the whole alogrithm cannot acquire overlap relationship in unregistrated pairs. Some relevant codes are shown as follows:

def find_voxel_overlaps(p0, p1, voxel):
    xmin, ymin, zmin = np.max(np.stack([np.min(p0, 0), np.min(p1, 0)]), 0)
    xmax, ymax, zmax = np.min(np.stack([np.max(p0, 0), np.max(p1, 0)]), 0)
    
    # truncate the point cloud
    eps = 1e-6
    p0_ = p0[np.all(p0>[xmin+eps,ymin+eps,zmin+eps], axis=1) & np.all(p0<[xmax-eps,ymax-eps,zmax-eps], axis=1)]
    p1_ = p1[np.all(p1>[xmin+eps,ymin+eps,zmin+eps], axis=1) & np.all(p1<[xmax-eps,ymax-eps,zmax-eps], axis=1)]
    
    # recalculate the constraints
    xmin, ymin, zmin = np.max(np.stack([np.min(p0, 0), np.min(p1, 0)]), 0)
    xmax, ymax, zmax = np.min(np.stack([np.max(p0, 0), np.max(p1, 0)]), 0)
    vx = (xmax - xmin) / voxel
    vy = (ymax - ymin) / voxel
    vz = (zmax - zmin) / voxel
    
    return p0_, p1_, xmin, ymin, zmin, xmax, ymax, zmax, vx, vy, vz


class ThreeDMatch_Testing(torch.utils.data.Dataset):
    def __init__(self, dataset_path, category, overlap_ratio, voxel_ratio, voxel, max_voxel_points, num_voxels, rigid_transform, vis):
        self.dataset_path = dataset_path
        self.pairs = []
        with open(category, 'r') as fi:
            cinfo_fi = fi.read().split()   # category names
            for i in range(len(cinfo_fi)):
                cat_name = cinfo_fi[i]
                cinfo_name = cat_name + '*%.2f.txt' % overlap_ratio
                cinfo = glob.glob(os.path.join(self.dataset_path, cinfo_name))
                for fi_name in cinfo:
                    with open(fi_name) as fi:
                        fi_list = [x.strip().split() for x in fi.readlines()]
                    for fi in fi_list:
                        self.pairs.append([fi[0], fi[1]])
                        
        self.voxel_ratio = voxel_ratio
        self.voxel = int(voxel)
        self.max_voxel_points = max_voxel_points
        self.num_voxels = num_voxels
        self.perturbation = load_pose(rigid_transform, len(self.pairs))
        self.vis = vis

    def __len__(self):
        return len(self.pairs)

    def do_transform(self, p0, x):
        # p0: [N, 3]
        # x: [1, 6], twist-params
        g = utils.exp(x).to(p0) # [1, 4, 4]
        p1 = utils.transform(g, p0)
        igt = g.squeeze(0) # igt: p0 -> p1
        return p1, igt

    def __getitem__(self, index):
        p0_pre, p1_pre = load_3dmatch_batch_data(os.path.join(self.dataset_path, self.pairs[index][0]), os.path.join(self.dataset_path, self.pairs[index][1]), self.voxel_ratio)
        
        # voxelization
        p0, p1, xmin, ymin, zmin, xmax, ymax, zmax, vx, vy, vz = find_voxel_overlaps(p0_pre, p1_pre, self.voxel)   # constraints of P1 ^ P2, where contains roughly overlapped area
        voxels_p0, coords_p0, num_points_per_voxel_p0 = points_to_voxel_second(p0, (xmin, ymin, zmin, xmax, ymax, zmax), 
                        (vx, vy, vz), self.max_voxel_points, reverse_index=False, max_voxels=self.num_voxels)
        voxels_p1, coords_p1, num_points_per_voxel_p1 = points_to_voxel_second(p1, (xmin, ymin, zmin, xmax, ymax, zmax), 
                        (vx, vy, vz), self.max_voxel_points, reverse_index=False, max_voxels=self.num_voxels)
        
        coords_p0_idx = coords_p0[:,1]*(int(self.voxel**2)) + coords_p0[:,0]*(int(self.voxel)) + coords_p0[:,2]
        coords_p1_idx = coords_p1[:,1]*(int(self.voxel**2)) + coords_p1[:,0]*(int(self.voxel)) + coords_p1[:,2]
        
        # calculate for the voxel medium
        xm_x = np.linspace(xmin+vx/2, xmax-vx/2, int(self.voxel))
        xm_y = np.linspace(ymin+vy/2, ymax-vy/2, int(self.voxel))
        xm_z = np.linspace(zmin+vz/2, zmax-vz/2, int(self.voxel))
        mesh3d = np.vstack(np.meshgrid(xm_x,xm_y,xm_z)).reshape(3,-1).T
        voxel_coords_p0 = mesh3d[coords_p0_idx]
        voxel_coords_p1 = mesh3d[coords_p1_idx]
        
        # find voxels where number of points >= 90% of the maximum number of points
        idx_conditioned_p0 = coords_p0_idx[np.where(num_points_per_voxel_p0>=0.1*self.max_voxel_points)]
        idx_conditioned_p1 = coords_p1_idx[np.where(num_points_per_voxel_p1>=0.1*self.max_voxel_points)]
        idx_conditioned, _, _ = np.intersect1d(idx_conditioned_p0, idx_conditioned_p1, assume_unique=True, return_indices=True)
        _, _, idx_p0 = np.intersect1d(idx_conditioned, coords_p0_idx, assume_unique=True, return_indices=True)
        _, _, idx_p1 = np.intersect1d(idx_conditioned, coords_p1_idx, assume_unique=True, return_indices=True)
        voxel_coords_p0 = voxel_coords_p0[idx_p0]
        voxel_coords_p1 = voxel_coords_p1[idx_p1]
        voxels_p0 = voxels_p0[idx_p0]
        voxels_p1 = voxels_p1[idx_p1]
        
        x = torch.from_numpy(self.perturbation[index][np.newaxis,...])
        voxels_p1_, igt = self.do_transform(torch.from_numpy(voxels_p1.reshape(-1,3)), x)
        voxels_p1 = voxels_p1_.reshape(voxels_p1.shape)
        voxel_coords_p1, _ = self.do_transform(torch.from_numpy(voxel_coords_p1).double(), x)
        p1, _ = self.do_transform(torch.from_numpy(p1), x)
        
        if self.vis:
            return voxels_p0, voxel_coords_p0, voxels_p1, voxel_coords_p1, igt, p0, p1
        else:    
            return voxels_p0, voxel_coords_p0, voxels_p1, voxel_coords_p1, igt

My Q:

In synthetic, no problem, as the zero-mean is with point cloud mean which merely need one point cloud itself. But on 3DMatch test, the zero-mean is with voxel-mean. And this voxel-mean is taken from two aligned point cloud. This is the difference and that's why I say the bug is only on voxelization.

"then the p1 is transformed back with a pose "x" for registration" . For coarse-to-fine registration, P---T1--->P^{1}---T2--->P^{2}.
But in the 3DMatch test, it is P---T1--->P^{1}---x--->P---PointNetLK_Revisited--->P^{2}.
T1 is somehow global result, P^{1} is one well aligned point cloud. Then with a "x", you align it back to a non-aligned cases P. >> This is in the data preprocessing part.

I also checked, with voxel_zero_mean=True and voxel=1 on the 3DMatch test, it is like the PointNetLK_revisited merely solve rotation during the registration. Please check!

Lilac-Lee's:

zero_mean is required by our method. The voxelization part is the same analog as the synthetic part. voxel_mean is required when computing global Jacobian.
Our method solves both rotation and translation, and the estimated translation compensates for the deducted mean.

Hi @Lilac-Lee,

Thank you very much for releasing your project code! I have a question about how the voxel overlaps are found in the testing phase.

As far as I read the code, the following lines are for finding the voxel overlaps, but it seems that the input data, namely p0 and p1 here, are aligned.

Also, it seems that the random rigid transformation is applied after the voxel overlap is found.

According to my understanding, this seems a bit weird because we do not know the ground-truth posture in the testing phase and cannot compute the voxel overlaps in this way.

Excuse me if there is something wrong in my understanding. I'd appreciate if you explain how the voxel overlap is found in the testing phase.

I download ModelNet40 dataset and run train.py for three times. No warnings or errors for my running but the train loss and val loss were both turned to nan (always happens >100 epochs) and the training processing continues until 200 epochs. I use the test_topysample.ipynb to verify my trained model and the success flags are all zeros, which means the failure of training. logs are shown as follows:
DEBUG:PointNetLK-Revisited, 2022-03-10 16:41:38,117, Training
Namespace(batch_size=32, categoryfile='./dataset/modelnet40_half1.txt', clip=0.0, data_type='synthetic', dataset_path='./dataset/ModelNet', dataset_type='modelnet', decay_rate=0.0001, device='cuda:0', dim_k=1024, embedding='pointnet', logfile='./logs/2022_0417.log', lr=0.001, mag=0.8, max_epochs=200, max_iter=10, num_points=1000, num_random_points=100, optimizer='Adam', outfile='./logs/2021_04_17_train_modelnet', pretrained='', resume='', sigma=0.0, start_epoch=0, workers=8)
DEBUG:PointNetLK-Revisited, 2022-03-10 16:41:41,254, Begin Training!
INFO:PointNetLK-Revisited, 2022-03-10 16:44:36,515, epoch, 0001, 0.170037, 0.054935, 0.001698, 0.001877
INFO:PointNetLK-Revisited, 2022-03-10 16:47:03,908, epoch, 0002, 0.130081, 0.588912, 0.001903, 0.000955
INFO:PointNetLK-Revisited, 2022-03-10 16:49:46,033, epoch, 0003, 0.094048, 0.108086, 0.001868, 0.001699
INFO:PointNetLK-Revisited, 2022-03-10 16:52:16,404, epoch, 0004, 0.148174, 0.026301, 0.001927, 0.000721
INFO:PointNetLK-Revisited, 2022-03-10 16:54:53,449, epoch, 0005, 0.303678, 0.119922, 0.002032, 0.002151
INFO:PointNetLK-Revisited, 2022-03-10 16:57:26,052, epoch, 0006, 0.145921, 0.221103, 0.005942, 0.004791
INFO:PointNetLK-Revisited, 2022-03-10 16:59:55,651, epoch, 0007, 0.313405, 0.249224, 0.005145, 0.011014
DEBUG:PointNetLK-Revisited, 2022-03-10 17:10:44,781, Training
Namespace(batch_size=32, categoryfile='./dataset/modelnet40_half1.txt', clip=0.0, data_type='synthetic', dataset_path='./dataset/ModelNet', dataset_type='modelnet', decay_rate=0.0001, device='cuda:0', dim_k=1024, embedding='pointnet', logfile='./logs/2022_0417.log', lr=0.001, mag=0.8, max_epochs=200, max_iter=10, num_points=1000, num_random_points=100, optimizer='Adam', outfile='./logs/2021_04_17_train_modelnet', pretrained='', resume='/data2/github/PointNetLK_Revisited-main/logs/2021_04_17_train_modelnet_snap_last.pth', sigma=0.0, start_epoch=0, workers=8)
DEBUG:PointNetLK-Revisited, 2022-03-10 17:10:48,040, Begin Training!
INFO:PointNetLK-Revisited, 2022-03-10 17:13:40,535, epoch, 0008, 0.187465, 0.095352, 0.002529, 0.003191
INFO:PointNetLK-Revisited, 2022-03-10 17:16:12,450, epoch, 0009, 0.291860, 0.274840, 0.005754, 0.003754
INFO:PointNetLK-Revisited, 2022-03-10 17:18:47,849, epoch, 0010, 0.303363, 0.018148, 0.004605, 0.000430
INFO:PointNetLK-Revisited, 2022-03-10 17:21:22,431, epoch, 0011, 0.156910, 0.199597, 0.005701, 0.004699
INFO:PointNetLK-Revisited, 2022-03-10 17:23:57,827, epoch, 0012, 0.092899, 0.294203, 0.003480, 0.005942
INFO:PointNetLK-Revisited, 2022-03-10 17:26:32,264, epoch, 0013, 0.098645, 0.019350, 0.005084, 0.001453
INFO:PointNetLK-Revisited, 2022-03-10 17:29:02,500, epoch, 0014, 0.169169, 0.091433, 0.003667, 0.002034
INFO:PointNetLK-Revisited, 2022-03-10 17:31:35,792, epoch, 0015, 0.783368, 0.311817, 0.008565, 0.001611
INFO:PointNetLK-Revisited, 2022-03-10 17:34:11,050, epoch, 0016, 0.287108, 0.899135, 0.001803, 0.009333
INFO:PointNetLK-Revisited, 2022-03-10 17:36:47,262, epoch, 0017, 0.130214, 0.147271, 0.004243, 0.001361
INFO:PointNetLK-Revisited, 2022-03-10 17:39:22,997, epoch, 0018, 0.149582, 0.158802, 0.004017, 0.008990
INFO:PointNetLK-Revisited, 2022-03-10 17:42:00,531, epoch, 0019, 0.781333, 0.038243, 0.004229, 0.000619
INFO:PointNetLK-Revisited, 2022-03-10 17:44:24,212, epoch, 0020, 16.817535, 0.466524, 0.003127, 0.007263
INFO:PointNetLK-Revisited, 2022-03-10 17:47:00,055, epoch, 0021, 0.154796, 0.005723, 0.001608, 0.001008
INFO:PointNetLK-Revisited, 2022-03-10 17:49:34,445, epoch, 0022, 0.281341, 0.039673, 0.001210, 0.000819
INFO:PointNetLK-Revisited, 2022-03-10 17:52:10,130, epoch, 0023, 0.069749, 0.004233, 0.001680, 0.001241
INFO:PointNetLK-Revisited, 2022-03-10 17:54:35,704, epoch, 0024, 0.033693, 0.007198, 0.001412, 0.000674
INFO:PointNetLK-Revisited, 2022-03-10 17:57:07,702, epoch, 0025, 0.257840, 0.142363, 0.005358, 0.000670
INFO:PointNetLK-Revisited, 2022-03-10 17:59:42,514, epoch, 0026, 0.211333, 0.834852, 0.001297, 0.009504
INFO:PointNetLK-Revisited, 2022-03-10 18:02:16,506, epoch, 0027, 0.071101, 0.037572, 0.002009, 0.001167
INFO:PointNetLK-Revisited, 2022-03-10 18:04:48,060, epoch, 0028, 0.091675, 0.363744, 0.001870, 0.003710
INFO:PointNetLK-Revisited, 2022-03-10 18:07:21,949, epoch, 0029, 0.074227, 0.101162, 0.003011, 0.000524
INFO:PointNetLK-Revisited, 2022-03-10 18:09:58,866, epoch, 0030, 0.144340, 0.800961, 0.001821, 0.001563
INFO:PointNetLK-Revisited, 2022-03-10 18:12:26,100, epoch, 0031, 0.074152, 0.245757, 0.004200, 0.002968
INFO:PointNetLK-Revisited, 2022-03-10 18:14:58,278, epoch, 0032, 0.131790, 0.011855, 0.002121, 0.002741
INFO:PointNetLK-Revisited, 2022-03-10 18:17:28,613, epoch, 0033, 0.084018, 0.274896, 0.001996, 0.000231
INFO:PointNetLK-Revisited, 2022-03-10 18:19:58,861, epoch, 0034, 0.150297, 0.019243, 0.002860, 0.001308
INFO:PointNetLK-Revisited, 2022-03-10 18:22:35,994, epoch, 0035, 0.024355, 0.003997, 0.004953, 0.001020
INFO:PointNetLK-Revisited, 2022-03-10 18:25:05,586, epoch, 0036, 0.169360, 0.082234, 0.003700, 0.000431
INFO:PointNetLK-Revisited, 2022-03-10 18:27:35,678, epoch, 0037, 0.026734, 0.006190, 0.002101, 0.002517
INFO:PointNetLK-Revisited, 2022-03-10 18:30:07,666, epoch, 0038, 0.029737, 0.010483, 0.002160, 0.000591
INFO:PointNetLK-Revisited, 2022-03-10 18:32:43,218, epoch, 0039, 0.020351, 0.010210, 0.001612, 0.000980
INFO:PointNetLK-Revisited, 2022-03-10 18:35:12,212, epoch, 0040, 0.015912, 0.017028, 0.000875, 0.000351
INFO:PointNetLK-Revisited, 2022-03-10 18:37:41,959, epoch, 0041, 0.045666, 0.015000, 0.002034, 0.003338
INFO:PointNetLK-Revisited, 2022-03-10 18:40:17,487, epoch, 0042, 0.062232, 0.063447, 0.002472, 0.003605
INFO:PointNetLK-Revisited, 2022-03-10 18:42:56,331, epoch, 0043, 0.046416, 0.022350, 0.001514, 0.001296
INFO:PointNetLK-Revisited, 2022-03-10 18:45:34,813, epoch, 0044, 0.087938, 0.087670, 0.001750, 0.001418
INFO:PointNetLK-Revisited, 2022-03-10 18:48:19,179, epoch, 0045, 0.052378, 0.018045, 0.001702, 0.000874
INFO:PointNetLK-Revisited, 2022-03-10 18:50:52,696, epoch, 0046, 0.051639, 0.077453, 0.003012, 0.000862
INFO:PointNetLK-Revisited, 2022-03-10 18:53:27,192, epoch, 0047, 0.041800, 0.253327, 0.001751, 0.000244
INFO:PointNetLK-Revisited, 2022-03-10 18:56:07,068, epoch, 0048, 0.074570, 0.025479, 0.002340, 0.001880
INFO:PointNetLK-Revisited, 2022-03-10 18:58:53,588, epoch, 0049, 0.075841, 0.001447, 0.002117, 0.000992
INFO:PointNetLK-Revisited, 2022-03-10 19:01:23,013, epoch, 0050, 0.039845, 0.270906, 0.001372, 0.002702
INFO:PointNetLK-Revisited, 2022-03-10 19:03:59,507, epoch, 0051, 0.130819, 0.158293, 0.002643, 0.003603
INFO:PointNetLK-Revisited, 2022-03-10 19:06:31,804, epoch, 0052, 17.802472, 0.132097, 0.003683, 0.002070
INFO:PointNetLK-Revisited, 2022-03-10 19:08:59,888, epoch, 0053, 0.075139, 0.010002, 0.001538, 0.001857
INFO:PointNetLK-Revisited, 2022-03-10 19:11:33,959, epoch, 0054, 0.152226, 0.044727, 0.002686, 0.000697
INFO:PointNetLK-Revisited, 2022-03-10 19:14:02,254, epoch, 0055, 0.068913, 0.074569, 0.002104, 0.000211
INFO:PointNetLK-Revisited, 2022-03-10 19:16:32,868, epoch, 0056, 0.121622, 0.042703, 0.003145, 0.000394
INFO:PointNetLK-Revisited, 2022-03-10 19:19:07,215, epoch, 0057, 0.100887, 0.509697, 0.001300, 0.000193
INFO:PointNetLK-Revisited, 2022-03-10 19:21:39,852, epoch, 0058, 0.240458, 0.015640, 0.003049, 0.000630
INFO:PointNetLK-Revisited, 2022-03-10 19:24:07,791, epoch, 0059, 0.074161, 0.009442, 0.002788, 0.000542
INFO:PointNetLK-Revisited, 2022-03-10 19:26:44,856, epoch, 0060, 0.375591, 0.128816, 0.002333, 0.001660
INFO:PointNetLK-Revisited, 2022-03-10 19:29:16,400, epoch, 0061, 0.184603, 0.518123, 0.001991, 0.004989
INFO:PointNetLK-Revisited, 2022-03-10 19:31:58,314, epoch, 0062, 0.166427, 0.476962, 0.001698, 0.002283
INFO:PointNetLK-Revisited, 2022-03-10 19:34:34,540, epoch, 0063, 0.121247, 0.045517, 0.001697, 0.001985
INFO:PointNetLK-Revisited, 2022-03-10 19:36:58,583, epoch, 0064, 0.187386, 0.305053, 0.002426, 0.000176
INFO:PointNetLK-Revisited, 2022-03-10 19:39:27,012, epoch, 0065, 0.122354, 0.015075, 0.001960, 0.002454
INFO:PointNetLK-Revisited, 2022-03-10 19:42:01,193, epoch, 0066, 0.172558, 1.074466, 0.002626, 0.000355
INFO:PointNetLK-Revisited, 2022-03-10 19:44:29,733, epoch, 0067, 0.136128, 0.011255, 0.002363, 0.000335
INFO:PointNetLK-Revisited, 2022-03-10 19:47:00,298, epoch, 0068, 0.036978, 0.365131, 0.003330, 0.000678
INFO:PointNetLK-Revisited, 2022-03-10 19:49:36,140, epoch, 0069, 0.119210, 0.035312, 0.002501, 0.000115
INFO:PointNetLK-Revisited, 2022-03-10 19:52:10,512, epoch, 0070, 0.149505, 0.019858, 0.004186, 0.000355
INFO:PointNetLK-Revisited, 2022-03-10 19:54:40,848, epoch, 0071, 0.035973, 0.259074, 0.001990, 0.001969
INFO:PointNetLK-Revisited, 2022-03-10 19:57:15,074, epoch, 0072, 13.114001, 0.004850, 0.003913, 0.000362
INFO:PointNetLK-Revisited, 2022-03-10 19:59:57,950, epoch, 0073, 0.152484, 0.029237, 0.003848, 0.001505
INFO:PointNetLK-Revisited, 2022-03-10 20:02:35,756, epoch, 0074, 0.420640, 0.574864, 0.003005, 0.005267
INFO:PointNetLK-Revisited, 2022-03-10 20:05:15,500, epoch, 0075, 0.017132, 0.039054, 0.001496, 0.001722
INFO:PointNetLK-Revisited, 2022-03-10 20:07:47,578, epoch, 0076, 0.144417, 0.020392, 0.002531, 0.006464
INFO:PointNetLK-Revisited, 2022-03-10 20:10:19,044, epoch, 0077, 0.044139, 0.068189, 0.002943, 0.000016
INFO:PointNetLK-Revisited, 2022-03-10 20:12:53,387, epoch, 0078, 0.037475, 0.565321, 0.001002, 0.000137
DEBUG:PointNetLK-Revisited, 2022-03-11 09:08:17,628, Training
Namespace(batch_size=32, categoryfile='./dataset/modelnet40_half1.txt', clip=0.0, data_type='synthetic', dataset_path='./dataset/ModelNet', dataset_type='modelnet', decay_rate=0.0001, device='cuda:0', dim_k=1024, embedding='pointnet', logfile='./logs/2022_0417.log', lr=0.001, mag=0.8, max_epochs=200, max_iter=10, num_points=1000, num_random_points=100, optimizer='Adam', outfile='./logs/2021_04_17_train_modelnet', pretrained='', resume='/data2/github/PointNetLK_Revisited-main/logs/2021_04_17_train_modelnet_snap_last.pth', sigma=0.0, start_epoch=0, workers=8)
DEBUG:PointNetLK-Revisited, 2022-03-11 09:08:20,674, Begin Training!
INFO:PointNetLK-Revisited, 2022-03-11 09:11:13,297, epoch, 0079, 0.213663, 0.810951, 0.003458, 0.000340
INFO:PointNetLK-Revisited, 2022-03-11 09:13:44,955, epoch, 0080, 0.140009, 0.004876, 0.001254, 0.000302
INFO:PointNetLK-Revisited, 2022-03-11 09:16:11,166, epoch, 0081, 0.038396, 0.047454, 0.000864, 0.001421
INFO:PointNetLK-Revisited, 2022-03-11 09:18:40,489, epoch, 0082, 0.073157, 0.027878, 0.003295, 0.001389
INFO:PointNetLK-Revisited, 2022-03-11 09:21:18,770, epoch, 0083, 0.414847, 0.068837, 0.003179, 0.000230
INFO:PointNetLK-Revisited, 2022-03-11 09:23:48,582, epoch, 0084, 0.021456, 0.011892, 0.002591, 0.001750
INFO:PointNetLK-Revisited, 2022-03-11 09:26:25,660, epoch, 0085, 0.138208, 0.033888, 0.001834, 0.000801
INFO:PointNetLK-Revisited, 2022-03-11 09:28:57,572, epoch, 0086, 0.112301, 0.020767, 0.002049, 0.000849
INFO:PointNetLK-Revisited, 2022-03-11 09:31:23,372, epoch, 0087, 0.176929, 0.110989, 0.002112, 0.001972
INFO:PointNetLK-Revisited, 2022-03-11 09:33:55,483, epoch, 0088, 0.033041, 0.052230, 0.001401, 0.000115
INFO:PointNetLK-Revisited, 2022-03-11 09:36:30,690, epoch, 0089, 0.127450, 0.018011, 0.002422, 0.000114
INFO:PointNetLK-Revisited, 2022-03-11 09:38:58,174, epoch, 0090, 0.024315, 0.008478, 0.001742, 0.000605
INFO:PointNetLK-Revisited, 2022-03-11 09:41:33,921, epoch, 0091, 0.018989, 0.060334, 0.002259, 0.000702
INFO:PointNetLK-Revisited, 2022-03-11 09:43:58,482, epoch, 0092, 0.139491, 0.036031, 0.001820, 0.001375
INFO:PointNetLK-Revisited, 2022-03-11 09:46:35,882, epoch, 0093, 0.323010, 0.009709, 0.001690, 0.000556
INFO:PointNetLK-Revisited, 2022-03-11 09:49:12,343, epoch, 0094, 0.025194, 0.016763, 0.002293, 0.001005
INFO:PointNetLK-Revisited, 2022-03-11 09:51:41,792, epoch, 0095, 0.030998, 0.027101, 0.001826, 0.000837
INFO:PointNetLK-Revisited, 2022-03-11 09:54:23,379, epoch, 0096, 0.052520, 0.001234, 0.002821, 0.000576
INFO:PointNetLK-Revisited, 2022-03-11 09:56:57,805, epoch, 0097, 0.051577, 0.022784, 0.002912, 0.001697
INFO:PointNetLK-Revisited, 2022-03-11 09:59:31,789, epoch, 0098, 0.037557, 0.608210, 0.001920, 0.000635
INFO:PointNetLK-Revisited, 2022-03-11 10:01:59,400, epoch, 0099, 0.110256, 0.029206, 0.002004, 0.004488
INFO:PointNetLK-Revisited, 2022-03-11 10:04:37,261, epoch, 0100, 0.036857, 0.047710, 0.001596, 0.000937
INFO:PointNetLK-Revisited, 2022-03-11 10:07:13,210, epoch, 0101, 0.099490, 0.019117, 0.002162, 0.000831
INFO:PointNetLK-Revisited, 2022-03-11 10:09:46,357, epoch, 0102, 0.050239, 0.026963, 0.001079, 0.000818
INFO:PointNetLK-Revisited, 2022-03-11 10:12:29,466, epoch, 0103, 0.241703, 0.024367, 0.001569, 0.003244
DEBUG:PointNetLK-Revisited, 2022-03-11 10:14:33,155, Training
Namespace(batch_size=32, categoryfile='./dataset/modelnet40_half1.txt', clip=0.0, data_type='synthetic', dataset_path='./dataset/ModelNet', dataset_type='modelnet', decay_rate=0.0001, device='cuda:0', dim_k=1024, embedding='pointnet', logfile='./logs/2022_0417.log', lr=0.001, mag=0.8, max_epochs=200, max_iter=10, num_points=1000, num_random_points=100, optimizer='Adam', outfile='./logs/2021_04_17_train_modelnet', pretrained='', resume='/data2/github/PointNetLK_Revisited-main/logs/2021_04_17_train_modelnet_snap_last.pth', sigma=0.0, start_epoch=0, workers=8)
DEBUG:PointNetLK-Revisited, 2022-03-11 10:14:34,445, Begin Training!
INFO:PointNetLK-Revisited, 2022-03-11 10:17:09,467, epoch, 0104, 0.068009, 0.098571, 0.002464, 0.002341
INFO:PointNetLK-Revisited, 2022-03-11 10:19:51,684, epoch, 0105, 0.110054, 0.004777, 0.002501, 0.000276
INFO:PointNetLK-Revisited, 2022-03-11 10:22:21,286, epoch, 0106, 0.032903, 0.032662, 0.003280, 0.000346
INFO:PointNetLK-Revisited, 2022-03-11 10:24:51,309, epoch, 0107, 0.092826, 0.044243, 0.001972, 0.000902
INFO:PointNetLK-Revisited, 2022-03-11 10:27:30,668, epoch, 0108, 0.048131, 0.003804, 0.002717, 0.000569
INFO:PointNetLK-Revisited, 2022-03-11 10:30:08,487, epoch, 0109, 0.181231, 0.004332, 0.002489, 0.002159
INFO:PointNetLK-Revisited, 2022-03-11 10:32:36,675, epoch, 0110, 0.080696, 0.190642, 0.003342, 0.000658
INFO:PointNetLK-Revisited, 2022-03-11 10:35:08,155, epoch, 0111, 0.026284, 0.003893, 0.003751, 0.000415
INFO:PointNetLK-Revisited, 2022-03-11 10:37:40,882, epoch, 0112, 0.095747, 0.014503, 0.001508, 0.003598
INFO:PointNetLK-Revisited, 2022-03-11 10:40:14,736, epoch, 0113, 0.039804, 0.005039, 0.001781, 0.001496
INFO:PointNetLK-Revisited, 2022-03-11 10:42:45,301, epoch, 0114, 0.039532, 0.015642, 0.001741, 0.000565
INFO:PointNetLK-Revisited, 2022-03-11 10:45:22,360, epoch, 0115, 0.297562, 0.447450, 0.002522, 0.002719
INFO:PointNetLK-Revisited, 2022-03-11 10:48:03,594, epoch, 0116, 0.085900, 0.483639, 0.000913, 0.002395
INFO:PointNetLK-Revisited, 2022-03-11 10:50:39,164, epoch, 0117, 0.054800, 0.011802, 0.003740, 0.001069
INFO:PointNetLK-Revisited, 2022-03-11 10:53:10,576, epoch, 0118, 0.142158, 0.037577, 0.001820, 0.000123
INFO:PointNetLK-Revisited, 2022-03-11 10:55:41,491, epoch, 0119, 0.121025, 0.023546, 0.002158, 0.001661
INFO:PointNetLK-Revisited, 2022-03-11 10:58:12,831, epoch, 0120, 0.123927, 0.005534, 0.001173, 0.000396
INFO:PointNetLK-Revisited, 2022-03-11 11:00:49,405, epoch, 0121, 0.150709, 0.042656, 0.001294, 0.001661
INFO:PointNetLK-Revisited, 2022-03-11 11:03:25,648, epoch, 0122, 0.218797, 0.002490, 0.001837, 0.000429
INFO:PointNetLK-Revisited, 2022-03-11 11:05:54,471, epoch, 0123, 0.099233, 0.128338, 0.002930, 0.002190
INFO:PointNetLK-Revisited, 2022-03-11 11:08:31,192, epoch, 0124, 0.036577, 0.030536, 0.002084, 0.000944
INFO:PointNetLK-Revisited, 2022-03-11 11:11:01,735, epoch, 0125, 50.374570, 0.252508, 0.003859, 0.004319
INFO:PointNetLK-Revisited, 2022-03-11 11:13:33,085, epoch, 0126, 0.380346, 0.222480, 0.002994, 0.001278
INFO:PointNetLK-Revisited, 2022-03-11 11:16:05,745, epoch, 0127, 0.258064, 0.311652, 0.004815, 0.002239
INFO:PointNetLK-Revisited, 2022-03-11 11:18:43,705, epoch, 0128, 0.230647, 0.274638, 0.001881, 0.006320
INFO:PointNetLK-Revisited, 2022-03-11 11:21:15,939, epoch, 0129, 0.618022, 0.156278, 0.004447, 0.000728
INFO:PointNetLK-Revisited, 2022-03-11 11:23:59,808, epoch, 0130, 0.192393, 0.349358, 0.002291, 0.001919
INFO:PointNetLK-Revisited, 2022-03-11 11:26:31,234, epoch, 0131, 0.607722, 0.020067, 0.006311, 0.002016
INFO:PointNetLK-Revisited, 2022-03-11 11:29:02,317, epoch, 0132, 0.043591, 0.067146, 0.001762, 0.001929
INFO:PointNetLK-Revisited, 2022-03-11 11:31:33,317, epoch, 0133, 0.384790, 0.116907, 0.004855, 0.003452
INFO:PointNetLK-Revisited, 2022-03-11 11:34:15,600, epoch, 0134, 0.495460, 0.141017, 0.004015, 0.001058
INFO:PointNetLK-Revisited, 2022-03-11 11:36:53,318, epoch, 0135, 0.155873, 0.174062, 0.002686, 0.000547
INFO:PointNetLK-Revisited, 2022-03-11 11:39:21,723, epoch, 0136, 0.534734, 0.256989, 0.004768, 0.001803
INFO:PointNetLK-Revisited, 2022-03-11 11:41:58,130, epoch, 0137, 0.617583, 0.447041, 0.005822, 0.004474
INFO:PointNetLK-Revisited, 2022-03-11 11:44:36,164, epoch, 0138, 0.761747, 0.321785, 0.007364, 0.001343
INFO:PointNetLK-Revisited, 2022-03-11 11:47:07,615, epoch, 0139, 0.501535, 0.109769, 0.003822, 0.001315
INFO:PointNetLK-Revisited, 2022-03-11 11:49:32,891, epoch, 0140, 0.129618, 0.064163, 0.003040, 0.001869
INFO:PointNetLK-Revisited, 2022-03-11 11:52:02,305, epoch, 0141, 0.138739, 0.090836, 0.003916, 0.000441
INFO:PointNetLK-Revisited, 2022-03-11 11:54:35,215, epoch, 0142, 0.110587, 0.067007, 0.004407, 0.003057
INFO:PointNetLK-Revisited, 2022-03-11 11:57:01,675, epoch, 0143, 0.634760, 0.029948, 0.004851, 0.004554
INFO:PointNetLK-Revisited, 2022-03-11 11:59:34,719, epoch, 0144, 0.121186, 0.536365, 0.003910, 0.005949
INFO:PointNetLK-Revisited, 2022-03-11 12:02:18,761, epoch, 0145, 0.353981, 0.006152, 0.004003, 0.002119
INFO:PointNetLK-Revisited, 2022-03-11 12:05:01,415, epoch, 0146, 0.694202, 0.106386, 0.004823, 0.005594
INFO:PointNetLK-Revisited, 2022-03-11 12:07:45,130, epoch, 0147, 0.288819, 0.250787, 0.004742, 0.005356
INFO:PointNetLK-Revisited, 2022-03-11 12:10:24,566, epoch, 0148, 0.655000, 0.062998, 0.003096, 0.002218
INFO:PointNetLK-Revisited, 2022-03-11 12:12:59,734, epoch, 0149, 0.098685, 0.038985, 0.003580, 0.002261
INFO:PointNetLK-Revisited, 2022-03-11 12:15:28,916, epoch, 0150, 0.556753, 0.506880, 0.003622, 0.004770
INFO:PointNetLK-Revisited, 2022-03-11 12:18:10,964, epoch, 0151, 0.268293, 0.027083, 0.008154, 0.006436
INFO:PointNetLK-Revisited, 2022-03-11 12:20:43,907, epoch, 0152, 0.305062, 0.381557, 0.006124, 0.003802
INFO:PointNetLK-Revisited, 2022-03-11 12:23:26,402, epoch, 0153, 0.494955, 1.399305, 0.005527, 0.006500
INFO:PointNetLK-Revisited, 2022-03-11 12:25:59,838, epoch, 0154, 0.236421, 0.065292, 0.003523, 0.001593
INFO:PointNetLK-Revisited, 2022-03-11 12:28:33,502, epoch, 0155, 0.197157, 1.039769, 0.003955, 0.007868
INFO:PointNetLK-Revisited, 2022-03-11 12:31:07,733, epoch, 0156, 0.165022, 0.320504, 0.003374, 0.009487
INFO:PointNetLK-Revisited, 2022-03-11 12:33:38,229, epoch, 0157, 1.038125, 0.025409, 0.004063, 0.001179
INFO:PointNetLK-Revisited, 2022-03-11 12:36:16,017, epoch, 0158, 0.344455, 1.661829, 0.005228, 0.007076
INFO:PointNetLK-Revisited, 2022-03-11 12:38:52,362, epoch, 0159, 0.413516, 0.298130, 0.005902, 0.004129
INFO:PointNetLK-Revisited, 2022-03-11 12:41:15,916, epoch, 0160, 0.433278, 0.593314, 0.004732, 0.001675
INFO:PointNetLK-Revisited, 2022-03-11 12:43:49,288, epoch, 0161, 0.260577, 1.089357, 0.004256, 0.006866
INFO:PointNetLK-Revisited, 2022-03-11 12:46:19,077, epoch, 0162, 0.319152, 0.106169, 0.004021, 0.001111
INFO:PointNetLK-Revisited, 2022-03-11 12:48:55,346, epoch, 0163, 0.076889, 0.262669, 0.003367, 0.001557
INFO:PointNetLK-Revisited, 2022-03-11 12:51:29,804, epoch, 0164, 0.355431, 1.330229, 0.006717, 0.000919
INFO:PointNetLK-Revisited, 2022-03-11 12:54:07,160, epoch, 0165, 0.249685, 0.695663, 0.004763, 0.007739
INFO:PointNetLK-Revisited, 2022-03-11 12:56:36,640, epoch, 0166, 0.345618, 0.805674, 0.004495, 0.003632
INFO:PointNetLK-Revisited, 2022-03-11 12:59:18,377, epoch, 0167, 0.221269, 0.418254, 0.006726, 0.009431
INFO:PointNetLK-Revisited, 2022-03-11 13:01:54,746, epoch, 0168, 0.260704, 0.217300, 0.006099, 0.000908
INFO:PointNetLK-Revisited, 2022-03-11 13:04:25,513, epoch, 0169, 0.364981, 0.593052, 0.004057, 0.001531
INFO:PointNetLK-Revisited, 2022-03-11 13:07:01,114, epoch, 0170, 0.340524, 0.374222, 0.003959, 0.004853
INFO:PointNetLK-Revisited, 2022-03-11 13:09:29,052, epoch, 0171, 0.734925, 0.187461, 0.003622, 0.005651
INFO:PointNetLK-Revisited, 2022-03-11 13:12:10,843, epoch, 0172, 0.824490, 0.172706, 0.007706, 0.003058
INFO:PointNetLK-Revisited, 2022-03-11 13:14:39,912, epoch, 0173, 0.376474, 0.496356, 0.004651, 0.001374
INFO:PointNetLK-Revisited, 2022-03-11 13:17:17,734, epoch, 0174, 0.562465, 4.021641, 0.004550, 0.006676
INFO:PointNetLK-Revisited, 2022-03-11 13:19:48,412, epoch, 0175, 0.864487, 0.431750, 0.005073, 0.008651
INFO:PointNetLK-Revisited, 2022-03-11 13:22:21,469, epoch, 0176, 0.531715, 0.071059, 0.005800, 0.001760
INFO:PointNetLK-Revisited, 2022-03-11 13:24:56,698, epoch, 0177, 0.266958, 0.391326, 0.005069, 0.002344
INFO:PointNetLK-Revisited, 2022-03-11 13:27:31,032, epoch, 0178, 0.322116, 0.184289, 0.004207, 0.000645
INFO:PointNetLK-Revisited, 2022-03-11 13:30:13,313, epoch, 0179, 0.533469, 0.467534, 0.006579, 0.005560
INFO:PointNetLK-Revisited, 2022-03-11 13:32:54,218, epoch, 0180, nan, nan, nan, nan
INFO:PointNetLK-Revisited, 2022-03-11 13:35:30,005, epoch, 0181, nan, nan, nan, nan
INFO:PointNetLK-Revisited, 2022-03-11 13:38:03,680, epoch, 0182, nan, nan, nan, nan
INFO:PointNetLK-Revisited, 2022-03-11 13:40:41,686, epoch, 0183, nan, nan, nan, nan
INFO:PointNetLK-Revisited, 2022-03-11 13:43:09,421, epoch, 0184, nan, nan, nan, nan
INFO:PointNetLK-Revisited, 2022-03-11 13:45:38,326, epoch, 0185, nan, nan, nan, nan
INFO:PointNetLK-Revisited, 2022-03-11 13:48:09,060, epoch, 0186, nan, nan, nan, nan
INFO:PointNetLK-Revisited, 2022-03-11 13:50:42,674, epoch, 0187, nan, nan, nan, nan
INFO:PointNetLK-Revisited, 2022-03-11 13:53:12,323, epoch, 0188, nan, nan, nan, nan
INFO:PointNetLK-Revisited, 2022-03-11 13:56:00,298, epoch, 0189, nan, nan, nan, nan
INFO:PointNetLK-Revisited, 2022-03-11 13:58:23,408, epoch, 0190, nan, nan, nan, nan
INFO:PointNetLK-Revisited, 2022-03-11 14:00:56,805, epoch, 0191, nan, nan, nan, nan
INFO:PointNetLK-Revisited, 2022-03-11 14:03:33,107, epoch, 0192, nan, nan, nan, nan
INFO:PointNetLK-Revisited, 2022-03-11 14:06:10,206, epoch, 0193, nan, nan, nan, nan
INFO:PointNetLK-Revisited, 2022-03-11 14:08:50,068, epoch, 0194, nan, nan, nan, nan
INFO:PointNetLK-Revisited, 2022-03-11 14:11:20,876, epoch, 0195, nan, nan, nan, nan
INFO:PointNetLK-Revisited, 2022-03-11 14:13:49,485, epoch, 0196, nan, nan, nan, nan
INFO:PointNetLK-Revisited, 2022-03-11 14:16:20,735, epoch, 0197, nan, nan, nan, nan
INFO:PointNetLK-Revisited, 2022-03-11 14:18:54,833, epoch, 0198, nan, nan, nan, nan
INFO:PointNetLK-Revisited, 2022-03-11 14:21:24,647, epoch, 0199, nan, nan, nan, nan
INFO:PointNetLK-Revisited, 2022-03-11 14:23:56,720, epoch, 0200, nan, nan, nan, nan
DEBUG:PointNetLK-Revisited, 2022-03-11 14:23:56,727, Training completed!

Hi,

first of all, thanks a lot for sharing your code! I have some questions about the training process:

1. Do you also use 10 iterations of the IC-LK algorithm during training?

2. In line 222 of trainer.py (quoted below) , you seem to define two different variants for the feature loss. If I understand correctly, the variant when pr is None is what is described in the paper, whereas the second variant seems to compare the feature difference between the last and the previous iteration of the LK algorithm. Can you explain the second variant and give an intuition of which to use when?

        pr = ptnetlk.prev_r
        if pr is not None:
            loss_r = model.AnalyticalPointNetLK.rsq(r - pr)
        else:
            loss_r = model.AnalyticalPointNetLK.rsq(r)

3. I'm a bit confused about how feature-aggregation/random features is implemented:
   The code snippet below seems to implement the splitting strategy for feature computation described in the supplementary material. However, the computed features are overwritten for each new split of the point cloud, so I take it this corresponds to the random feature selection approach? Furthermore, f1 is never used in the following code and f0 is overwritten when the jacobian is computed, so I'm wondering if this computation actually serves any purpose (except for initializing the batch norm layers) or if it is just there for reference?

        # create a data sampler
        if mode != 'test':
            data_sampler = np.random.choice(num_points,  (num_points//num_random_points, num_random_points), replace=False)
        # input through entire pointnet
        if training:
            # first, update BatchNorm modules
            f0 = self.ptnet(p0[:, data_sampler[0], :], 0)
            f1 = self.ptnet(p1[:, data_sampler[0], :], 0)
        self.ptnet.eval()

        if mode != 'test':
            for i in range(1, num_points//num_random_points-1):
                f0 = self.ptnet(p0[:, data_sampler[i], :], i)
                f1 = self.ptnet(p1[:, data_sampler[i], :], i)

4. Random point selection for computing the jacobian:
   I was wondering if it is important to compute the feature vector using the same subset of the pointcloud that was used to compute the jacobian or if it would also be possible to e.g. compute the jacobian on a random subset but the feature vector on the full point cloud?

5. In general, what would be the recommended setup for training? From the supplementary material, it seems that random features + random jacobian gives the best results (and this also seems to be what is implemented), but my initial tests loading the pretrained model and using this setup give relatively poor results (even if I sample more than 100 points), unless I turn on voxelization (which is not practical during training since the number of voxels with points in them is not constant). Any guidance on this?

Thanks again for sharing your code and sorry about the wall of text. I would be super grateful for your help!

When trying to run the train.py on my custom dataset, I am getting the following error:-

"RuntimeError: stack expects each tensor to be equal size, but got [4, 1000, 3] at entry 0 and [2, 1000, 3] at entry 1"

The data loader function, defined in data_utils.py outputs variable size tensors. Because the number of voxels output by function points_to_voxel_second() outputs the variable number of voxels.

Can you suggest some solution for this?

Hi @Lilac-Lee,

When I try running test,py on my computer, it gives me the RuntimeError: CUDA out of memory. This, however, does not happen for train.py. Why does the code require more GPU memory in testing, than in training? The training uses a higher batch size, while the test does not. Is this a bug?

PS - I changed the dataset from 3DMatch to modelnet. The issue remains. The test.py for 3DMatch attempts to allocate 3.91 GB, while test.py for ModelNet attempts to allocate 22.01GB.

Thank you,

Hi ,Lilac-Lee, I found Analytical PointNet-LK consumes much more CUDA memory than the standard PointNet-LK when testing. I have run your original codes for modelnet and 3dmatch. Actually, I have to restrict the maximum number of resampled points from datasets to almost 5000 to fit a 12GB GPU memory (for 3dmatch, num_points per voxel must be lower than 700), or there will be a Out of Mermory Error. Is that normal, or I went somehting wrong?

I want to perform point cloud registration on my own task, how can I only do my own data set, please advise, thank you very much！