open-mmlab / openpcdet Goto Github PK

As shown on the fig, ### rcnn_loss don't decline sharply; on the contrary, rpn_loss worked well.
So, what are the possible reasons ? Thank you for your suggestion.

Hi Sshaoshuai,

Thanks for the great work! I am currently running into an error trying to train with surround point cloud data. I am using the argoverse dataset that has been converted to the same format as KITTI data, all labels have been put in camera coordinate frames and calibration files have been extracted. I've created a config file and redefined all the mean sizes and anchor points. I also created a dataloader and was able to generate the pkl and groundtruth databases.

I am currently able to train with the dataset only when I limit point cloud range to [0, -40, -3, 70.4, 40, 1] (as provided). When I expand point cloud range to [-80,-80,-10, 80, 80, 10] I get the following error:

I am training using a docker image with 8 V100 GPUs.

Thanks!

UPDATE: I had to increase the number of input features to get it to work, is this the correct approach?

File "train.py", line 155, in
main()
File "train.py", line 148, in main
max_ckpt_save_num=args.max_ckpt_save_num
File "/s/dat/UserFolders/ctan24/PCDet-master/tools/train_utils/train_utils.py", line 81, in train_model
leave_pbar=(cur_epoch + 1 == total_epochs)
File "/s/dat/UserFolders/ctan24/PCDet-master/tools/train_utils/train_utils.py", line 36, in train_one_epoch
loss, tb_dict, disp_dict = model_func(model, batch)
File "/s/dat/UserFolders/ctan24/PCDet-master/pcdet/models/init.py", line 25, in model_func
ret_dict, tb_dict, disp_dict = model(input_dict)
File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 493, in call
result = self.forward(*input, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/torch/nn/parallel/distributed.py", line 376, in forward
output = self.module(*inputs[0], **kwargs[0])
File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 493, in call
result = self.forward(*input, **kwargs)
File "/s/dat/UserFolders/ctan24/PCDet-master/pcdet/models/detectors/PartA2_net.py", line 106, in forward
rpn_ret_dict = self.forward_rpn(**input_dict)
File "/s/dat/UserFolders/ctan24/PCDet-master/pcdet/models/detectors/PartA2_net.py", line 38, in forward_rpn
**{'gt_boxes': kwargs.get('gt_boxes', None)}
File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 493, in call
result = self.forward(*input, **kwargs)
File "/s/dat/UserFolders/ctan24/PCDet-master/pcdet/models/bbox_heads/rpn_head.py", line 292, in forward
x = self.blocksi
File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 493, in call
result = self.forward(*input, **kwargs)
File "/s/dat/UserFolders/ctan24/PCDet-master/pcdet/models/model_utils/pytorch_utils.py", line 88, in forward
input = module(input)
File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 493, in call
result = self.forward(*input, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/conv.py", line 338, in forward
self.padding, self.dilation, self.groups)
RuntimeError: Given groups=1, weight of size 128 256 3 3, expected input[1, 1536, 252, 252] to have 256 channels, but got 1536 channels instead

A separate issue that I ran into when training with limited to image FOV is:
UPDATE: Seems like this issue only occur when I train with multiple GPU, I am able to train successfully with one GPU.

Traceback (most recent call last):
File "train.py", line 155, in
main()
File "train.py", line 148, in main
max_ckpt_save_num=args.max_ckpt_save_num
File "/s/dat/UserFolders/ctan24/PCDet-master/tools/train_utils/train_utils.py", line 81, in train_model
leave_pbar=(cur_epoch + 1 == total_epochs)
File "/s/dat/UserFolders/ctan24/PCDet-master/tools/train_utils/train_utils.py", line 36, in train_one_epoch
loss, tb_dict, disp_dict = model_func(model, batch)
File "/s/dat/UserFolders/ctan24/PCDet-master/pcdet/models/init.py", line 25, in model_func
ret_dict, tb_dict, disp_dict = model(input_dict)
File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 493, in call
result = self.forward(*input, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/torch/nn/parallel/distributed.py", line 376, in forward
output = self.module(*inputs[0], **kwargs[0])
File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 493, in call
result = self.forward(*input, **kwargs)
File "/s/dat/UserFolders/ctan24/PCDet-master/pcdet/models/detectors/PartA2_net.py", line 112, in forward
batch_size, voxel_centers, coords, rpn_ret_dict, input_dict
File "/s/dat/UserFolders/ctan24/PCDet-master/pcdet/models/detectors/PartA2_net.py", line 98, in forward_rcnn
rcnn_ret_dict = self.rcnn_net.forward(rcnn_input_dict)
File "/s/dat/UserFolders/ctan24/PCDet-master/pcdet/models/rcnn/partA2_rcnn_net.py", line 547, in forward
targets_dict = self.assign_targets(batch_size, rcnn_dict)
File "/s/dat/UserFolders/ctan24/PCDet-master/pcdet/models/rcnn/partA2_rcnn_net.py", line 27, in assign_targets
targets_dict = proposal_target_layer(rcnn_dict, roi_sampler_cfg=self.rcnn_target_config)
File "/s/dat/UserFolders/ctan24/PCDet-master/pcdet/models/model_utils/proposal_target_layer.py", line 14, in proposal_target_layer
sample_rois_for_rcnn(rois, gt_boxes, roi_raw_scores, roi_labels, roi_sampler_cfg)
File "/s/dat/UserFolders/ctan24/PCDet-master/pcdet/models/model_utils/proposal_target_layer.py", line 141, in sample_rois_for_rcnn
gt_of_bg_rois = cur_gt[gt_assignment[bg_inds]]
IndexError: index is out of bounds for dimension with size 0

Hi Shaoshuai, Thanks for sharing the code. Is the training time of 5 hours produced by 8 GPUs?

Waymo and nuScenes are really good dataset to test.

Hi @sshaoshuai,

I downloaded and saved the four pretrained models into the /PCDet/tools folder.
When I run the following command:

python test.py --cfg_file cfgs/PartA2_car.yaml --batch_size 4 --ckpt PartA2.pth --set MODEL.RPN.BACKBONE.NAME UNetV0 MODEL.RPN.RPN_HEAD.ARGS use_binary_dir_classifier:True

I get this error:

  File "test.py", line 98
    % (wait_second, total_time * 1.0 / 60, args.max_waiting_mins, ckpt_dir), end='', flush=True)
                                                                                ^
SyntaxError: invalid syntax

How can I fix this?

Thank you for your attention.

Rt, Object as Hotspots

Hi, recently i use "d3_box_overlap" in eval.py, to compute the same two 3D boxes 3D IoU. The IoU should be 1.0, but i get 8.3721865e-09. Some thing must be wrong，but i don`t know why.

when i run command:
python train.py --cfg_file cfgs/second.yaml --batch_size 2

I met the problem above ,How can i solved the problem? Thanks.

Hello, could anyone help me understanding what the parameters in the calibration files mean?

P0, P1, P2, P3, R0_rect, Tr_velo_to_cam, Tr_imu_to_velo

Hi, shaoshuai.

I have tried to run both PartA2 and PartA2_fc and get similar results in both car, pedestrian and cyclist classes, as follows.

 Car [email protected], 0.70, 0.70:
bbox AP:95.7255, 89.4638, 89.1119
bev  AP:90.2068, 87.8669, 87.2665
3d   AP:89.3621, 79.2417, 78.5225
aos  AP:95.70, 89.38, 88.96
Car [email protected], 0.70, 0.70:
bbox AP:98.3415, 93.9843, 91.9963
bev  AP:94.8540, 88.7180, 88.4109
3d   AP:92.0074, 82.7198, 80.3088
aos  AP:98.31, 93.87, 91.82
Car [email protected], 0.50, 0.50:
bbox AP:95.7255, 89.4638, 89.1119
bev  AP:95.7246, 89.4733, 89.2078
3d   AP:95.6730, 89.4277, 89.1581
aos  AP:95.70, 89.38, 88.96
Car [email protected], 0.50, 0.50:
bbox AP:98.3415, 93.9843, 91.9963
bev  AP:98.3214, 94.4235, 94.1939
3d   AP:98.2968, 94.3267, 93.9227
aos  AP:98.31, 93.87, 91.82
Pedestrian [email protected], 0.50, 0.50:
bbox AP:70.7935, 67.2700, 63.9463
bev  AP:65.4293, 58.6606, 54.8699
3d   AP:61.8364, 56.1250, 49.9829
aos  AP:67.70, 63.70, 59.98
Pedestrian [email protected], 0.50, 0.50:
bbox AP:72.1216, 67.7394, 63.4741
bev  AP:65.3724, 58.7195, 53.6765
3d   AP:61.7966, 55.4602, 49.3486
aos  AP:68.66, 63.85, 59.09
Pedestrian [email protected], 0.25, 0.25:
bbox AP:70.7935, 67.2700, 63.9463
bev  AP:76.2861, 72.4894, 67.8042
3d   AP:76.2115, 72.3084, 67.6083
aos  AP:67.70, 63.70, 59.98
Pedestrian [email protected], 0.25, 0.25:
bbox AP:72.1216, 67.7394, 63.4741
bev  AP:77.3366, 73.6643, 68.9374
3d   AP:77.2502, 73.4974, 68.7337
aos  AP:68.66, 63.85, 59.09
Cyclist [email protected], 0.50, 0.50:
bbox AP:91.6235, 80.4274, 76.0859
bev  AP:87.3458, 73.9558, 70.8896
3d   AP:85.9091, 71.2065, 68.1585
aos  AP:91.52, 80.08, 75.71
Cyclist [email protected], 0.50, 0.50:
bbox AP:94.3850, 81.4298, 77.3967
bev  AP:91.6213, 74.9996, 71.4178
3d   AP:89.9500, 71.3220, 67.8678
aos  AP:94.26, 81.04, 77.02
Cyclist [email protected], 0.25, 0.25:
bbox AP:91.6235, 80.4274, 76.0859
bev  AP:90.6499, 77.5425, 73.2080
3d   AP:90.6464, 77.5408, 73.2018
aos  AP:91.52, 80.08, 75.71
Cyclist [email protected], 0.25, 0.25:
bbox AP:94.3850, 81.4298, 77.3967
bev  AP:94.0451, 78.3774, 74.3371
3d   AP:94.0441, 78.3747, 74.3336
aos  AP:94.26, 81.04, 77.02

Results on Car are good but results on Pedestrian (61.7966, 55.4602, 49.3486) and Cyclist (85.9091, 71.2065, 68.1585) have a big gap compared to results in the paper, which is (70.37, 63.84, 57.48) on Ped and (88.31, 73.07, 70.20) on Cyclist, especially on the Ped.

Could you please suggest a reason for the degradation?

Hi Shaoshuai, I am running your codes about Part-A2, and I find that the pre-trained model uesd UNet_V0 instead of the default UNet_V2 in config files. However, there is only one difference between them in your codes.

UNet_V0:
https://github.com/sshaoshuai/PCDet/blob/afd40d822b1330bb010a834673a0aa415c9a1053/pcdet/models/rpn/rpn_unet.py#L188

UNet_V2:
https://github.com/sshaoshuai/PCDet/blob/afd40d822b1330bb010a834673a0aa415c9a1053/pcdet/models/rpn/rpn_unet.py#L381

Does it really affects the final performance? Thanks for your reply~

I test my own dataset using the following code
python test.py --cfg_file cfgs/PartA2_car.yaml --batch_size 4 --ckpt PartA2.pth \ --set MODEL.RPN.BACKBONE.NAME UNetV0 MODEL.RPN.RPN_HEAD.ARGS use_binary_dir_classifier:True
But the results is bad that the detected results don't in the correct situation whether in the pointcloud or in the images. Does anyone know where this problem comes from and how I can solve it?

I'm trying to train model, but when I run python train.py --cfg_file cfgs/pointpillar.yaml --batch_size 4
I get this error:
spconv = 1.0
thanks

Hi, Shaoshuai,

Thanks for your wonderful work!

I'm wondering if there is a plan for releasing the code of PV-RCNN?

Thanks!

Nice work.
Model links, and road planes links are not accessible. Need to use VPN?
Thanks.

These libs are not installed in the dockerfile:
'numba',
'tensorboardX',
'easydict',
'pyyaml'

I really understand that the function "boxes3d_to_bevboxes_lidar_torch" aims to convert a 3D box [x, y, z, w, l, h, ry] into the bev form [x1, y1, x2, y2, ry] when conducting NMS. However, there may be some mistakes in the function code. "boxes_bev[:, 0], boxes_bev[:, 1] = cu - half_w, cv - half_l" does not consider yaw angle. I mean "boxes_bev[:, 0]" is not the real x1

2020-03-22 16:34:58,582 INFO Start logging
2020-03-22 16:34:58,584 INFO CUDA_VISIBLE_DEVICES=ALL
2020-03-22 16:34:58,584 INFO cfg_file cfgs/pointpillar.yaml
2020-03-22 16:34:58,585 INFO data_dir None
2020-03-22 16:34:58,586 INFO batch_size 4
2020-03-22 16:34:58,586 INFO epochs 80
2020-03-22 16:34:58,587 INFO workers 4
2020-03-22 16:34:58,588 INFO extra_tag default
2020-03-22 16:34:58,588 INFO ckpt pointpillar.pth
2020-03-22 16:34:58,589 INFO mgpus False
2020-03-22 16:34:58,589 INFO launcher none
2020-03-22 16:34:58,590 INFO tcp_port 18888
2020-03-22 16:34:58,591 INFO local_rank 0
2020-03-22 16:34:58,592 INFO set_cfgs None
2020-03-22 16:34:58,592 INFO max_waiting_mins 30
2020-03-22 16:34:58,593 INFO start_epoch 0
2020-03-22 16:34:58,593 INFO eval_tag default
2020-03-22 16:34:58,594 INFO eval_all False
2020-03-22 16:34:58,595 INFO ckpt_dir None
2020-03-22 16:34:58,596 INFO save_to_file False
2020-03-22 16:34:58,596 INFO cfg.ROOT_DIR: /media/buaa/My Passport/PCDet
2020-03-22 16:34:58,597 INFO cfg.LOCAL_RANK: 0
2020-03-22 16:34:58,598 INFO cfg.CLASS_NAMES: ['Car', 'Pedestrian', 'Cyclist']
2020-03-22 16:34:58,599 INFO
cfg.DATA_CONFIG = edict()
2020-03-22 16:34:58,600 INFO cfg.DATA_CONFIG.DATASET: KittiDataset
2020-03-22 16:34:58,601 INFO cfg.DATA_CONFIG.DATA_DIR: data/kitti
2020-03-22 16:34:58,602 INFO cfg.DATA_CONFIG.FOV_POINTS_ONLY: True
2020-03-22 16:34:58,602 INFO
cfg.DATA_CONFIG.NUM_POINT_FEATURES = edict()
2020-03-22 16:34:58,603 INFO cfg.DATA_CONFIG.NUM_POINT_FEATURES.total: 4
2020-03-22 16:34:58,604 INFO cfg.DATA_CONFIG.NUM_POINT_FEATURES.use: 4
2020-03-22 16:34:58,605 INFO cfg.DATA_CONFIG.POINT_CLOUD_RANGE: [0, -39.68, -3, 69.12, 39.68, 1]
2020-03-22 16:34:58,605 INFO cfg.DATA_CONFIG.MASK_POINTS_BY_RANGE: True
2020-03-22 16:34:58,606 INFO
cfg.DATA_CONFIG.TRAIN = edict()
2020-03-22 16:34:58,606 INFO cfg.DATA_CONFIG.TRAIN.INFO_PATH: ['data/kitti/kitti_infos_train.pkl']
2020-03-22 16:34:58,607 INFO cfg.DATA_CONFIG.TRAIN.SHUFFLE_POINTS: True
2020-03-22 16:34:58,608 INFO cfg.DATA_CONFIG.TRAIN.MAX_NUMBER_OF_VOXELS: 16000
2020-03-22 16:34:58,608 INFO
cfg.DATA_CONFIG.TEST = edict()
2020-03-22 16:34:58,609 INFO cfg.DATA_CONFIG.TEST.INFO_PATH: ['data/kitti/kitti_infos_val.pkl']
2020-03-22 16:34:58,610 INFO cfg.DATA_CONFIG.TEST.SHUFFLE_POINTS: False
2020-03-22 16:34:58,610 INFO cfg.DATA_CONFIG.TEST.MAX_NUMBER_OF_VOXELS: 40000
2020-03-22 16:34:58,611 INFO
cfg.DATA_CONFIG.AUGMENTATION = edict()
2020-03-22 16:34:58,612 INFO
cfg.DATA_CONFIG.AUGMENTATION.NOISE_PER_OBJECT = edict()
2020-03-22 16:34:58,613 INFO cfg.DATA_CONFIG.AUGMENTATION.NOISE_PER_OBJECT.ENABLED: True
2020-03-22 16:34:58,613 INFO cfg.DATA_CONFIG.AUGMENTATION.NOISE_PER_OBJECT.GT_LOC_NOISE_STD: [1.0, 1.0, 0.1]
2020-03-22 16:34:58,614 INFO cfg.DATA_CONFIG.AUGMENTATION.NOISE_PER_OBJECT.GT_ROT_UNIFORM_NOISE: [-0.78539816, 0.78539816]
2020-03-22 16:34:58,615 INFO
cfg.DATA_CONFIG.AUGMENTATION.NOISE_GLOBAL_SCENE = edict()
2020-03-22 16:34:58,615 INFO cfg.DATA_CONFIG.AUGMENTATION.NOISE_GLOBAL_SCENE.ENABLED: True
2020-03-22 16:34:58,616 INFO cfg.DATA_CONFIG.AUGMENTATION.NOISE_GLOBAL_SCENE.GLOBAL_ROT_UNIFORM_NOISE: [-0.78539816, 0.78539816]
2020-03-22 16:34:58,616 INFO cfg.DATA_CONFIG.AUGMENTATION.NOISE_GLOBAL_SCENE.GLOBAL_SCALING_UNIFORM_NOISE: [0.95, 1.05]
2020-03-22 16:34:58,617 INFO
cfg.DATA_CONFIG.AUGMENTATION.DB_SAMPLER = edict()
2020-03-22 16:34:58,618 INFO cfg.DATA_CONFIG.AUGMENTATION.DB_SAMPLER.ENABLED: True
2020-03-22 16:34:58,619 INFO cfg.DATA_CONFIG.AUGMENTATION.DB_SAMPLER.DB_INFO_PATH: ['data/kitti/kitti_dbinfos_train.pkl']
2020-03-22 16:34:58,620 INFO
cfg.DATA_CONFIG.AUGMENTATION.DB_SAMPLER.PREPARE = edict()
2020-03-22 16:34:58,621 INFO cfg.DATA_CONFIG.AUGMENTATION.DB_SAMPLER.PREPARE.filter_by_difficulty: [-1]
2020-03-22 16:34:58,621 INFO cfg.DATA_CONFIG.AUGMENTATION.DB_SAMPLER.PREPARE.filter_by_min_points: ['Car:5', 'Pedestrian:5', 'Cyclist:5']
2020-03-22 16:34:58,622 INFO cfg.DATA_CONFIG.AUGMENTATION.DB_SAMPLER.RATE: 1.0
2020-03-22 16:34:58,623 INFO cfg.DATA_CONFIG.AUGMENTATION.DB_SAMPLER.SAMPLE_GROUPS: ['Car:15', 'Pedestrian:10', 'Cyclist:10']
2020-03-22 16:34:58,624 INFO cfg.DATA_CONFIG.AUGMENTATION.DB_SAMPLER.USE_ROAD_PLANE: True
2020-03-22 16:34:58,624 INFO
cfg.DATA_CONFIG.VOXEL_GENERATOR = edict()
2020-03-22 16:34:58,625 INFO cfg.DATA_CONFIG.VOXEL_GENERATOR.MAX_POINTS_PER_VOXEL: 32
2020-03-22 16:34:58,625 INFO cfg.DATA_CONFIG.VOXEL_GENERATOR.VOXEL_SIZE: [0.16, 0.16, 4]
2020-03-22 16:34:58,626 INFO
cfg.MODEL = edict()
2020-03-22 16:34:58,627 INFO cfg.MODEL.NAME: PointPillar
2020-03-22 16:34:58,627 INFO
cfg.MODEL.VFE = edict()
2020-03-22 16:34:58,628 INFO cfg.MODEL.VFE.NAME: PillarFeatureNetOld2
2020-03-22 16:34:58,628 INFO
cfg.MODEL.VFE.ARGS = edict()
2020-03-22 16:34:58,629 INFO cfg.MODEL.VFE.ARGS.use_norm: True
2020-03-22 16:34:58,630 INFO cfg.MODEL.VFE.ARGS.num_filters: [64]
2020-03-22 16:34:58,630 INFO cfg.MODEL.VFE.ARGS.with_distance: False
2020-03-22 16:34:58,631 INFO
cfg.MODEL.RPN = edict()
2020-03-22 16:34:58,631 INFO cfg.MODEL.RPN.PARAMS_FIXED: False
2020-03-22 16:34:58,632 INFO
cfg.MODEL.RPN.BACKBONE = edict()
2020-03-22 16:34:58,632 INFO cfg.MODEL.RPN.BACKBONE.NAME: PointPillarsScatter
2020-03-22 16:34:58,633 INFO
cfg.MODEL.RPN.BACKBONE.ARGS = edict()
2020-03-22 16:34:58,633 INFO
cfg.MODEL.RPN.RPN_HEAD = edict()
2020-03-22 16:34:58,634 INFO cfg.MODEL.RPN.RPN_HEAD.NAME: RPNV2
2020-03-22 16:34:58,634 INFO cfg.MODEL.RPN.RPN_HEAD.DOWNSAMPLE_FACTOR: 8
2020-03-22 16:34:58,635 INFO
cfg.MODEL.RPN.RPN_HEAD.ARGS = edict()
2020-03-22 16:34:58,635 INFO cfg.MODEL.RPN.RPN_HEAD.ARGS.use_norm: True
2020-03-22 16:34:58,636 INFO cfg.MODEL.RPN.RPN_HEAD.ARGS.concat_input: False
2020-03-22 16:34:58,636 INFO cfg.MODEL.RPN.RPN_HEAD.ARGS.num_input_features: 64
2020-03-22 16:34:58,637 INFO cfg.MODEL.RPN.RPN_HEAD.ARGS.layer_nums: [3, 5, 5]
2020-03-22 16:34:58,637 INFO cfg.MODEL.RPN.RPN_HEAD.ARGS.layer_strides: [2, 2, 2]
2020-03-22 16:34:58,638 INFO cfg.MODEL.RPN.RPN_HEAD.ARGS.num_filters: [64, 128, 256]
2020-03-22 16:34:58,638 INFO cfg.MODEL.RPN.RPN_HEAD.ARGS.upsample_strides: [1, 2, 4]
2020-03-22 16:34:58,639 INFO cfg.MODEL.RPN.RPN_HEAD.ARGS.num_upsample_filters: [128, 128, 128]
2020-03-22 16:34:58,639 INFO cfg.MODEL.RPN.RPN_HEAD.ARGS.encode_background_as_zeros: True
2020-03-22 16:34:58,640 INFO cfg.MODEL.RPN.RPN_HEAD.ARGS.use_direction_classifier: True
2020-03-22 16:34:58,640 INFO cfg.MODEL.RPN.RPN_HEAD.ARGS.num_direction_bins: 2
2020-03-22 16:34:58,641 INFO cfg.MODEL.RPN.RPN_HEAD.ARGS.dir_offset: 0.78539
2020-03-22 16:34:58,641 INFO cfg.MODEL.RPN.RPN_HEAD.ARGS.dir_limit_offset: 0.0
2020-03-22 16:34:58,642 INFO cfg.MODEL.RPN.RPN_HEAD.ARGS.use_binary_dir_classifier: False
2020-03-22 16:34:58,643 INFO
cfg.MODEL.RPN.RPN_HEAD.TARGET_CONFIG = edict()
2020-03-22 16:34:58,643 INFO cfg.MODEL.RPN.RPN_HEAD.TARGET_CONFIG.DOWNSAMPLED_FACTOR: 2
2020-03-22 16:34:58,644 INFO cfg.MODEL.RPN.RPN_HEAD.TARGET_CONFIG.BOX_CODER: ResidualCoder
2020-03-22 16:34:58,644 INFO cfg.MODEL.RPN.RPN_HEAD.TARGET_CONFIG.REGION_SIMILARITY_FN: nearest_iou_similarity
2020-03-22 16:34:58,645 INFO cfg.MODEL.RPN.RPN_HEAD.TARGET_CONFIG.SAMPLE_POS_FRACTION: -1.0
2020-03-22 16:34:58,645 INFO cfg.MODEL.RPN.RPN_HEAD.TARGET_CONFIG.SAMPLE_SIZE: 512
2020-03-22 16:34:58,646 INFO cfg.MODEL.RPN.RPN_HEAD.TARGET_CONFIG.ANCHOR_GENERATOR: [{'anchor_range': [0, -40.0, -1.78, 70.4, 40.0, -1.78], 'sizes': [[1.6, 3.9, 1.56]], 'rotations': [0, 1.57], 'matched_threshold': 0.6, 'unmatched_threshold': 0.45, 'class_name': 'Car'}, {'anchor_range': [0, -40, -0.6, 70.4, 40, -0.6], 'sizes': [[0.6, 0.8, 1.73]], 'rotations': [0, 1.57], 'matched_threshold': 0.5, 'unmatched_threshold': 0.35, 'class_name': 'Pedestrian'}, {'anchor_range': [0, -40, -0.6, 70.4, 40, -0.6], 'sizes': [[0.6, 1.76, 1.73]], 'rotations': [0, 1.57], 'matched_threshold': 0.5, 'unmatched_threshold': 0.35, 'class_name': 'Cyclist'}]
2020-03-22 16:34:58,646 INFO
cfg.MODEL.RCNN = edict()
2020-03-22 16:34:58,647 INFO cfg.MODEL.RCNN.ENABLED: False
2020-03-22 16:34:58,648 INFO
cfg.MODEL.LOSSES = edict()
2020-03-22 16:34:58,648 INFO cfg.MODEL.LOSSES.RPN_REG_LOSS: smooth-l1
2020-03-22 16:34:58,649 INFO
cfg.MODEL.LOSSES.LOSS_WEIGHTS = edict()
2020-03-22 16:34:58,650 INFO cfg.MODEL.LOSSES.LOSS_WEIGHTS.rpn_cls_weight: 1.0
2020-03-22 16:34:58,651 INFO cfg.MODEL.LOSSES.LOSS_WEIGHTS.rpn_loc_weight: 2.0
2020-03-22 16:34:58,652 INFO cfg.MODEL.LOSSES.LOSS_WEIGHTS.rpn_dir_weight: 0.2
2020-03-22 16:34:58,652 INFO cfg.MODEL.LOSSES.LOSS_WEIGHTS.code_weights: [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0]
2020-03-22 16:34:58,653 INFO
cfg.MODEL.TRAIN = edict()
2020-03-22 16:34:58,653 INFO cfg.MODEL.TRAIN.SPLIT: train
2020-03-22 16:34:58,654 INFO
cfg.MODEL.TRAIN.OPTIMIZATION = edict()
2020-03-22 16:34:58,655 INFO cfg.MODEL.TRAIN.OPTIMIZATION.OPTIMIZER: adam_onecycle
2020-03-22 16:34:58,655 INFO cfg.MODEL.TRAIN.OPTIMIZATION.LR: 0.003
2020-03-22 16:34:58,656 INFO cfg.MODEL.TRAIN.OPTIMIZATION.WEIGHT_DECAY: 0.01
2020-03-22 16:34:58,656 INFO cfg.MODEL.TRAIN.OPTIMIZATION.MOMENTUM: 0.9
2020-03-22 16:34:58,657 INFO cfg.MODEL.TRAIN.OPTIMIZATION.MOMS: [0.95, 0.85]
2020-03-22 16:34:58,658 INFO cfg.MODEL.TRAIN.OPTIMIZATION.PCT_START: 0.4
2020-03-22 16:34:58,659 INFO cfg.MODEL.TRAIN.OPTIMIZATION.DIV_FACTOR: 10
2020-03-22 16:34:58,659 INFO cfg.MODEL.TRAIN.OPTIMIZATION.DECAY_STEP_LIST: [35, 45]
2020-03-22 16:34:58,660 INFO cfg.MODEL.TRAIN.OPTIMIZATION.LR_DECAY: 0.1
2020-03-22 16:34:58,661 INFO cfg.MODEL.TRAIN.OPTIMIZATION.LR_CLIP: 1e-07
2020-03-22 16:34:58,661 INFO cfg.MODEL.TRAIN.OPTIMIZATION.LR_WARMUP: False
2020-03-22 16:34:58,662 INFO cfg.MODEL.TRAIN.OPTIMIZATION.WARMUP_EPOCH: 1
2020-03-22 16:34:58,662 INFO cfg.MODEL.TRAIN.OPTIMIZATION.GRAD_NORM_CLIP: 10
2020-03-22 16:34:58,663 INFO
cfg.MODEL.TEST = edict()
2020-03-22 16:34:58,664 INFO cfg.MODEL.TEST.SPLIT: val
2020-03-22 16:34:58,664 INFO cfg.MODEL.TEST.NMS_TYPE: nms_gpu
2020-03-22 16:34:58,665 INFO cfg.MODEL.TEST.MULTI_CLASSES_NMS: False
2020-03-22 16:34:58,665 INFO cfg.MODEL.TEST.NMS_THRESH: 0.01
2020-03-22 16:34:58,666 INFO cfg.MODEL.TEST.SCORE_THRESH: 0.1
2020-03-22 16:34:58,666 INFO cfg.MODEL.TEST.USE_RAW_SCORE: True
2020-03-22 16:34:58,667 INFO cfg.MODEL.TEST.NMS_PRE_MAXSIZE_LAST: 4096
2020-03-22 16:34:58,668 INFO cfg.MODEL.TEST.NMS_POST_MAXSIZE_LAST: 500
2020-03-22 16:34:58,668 INFO cfg.MODEL.TEST.RECALL_THRESH_LIST: [0.5, 0.7]
2020-03-22 16:34:58,669 INFO cfg.MODEL.TEST.EVAL_METRIC: kitti
2020-03-22 16:34:58,669 INFO
cfg.MODEL.TEST.BOX_FILTER = edict()
2020-03-22 16:34:58,670 INFO cfg.MODEL.TEST.BOX_FILTER.USE_IMAGE_AREA_FILTER: True
2020-03-22 16:34:58,670 INFO cfg.MODEL.TEST.BOX_FILTER.LIMIT_RANGE: [0, -40, -3.0, 70.4, 40, 3.0]
2020-03-22 16:34:58,671 INFO cfg.TAG: pointpillar
2020-03-22 16:34:58,688 INFO Loading KITTI dataset
2020-03-22 16:34:59,300 INFO Total samples for KITTI dataset: 3769
2020-03-22 16:35:14,877 INFO ==> Loading parameters from checkpoint pointpillar.pth to GPU
2020-03-22 16:35:15,316 INFO ==> Done (loaded 127/127)
2020-03-22 16:35:15,407 INFO *************** EPOCH no_number EVALUATION *****************
eval: 0%| | 0/943 [00:00<?, ?it/s]/media/nvidia/WD_BLUE_2.5_1TB/pytorch/20191015/pytorch-v1.3.0/aten/src/ATen/native/cuda/IndexKernel.cu:60: lambda ->auto::operator()(int)->auto: block: [213,0,0], thread: [64,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/media/nvidia/WD_BLUE_2.5_1TB/pytorch/20191015/pytorch-v1.3.0/aten/src/ATen/native/cuda/IndexKernel.cu:60: lambda ->auto::operator()(int)->auto: block: [213,0,0], thread: [65,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/media/nvidia/WD_BLUE_2.5_1TB/pytorch/20191015/pytorch-v1.3.0/aten/src/ATen/native/cuda/IndexKernel.cu:60: lambda ->auto::operator()(int)->auto: block: [213,0,0], thread: [66,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/media/nvidia/WD_BLUE_2.5_1TB/pytorch/20191015/pytorch-v1.3.0/aten/src/ATen/native/cuda/IndexKernel.cu:60: lambda ->auto::operator()(int)->auto: block: [213,0,0], thread: [67,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/media/nvidia/WD_BLUE_2.5_1TB/pytorch/20191015/pytorch-v1.3.0/aten/src/ATen/native/cuda/IndexKernel.cu:60: lambda ->auto::operator()(int)->auto: block: [213,0,0], thread: [68,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/media/nvidia/WD_BLUE_2.5_1TB/pytorch/20191015/pytorch-v1.3.0/aten/src/ATen/native/cuda/IndexKernel.cu:60: lambda ->auto::operator()(int)->auto: block: [213,0,0], thread: [69,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/media/nvidia/WD_BLUE_2.5_1TB/pytorch/20191015/pytorch-v1.3.0/aten/src/ATen/native/cuda/IndexKernel.cu:60: lambda ->auto::operator()(int)->auto: block: [213,0,0], thread: [70,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
...
...

Traceback (most recent call last):
File "test.py", line 181, in
main()
File "test.py", line 177, in main
eval_single_ckpt(model, test_loader, args, eval_output_dir, logger, epoch_id)
File "test.py", line 59, in eval_single_ckpt
model, test_loader, epoch_id, logger, result_dir=eval_output_dir, save_to_file=args.save_to_file
File "/media/buaa/My Passport/PCDet/tools/eval_utils/eval_utils.py", line 46, in eval_one_epoch
pred_dicts, ret_dict = model(input_dict)
File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 541, in call
result = self.forward(*input, **kwargs)
File "/media/buaa/My Passport/PCDet/pcdet/models/detectors/pointpillar.py", line 34, in forward
rpn_ret_dict = self.forward_rpn(**input_dict)
File "/media/buaa/My Passport/PCDet/pcdet/models/detectors/pointpillar.py", line 18, in forward_rpn
output_shape=self.grid_size[::-1]
File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 541, in call
result = self.forward(*input, **kwargs)
File "/media/buaa/My Passport/PCDet/pcdet/models/rpn/pillar_scatter.py", line 32, in forward
this_coords = coords[batch_mask, :]
RuntimeError: copy_if failed to synchronize: device-side assert triggered
eval: 0%|

when i run
python3 test.py --cfg_file cfgs/pointpillar.yaml --batch_size 4 --ckpt pointpillar.pth

I met the problem above ,How can i solved the problem?
THK

Hi, @sshaoshuai ,

Thanks for releasing such useful package. Could you also release the pre-trained model PartA2.pth?

THX!

Thanks for releasing such useful package.
I have some question when I tried to inference the KITTI Data

The first question:

I run
python ./tools/test.py --cfg_file ./cfgs/PartA2_car.yaml --batch_size 2 --ckpt ./weight/PartA2.pth --set MODEL.RPN.BACKBONE.NAME UNetV0 MODEL.RPN.RPN_HEAD.ARGS use_binary_dir_classifier:True

I can get this result

But when I run ./tools/test.py --cfg_file ./cfgs/PartA2.yaml --batch_size 2 --ckpt ./weight/PartA2.pth --set MODEL.RPN.BACKBONE.NAME UNetV0 MODEL.RPN.RPN_HEAD.ARGS use_binary_dir_classifier:True

It seems like have some problem on loading weight?
And How can I use the PartA2.yaml to inference KITTI data?

The second question:
In PartA2.yaml the model name is UNetV2, but I when I change the command
--set MODEL.RPN.BACKBONE.NAME UNetV2

Also get the problem about weight.
I don't know how to use UNetV2.

Can you provide some suggestion? Thank you!

@sshaoshuai 我们的坐标系跟kitti坐标系不一样（需要做转换），关于代码中旋转角度的细节，有些困惑，烦请赐教~

pcdet/models/rcnn/partA2_rcnn_net.py

when I run the train command or the test command I get this error

File "/home/mohamed/Mohamed/HTW/lib/python3.6/site-packages/spconv/utils/init.py", line 65, in points_to_voxel
assert block_filtering is False
AssertionError

and when I change the flag of the assert in the code in init.py to see what will happen I get this error

File "/home/mohamed/Mohamed/HTW/lib/python3.6/site-packages/spconv/utils/init.py", line 94, in points_to_voxel
coors_range.tolist(), max_points, max_voxels)
ValueError: array has incorrect number of dimensions: 2; expected 3

please help, I dont know what is wrong

File "/home/PCDet/pcdet/datasets/kitti/kitti_dataset.py", line 398, in getitem
sample_idx = info['point_cloud']['lidar_idx']
KeyError: 'lidar_idx'

I have been getting the following error when I try to train. It only happens when I try to limit classes I train on.

Traceback (most recent call last):
File "train.py", line 155, in
main()
File "train.py", line 148, in main
max_ckpt_save_num=args.max_ckpt_save_num
File "/s/dat/UserFolders/ctan24/PCDet-master/tools/train_utils/train_utils.py", line 81, in train_model
leave_pbar=(cur_epoch + 1 == total_epochs)
File "/s/dat/UserFolders/ctan24/PCDet-master/tools/train_utils/train_utils.py", line 36, in train_one_epoch
loss, tb_dict, disp_dict = model_func(model, batch)
File "/s/dat/UserFolders/ctan24/PCDet-master/pcdet/models/init.py", line 25, in model_func
ret_dict, tb_dict, disp_dict = model(input_dict)
File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 493, in call
result = self.forward(*input, **kwargs)
File "/s/dat/UserFolders/ctan24/PCDet-master/pcdet/models/detectors/PartA2_net.py", line 112, in forward
batch_size, voxel_centers, coords, rpn_ret_dict, input_dict
File "/s/dat/UserFolders/ctan24/PCDet-master/pcdet/models/detectors/PartA2_net.py", line 98, in forward_rcnn
rcnn_ret_dict = self.rcnn_net.forward(rcnn_input_dict)
File "/s/dat/UserFolders/ctan24/PCDet-master/pcdet/models/rcnn/partA2_rcnn_net.py", line 323, in forward
targets_dict = self.assign_targets(batch_size, rcnn_dict)
File "/s/dat/UserFolders/ctan24/PCDet-master/pcdet/models/rcnn/partA2_rcnn_net.py", line 27, in assign_targets
targets_dict = proposal_target_layer(rcnn_dict, roi_sampler_cfg=self.rcnn_target_config)
File "/s/dat/UserFolders/ctan24/PCDet-master/pcdet/models/model_utils/proposal_target_layer.py", line 14, in proposal_target_layer
sample_rois_for_rcnn(rois, gt_boxes, roi_raw_scores, roi_labels, roi_sampler_cfg)
File "/s/dat/UserFolders/ctan24/PCDet-master/pcdet/models/model_utils/proposal_target_layer.py", line 77, in sample_rois_for_rcnn
iou3d = iou3d_nms_utils.boxes_iou3d_gpu(cur_roi, cur_gt[:, 0:7]) # (M, N)
File "/s/dat/UserFolders/ctan24/PCDet-master/pcdet/ops/iou3d_nms/iou3d_nms_utils.py", line 52, in boxes_iou3d_gpu
vol_a = (boxes_a[:, 3] * boxes_a[:, 4] * boxes_a[:, 5]).view(-1, 1)
RuntimeError: CUDA error: invalid configuration argument

@sshaoshuai Thank you for your valuable advice in #24 , i did found some bugs and fixed them !
And i modified the PartA2.yaml configs to fit my custum dataset.

But, the recall_rcnn_* and recall_roi_* still very low when i eval on test sub-dataset. Could you help to provide some more suggestions ? Looking forward to your reply .

In addition, our pointclouds is labeld all 360 degrees, without 2d images. So, i set POINT_CLOUD_RANGE to [-60, -60, -1.5, 60, 60, 2.5].
Because without 2d bboxes, i'm modifying kitti_object_eval_python/eval.py, so can't provide the mAPs now .

The tensorboard graphs :

detailed experimental control:

Hi @sshaoshuai,

I'm trying to install pcdet, but when I run python setup.py develop I get this error:

  File "setup.py", line 26
    print('__version__ = "%s"' % version, file=f)
                                              ^
SyntaxError: invalid syntax

How can I solve it?

Thank you for your attention.

I am getting the following runtime error:

RuntimeError: result type Byte can't be cast to the desired output type Bool

It is caused by line 130 in box_coder_utils.py.

Thank you for your great contributions to this repo. Instead of the test.py scripts to evaluate the performance, could you please also provide some simple testing scripts to do the inference for a single image/ a set of images in the folder, and also provide some scripts/tools for visualization purposes? Looking forward to your reply!

Thank you!

I'm interested in trying anchor-free RPN head which is mentioned in your paper.
Is it supported in this repo?
How should I run?

Thanks for this nice code base.

Can you provided instructions on how to reproduce the reported performance on the val set?

I tried:

python test.py --cfg_file ${CONFIG_FILE} --batch_size 4 --ckpt ${CKPT}

with the pretrained model and I get:

Car_3d_easy = 54.303261947978974
Car_3d_moderate = 38.349761507380684
Car_3d_hard = 36.39773833709999

with a model trained with the default config:

Car_3d_easy = 76.20793977206878                                                
Car_3d_moderate = 60.98983744758463                                                          
Car_3d_hard = 57.58892404665682 

which is far from the reported values. Am I doing something wrong? Is the provided config the one with which you report the performance?

Thanks

Hi,
I am trying to use the pre-trained PartA^2_fc model to run it on the NuScenes dataset. I have successfully generated the required annotation files, such as infos_train.pkl, infos_test.pkl and so on.

However, when I try to run the code by using the PartA2_fc.yaml, I get the following error:

File ".../lib/python3.7/site-packages/torch/nn/modules/module.py", line 541, in __call__
    result = self.forward(*input, **kwargs)
  File "PCDet/pcdet/models/detectors/PartA2_net.py", line 106, in forward
    rpn_ret_dict = self.forward_rpn(**input_dict)
  File "PCDet/pcdet/models/detectors/PartA2_net.py", line 33, in forward_rpn
    **kwargs
  File ".../lib/python3.7/site-packages/torch/nn/modules/module.py", line 541, in __call__
    result = self.forward(*input, **kwargs)
  File "PCDet/pcdet/models/rpn/rpn_unet.py", line 471, in forward
    x = self.conv_input(input_sp_tensor)
  File ".../lib/python3.7/site-packages/torch/nn/modules/module.py", line 541, in __call__
    result = self.forward(*input, **kwargs)
  File ".../lib/python3.7/site-packages/spconv/modules.py", line 123, in forward
    input = module(input)
  File ".../lib/python3.7/site-packages/torch/nn/modules/module.py", line 541, in __call__
    result = self.forward(*input, **kwargs)
  File ".../lib/python3.7/site-packages/spconv/conv.py", line 151, in forward
    self.stride, self.padding, self.dilation, self.output_padding, self.subm, self.transposed, grid=input.grid)
  File ".../lib/python3.7/site-packages/spconv/ops.py", line 89, in get_indice_pairs
    stride, padding, dilation, out_padding, int(subm), int(transpose))
RuntimeError: PCDet/spconv/include/spconv/spconv_ops.h 40
kernelSize.size() == coorDim assert faild. error

If I am guessing correctly, this is because after voxelization of the point cloud, the NN model expects a particular dimension of voxel grid. I can probably correct this by changing the VOXEL_SIZE in PartA2_fc.yaml. Is this true? How should I calculate how to set this parameter?

Thanks for your help!

Thanks for your wonderful work~
But I am puzzled on the training strategy, are the two stages of Part A^2 Net trained separately like PointRCNN?
I have never submitted results on kitti website, but I am trying to do a research job based on this dataset currently, so I wonder is it possible to train easy cases and hard cases respectively?

Hello,

thank you for the great work. I think it is great that you are integrating many different models and networks into this framework. I am working on an automated build for your project since I had some issues with the installation. Maybe you want to integrate the Dockerfile at:

https://github.com/trn84/pcdet-docker

Into this projects. It is basically taken and modified from the second.pytorch repo. I have tested the evaluation and some training but it is still in prelim stage.

BR

I've noticed that your group's new work about PV-RCNN.
could you update some code about this amazing work?
thanks

Hello, I have a couple of doubts regarding how I can train the model on my own data.

1 - Do I need to use specifically one of the Velodyne lidar sensors to gather data?

2 - Does it automatically produces a calibration.txt file of every frame when saved? Essentially what is calibration for?

3 - How can I label my point cloud data?
Would this software work?
https://scale.com/3d-sensor-fusion

It would be great if you could help me clarifying these questions.

Thank you.

I am trying to run the pretrained model on the val split of the Nuscenes dataset. I have converted it to the KITTI format using their script.
Some frames in the Nuscenes val set contain 0 objects, and hence have no ground truth annotations. I wanted to know what the expected behavior should be in this case. When I run

python kitti_dataset.py create_kitti_infos

I run into the following error when there are no objects in the ground truth file.

File "kitti_dataset.py", line 510, in <module>
    save_path=cfg.ROOT_DIR / 'data' / 'kitti'
  File "kitti_dataset.py", line 484, in create_kitti_infos
    kitti_infos_val = dataset.get_infos(num_workers=workers, has_label=True, count_inside_pts=True)
  File "kitti_dataset.py", line 164, in get_infos
    info = process_single_scene(sampl_id)
  File "kitti_dataset.py", line 113, in process_single_scene
    annotations['bbox'] = np.concatenate([obj.box2d.reshape(1, 4) for obj in obj_list], axis=0)
  File "<__array_function__ internals>", line 6, in concatenate
ValueError: need at least one array to concatenate

Could you let me know what should be done in this case when there are no objects in the ground truth?

Hello,shishaoshuai!
Thanks for sharing the excellent code. I have q question about the point sampleing process. In each config model.yaml files, you set MAX_POINTS_PER_VOXEL = 5 or 32. But if in a voxel whose points more than 5/32 points or less than 5/32 points, how do you selcet these points, and where can I change the sampling policy in my own point cloud. Thanks a lot!

My gpu is nvidia-1080 Ti with 12 GB and the max lidar frames per gpu is 2 not 4.

Traceback (most recent call last):
File "test.py", line 188, in
main()
File "test.py", line 184, in main
eval_single_ckpt(model, test_loader, args, eval_output_dir, logger, epoch_id)
File "test.py", line 63, in eval_single_ckpt
model, test_loader, epoch_id, logger, result_dir=eval_output_dir, save_to_file=args.save_to_file
at kitt_dataset.py-dataset_init-index=16
File "/share/lycode/PCDet/tools/eval_utils/eval_utils.py", line 50, in eval_one_epoch
at kitt_dataset.py-dataset_init-index=24
for i, data in enumerate(dataloader):
File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 582, in next
return self._process_next_batch(batch)
File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 608, in _process_next_batch
raise batch.exc_type(batch.exc_msg)
IndexError: Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/_utils/worker.py", line 99, in _worker_loop
samples = collate_fn([dataset[i] for i in batch_indices])
File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/_utils/worker.py", line 99, in
samples = collate_fn([dataset[i] for i in batch_indices])
File "/share/lycode/PCDet/pcdet/datasets/kitti/kitti_dataset.py", line 451, in getitem
annos = common_utils.drop_info_with_name(annos, name='DontCare')
File "/share/lycode/PCDet/pcdet/utils/common_utils.py", line 76, in drop_info_with_name
ret_info[key] = info[key][keep_indices]
IndexError: index 3 is out of bounds for axis 0 with size 3

eval: 0%| | 0/2002 [00:00<?, ?it/s]

Could I ask about this error when i run CUDA_VISIBLE_DEVICES=2 python test.py --cfg_file cfgs/PartA2_car.yaml --batch_size 4 --ckpt cfgs/PartA2_car.pth --set MODEL.RPN.BACKBONE.NAME UNetV0 MODEL.RPN.RPN_HEAD.ARGS use_binary_dir_classifier:True?I don't know how to solve it ,Thanks for help!

Hello, sshaoshuai!
Thansk for sharing your great work! I have a problem, now I am working on a point cloud without intensity or concat a fake intensity. How to change pointpillars/second.yaml files to choose not to use intensity featues in the model training?

Thx a lot!!

Thanks for your useful project, how can I visualize the pointcloud and 3d box results such as the image you provided?

@sshaoshuai hello thanks for open sourcing the code i have few queries
Q1 in the paper you had mentioned about the inference time as 14fps on what sys
Q2 can you obtain the intermediate semantic segmentation output of point clouds ? i show which part of the code does it
Q3 Can we train your architecture onreduced point clouds will the performance varry
Q4 Can we train of static / immovable objects

Thanks in advance

when is run second , it shows above error at pcdet/models/detectors/detector3d.py/line 303

for key, val in model_state_disk.items():
        if key in self.state_dict() and self.state_dict()[key].shape == model_state_disk[key].shape:
        update_model_state[key] = val
        # logger.info('Update weight %s: %s' % (key, str(val.shape)))

do you know how to solve it?

Hi, thank you for the amazing work. I tried installing spconv1., but their cuda libs search locations and their build process kept breaking on my lab's servers. I installed spconv1.1 instead, and it just worked. However, I noticed in several places this repo stresses spconv1.0. I am wondering if there is anything that absolutely needs spconv1.0.

The 2 main places this repo uses spconv is the voxel_generator and the convolutions.

When cfg.DATA_CONFIG.FOV_POINTS_ONLY is false, local variable 'img_shape' will be referenced before assignment in line 437 of kitti_dataset.py.

Hi, @sshaoshuai ,

I got the following error when I perform evaluation by the following command:
python test.py --cfg_file cfgs/PartA2_car.yaml --batch_size 4 --ckpt PartA2_car.pth --set MODEL.RPN.BACKBONE.NAME UNetV0 MODEL.RPN.RPN_HEAD.ARGS use_binary_dir_classifier:True

2020-04-17 09:51:06,796   INFO  ==> Done (loaded 357/357)
2020-04-17 09:51:06,847   INFO  *************** EPOCH 2 EVALUATION *****************
eval:   0%|                                                     | 0/943 [00:00<?, ?it/s]Traceback (most recent call last):
  File "test.py", line 181, in <module>
    main()
  File "test.py", line 177, in main
    eval_single_ckpt(model, test_loader, args, eval_output_dir, logger, epoch_id)
  File "test.py", line 59, in eval_single_ckpt
    model, test_loader, epoch_id, logger, result_dir=eval_output_dir, save_to_file=args.save_to_file
  File "/data/code10/PCDet/tools/eval_utils/eval_utils.py", line 46, in eval_one_epoch
    pred_dicts, ret_dict = model(input_dict)
  File "/root/anaconda3/envs/pytorch1.3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 541, in __call__
    result = self.forward(*input, **kwargs)
  File "/data/code10/PCDet/pcdet/models/detectors/PartA2_net.py", line 112, in forward
    batch_size, voxel_centers, coords, rpn_ret_dict, input_dict
  File "/data/code10/PCDet/pcdet/models/detectors/PartA2_net.py", line 77, in forward_rcnn
    use_binary_dir_classifier=cfg.MODEL.RPN.RPN_HEAD.ARGS.get('use_binary_dir_classifier', False)
  File "/data/code10/PCDet/pcdet/utils/box_coder_utils.py", line 129, in decode_with_head_direction_torch
    opp_labels = (batch_box_preds[..., -1] > 0) ^ dir_labels.byte()
RuntimeError: Expected object of scalar type Bool but got scalar type Byte for argument #2 'other' in call to _th_xor
eval:   0%|                                                     | 0/943 [00:08<?, ?it/s]
(pytorch1.3) root@Lab-PC:/data/code10/PCDet/tools

The detailed log is here.

Any hints to solve this issue?
THX!

To train your model, I use the lidar coordinate system with input of each bounding box as
[x, y, z, w, l, h, Rz] with pointpillar configuration. The anchor generator of each object in the config is [0, 3.14]
However, the prediction result of Rz is around 6. which is outside the range of [-pi, pi].
Do you have any idea about this problem?
Thank you for your excellent work.

Thanks for sharing the code. I want to implement a one-stage bounding box predictor using the bottleneck layer. But I don't know how to get the corresponding regression gt of the bottleneck layer, x_conv4 layer for example. Can you provide some suggestions?