Problem with Checkpoint about vectormapnet_code HOT 8 OPEN

As you can see, the result is quite poor with the uploaded checkpoint on the nuscenes mini. I am wondering why this is the case?

from vectormapnet_code.

As you can see, the result is quite poor with the uploaded checkpoint on the nuscenes mini. I am wondering why this is the case?

work_dir: ./work_dirs/vectormapnet collecting samples... collected 81 samples in 0.00s 2023-07-05 14:38:13,830 - mmcv - INFO - load model from: open-mmlab://detectron2/resnet50_caffe 2023-07-05 14:38:13,830 - mmcv - INFO - Use load_from_openmmlab loader 2023-07-05 14:38:13,878 - mmcv - WARNING - The model and loaded state dict do not match exactly

unexpected key in source state_dict: conv1.bias

missing keys in source state_dict: layer3.0.conv2.conv_offset.weight, layer3.0.conv2.conv_offset.bias, layer3.1.conv2.conv_offset.weight, layer3.1.conv2.conv_offset.bias, layer3.2.conv2.conv_offset.weight, layer3.2.conv2.conv_offset.bias, layer3.3.conv2.conv_offset.weight, layer3.3.conv2.conv_offset.bias, layer3.4.conv2.conv_offset.weight, layer3.4.conv2.conv_offset.bias, layer3.5.conv2.conv_offset.weight, layer3.5.conv2.conv_offset.bias, layer4.0.conv2.conv_offset.weight, layer4.0.conv2.conv_offset.bias, layer4.1.conv2.conv_offset.weight, layer4.1.conv2.conv_offset.bias, layer4.2.conv2.conv_offset.weight, layer4.2.conv2.conv_offset.bias

Use load_from_local loader The model and loaded state dict do not match exactly

unexpected key in source state_dict: conv1x1.weight

[>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>] 81/81, 7.6 task/s, elapsed: 11s, ETA: 0sstart evaluation! len of the results 81 [>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>] 81/81, 11455.6 task/s, elapsed: 0s, ETA: 0s Done! ----------thershold:0.5---------- results path: ./work_dirs/vectormapnet/results_nuscence.pkl metric: chamfer threshold: -0.5 update: True fix_interval: False class_num: ['ped_crossing', 'divider', 'contours'] Formatting ... Data formatting done in 1.658856s!! cls:ped_crossing done in 0.008414s!! cls:divider done in 0.015380s!! cls:contours done in 0.020888s!!

+--------------+-----+------+--------+-------+ | class | gts | dets | recall | ap | +--------------+-----+------+--------+-------+ | ped_crossing | 76 | 656 | 0.605 | 0.435 | | divider | 460 | 1149 | 0.617 | 0.425 | | contours | 282 | 1030 | 0.245 | 0.093 | +--------------+-----+------+--------+-------+ | mAP | | | | 0.318 | +--------------+-----+------+--------+-------+ ----------thershold:1---------- results path: ./work_dirs/vectormapnet/results_nuscence.pkl metric: chamfer threshold: -1 update: False fix_interval: False class_num: ['ped_crossing', 'divider', 'contours'] Formatting ... Data formatting done in 1.180227s!! cls:ped_crossing done in 0.011127s!! cls:divider done in 0.016809s!! cls:contours done in 0.020043s!!

+--------------+-----+------+--------+-------+ | class | gts | dets | recall | ap | +--------------+-----+------+--------+-------+ | ped_crossing | 76 | 656 | 0.934 | 0.887 | | divider | 460 | 1149 | 0.876 | 0.806 | | contours | 282 | 1030 | 0.628 | 0.459 | +--------------+-----+------+--------+-------+ | mAP | | | | 0.717 | +--------------+-----+------+--------+-------+ ----------thershold:1.5---------- results path: ./work_dirs/vectormapnet/results_nuscence.pkl metric: chamfer threshold: -1.5 update: False fix_interval: False class_num: ['ped_crossing', 'divider', 'contours'] Formatting ... Data formatting done in 1.196037s!! cls:ped_crossing done in 0.013920s!! cls:divider done in 0.015027s!! cls:contours done in 0.022493s!!

+--------------+-----+------+--------+-------+ | class | gts | dets | recall | ap | +--------------+-----+------+--------+-------+ | ped_crossing | 76 | 656 | 0.974 | 0.956 | | divider | 460 | 1149 | 0.946 | 0.892 | | contours | 282 | 1030 | 0.794 | 0.669 | +--------------+-----+------+--------+-------+ | mAP | | | | 0.839 | +--------------+-----+------+--------+-------+ ped_crossing: 0.7592802941799164 divider: 0.7075984378655752 contours: 0.40709205220143 map: 0.6246569280823072 VectormapNet Evaluation Results: {'mAP': 0.6246569280823072} {'mAP': 0.6246569280823072}

Hi, may I ask where you did the test? colab?

from vectormapnet_code.

@colahe On my local machine

from vectormapnet_code.

@colahe On my local machine
Can you give me a peek at the instructions for the tests entered? I type: python tools/test.py configs/vectormapnet.py ./checkpoint/vectormapnet.pth --format-only but it always appears: CUDA out of memory. Tried to allocate 1.05 GiB (GPU 0; 5.80 GiB total capacity; 2.91 GiB already allocated; 686.94 MiB free; 3.72 GiB reserved in total by PyTorch), thanks a lot!

from vectormapnet_code.

@colahe Sorry for the late reply. It's been a while since I explored this repo. I don't quite remember the specific arguments but I think it is a similar command (maybe the same command that they posted)

from vectormapnet_code.

@colahe Sorry for the late reply. It's been a while since I explored this repo. I don't quite remember the specific arguments but I think it is a similar command (maybe the same command that they posted)

Thanks for your reply, i have solved it!!!

from vectormapnet_code.

Hello, I also faced this situation of very poor evaluation performance. It was the same as yours. How did you solve it? Is there any problem that could not be done properly?

from vectormapnet_code.

I do not quite remember, did you try on the entire val set? You can try visualizing it to see whether it is really poor or not

from vectormapnet_code.