hailanyi / casa Goto Github PK
View Code? Open in Web Editor NEWA Cascade Attention Network for 3D Object Detection from LiDAR point clouds
Home Page: https://ieeexplore.ieee.org/abstract/document/9870747
License: Apache License 2.0
A Cascade Attention Network for 3D Object Detection from LiDAR point clouds
Home Page: https://ieeexplore.ieee.org/abstract/document/9870747
License: Apache License 2.0
Hello, I have some questions about the preprocess of the Waymo datasets and the evaluation. I use the official code to generate the datasets and when I train the model, some information is as follows, is that right?
INFO Database filter by min points Vehicle: 175529 =>159486
INFO Database filter by min points Pedestrian: 63325 => 51739
INFO Database filter by min points Cyclist: 21442 => 19535
INFO Database filter by difficulty Vehicle: 159486 => 159486
INFO Database filter by difficulty Pedestrian: 51739 => 51739
INFO Database filter by difficulty Cyclist: 19535 => 19535
I used 20% training dataset to train, during training I changed the MAX_NUMBER_OF_VOXELS to 150000, BATCH_SIZE_PER_GPU to 1, when I evaluated the model on the whole validation set of Waymo with 29 epoch, the metrics of the recall are as follows:
INFO recall_roi_0.3: 0.385299
INFO recall_rcnn_0.3: 0.374685
INFO recall_roi_0.5: 0.308684
INFO recall_rcnn_0.5: 0.302918
INFO recall_roi_0.7: 0.169099
INFO recall_rcnn_0.7: 0.168316
INFO Average predicted number of objects(39987 samples): 380.692
I'd like to know if I'm having problems in preprocessing the dataset, or if I shouldn't change any settings. Hope to get your reply!
Hi,
im struggling with interpreting the results of the evaluation of the KITTI data.
Can somebody please explain how to read those results e.g.:
Car [email protected], 0.70, 0.70:
bbox AP:90.8339, 90.0913, 89.7052
bev AP:90.3835, 88.7014, 88.1821
3d AP:89.8144, 86.5609, 79.3242
aos AP:90.81, 90.01, 89.55
Car [email protected], 0.70, 0.70:
bbox AP:96.9829, 95.9043, 93.4644
bev AP:93.9380, 90.2077, 87.7245
3d AP:93.0861, 86.3323, 81.8653
aos AP:96.96, 95.77, 93.28
Car [email protected], 0.50, 0.50:
bbox AP:90.8339, 90.0913, 89.7052
bev AP:98.5752, 90.0662, 89.7175
3d AP:90.8197, 90.0441, 89.6911
aos AP:90.81, 90.01, 89.55
Car [email protected], 0.50, 0.50:
bbox AP:96.9829, 95.9043, 93.4644
bev AP:99.1163, 95.9381, 93.4898
3d AP:96.9698, 95.8939, 93.4472
aos AP:96.96, 95.77, 93.28
I already got the difference between AP and AP_R40.
Do the three columns stand for the three difficulty levels (easy, moderate, hard)?
What are e.g. 0.70, 0.50, 0.50
in the header of every block standing for?
As I am quite new to this topic, any help would be highly appreciated.
As the parameter in official config “waymo_tracking_dataset.yaml”
NUM_POINT_FEATURES: 5
used_feature_list: ['x', 'y', 'z', 'intensity', 'elongation'],
src_feature_list: ['x', 'y', 'z', 'intensity', 'elongation'],
the points features dim(1) is 5 in voxel_output = self._voxel_generator.point_to_voxel(tv.from_numpy(points)), which is 4 in kitti.
however it runs
voxel_output = self._voxel_generator.point_to_voxel(tv.from_numpy(points))
RuntimeError: /io/build/temp.linux-x86_64-cpython-39/spconv/build/core_cc/src/csrc/sparse/all/ops_cpu3d/Point2VoxelCPU/Point2VoxelCPU_point_to_voxel_static.cc(22)
num_features == voxels.dim(2) assert faild. your points num features doesn't equal to voxel.
which does not appear in kitti.
It should be ok whether the point feature dim(1) is 4 or 5.
So is this because some parameters in config should be changed or others and what should I do.
Thanks for your answer.
I trained the CasA-V for 80 epochs, 3 classes., while I found I can't reach car moderate 3D precision which recall is 11 and by the way ,the other two classes reached the precision as you write in paper.
I think my batchsize is too small, it's two. I wonder can I use two batchsize to obtain high precision? How to reset learning rate?
Here is the result.
*************** EPOCH 80 EVALUATION *****************
bbox AP:90.7547, 89.7766, 89.5010
bev AP:90.2147, 88.2992, 88.0006
3d AP:89.6871, 79.6853, 79.2249
aos AP:90.74, 89.67, 89.31
Car [email protected], 0.70, 0.70:
bbox AP:96.8148, 95.5741, 93.2506
bev AP:93.6572, 89.8203, 87.5100
3d AP:92.9158, 84.1482, 83.7333
作者您好,iou3d_nms_utils.py引用的iou3d_nms_cuda在框架中不存在,想问一下有什么解决的建议?
大佬你好,在我的设备上pcdet0.3版本似乎不能进行多卡训练,运行script文件后就没有响应了,但是最新版本的pcdet能够进行多卡训练,我的设备是8*3090!请问你遇到过这个问题吗!
AttributeError: 'EasyDict' object has no attribute 'TEST_AUGMENTOR', what should I do? thank you
Does Cas-A approach work for stream of point cloud data (i.e., for point cloud sequences)?
Is this a trick about 'get_anchor_mask'?
I am confuse about this aapear in CasA and TED.
It doesn't show up in original pcdet.
感谢你提供了一份很好的开源代码。
我在自己的数据集上训练时,仅训练了车这一类,map值在81左右,但 recall_roi_0.3: 0.988935 和 recall_rcnn: 0.902230 差距较大,请问这正常吗?
如果不正常,需要调节哪些参数呢?
In batch_dict there is no key such as st_features_2d. Can you explain briefly what are the does this key means?
when I use distributed.launch, the following message appears and is stuck,how can I deal this problem, thanks:
WARNING:torch.distributed.run:
Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.
Hello. I try to build our own algorithm in this baseline. I have trained all of the KITTI training set and submit the test results to the website of KITTI. The test results on Car categories are very good, but I find that our test results on the Pedestrian and Cyclist categories are worse. Can you give me some advice for these two categories or should I change some configs? Besides, which metrics should I employ to determine the optimal epoch for submission, and is it still advisable to use validation set metrics for evaluation?
作者您好,我在win10系统下运行test.py,提示没有ccimport.buildmeta模块如下:
File "D:\anaconda\lib\site-packages\spconv_init_.py", line 15, in
from . import build as build
File "D:\anaconda\lib\site-packages\spconv\build.py", line 17, in
import pccm
File "D:\anaconda\lib\site-packages\pccm_init.py", line 2, in
from pccm import builder, core, middlewares
File "D:\anaconda\lib\site-packages\pccm\builder_init_.py", line 1, in
from .pybind import build_library, build_pybind
File "D:\anaconda\lib\site-packages\pccm\builder\pybind.py", line 7, in
from pccm.core import Class, CodeFormatter, CodeGenerator, ManualClassGenerator
File "D:\anaconda\lib\site-packages\pccm\core_init_.py", line 16, in
from ccimport.buildmeta import BuildMeta, _unique_list_keep_order
ModuleNotFoundError: No module named 'ccimport.buildmeta'
想问一下需要怎么解决?
作者您好!
我们想把模型转成onnx,请问可以把发布的pth文件转成onnx上传吗
hi, rhank you for your work, I have some problems.
step1. I changed the input point cloud from a single camera to multiple cameras( modify get_fov_flag function), using a dataset similar to nuscene.
step2. I changed POINT_CLOUD_RANGE from [0, -40, -3, 70.4, 40, 1] to [-51.2, -51.2, -3.0, 51.2, 51.2, 1.0] and change GRID_OFFSETS: from [0., 40.] to [0., 0.]
but training loss is unstable and results are terrible, only a few cars are detected.
I'd like to ask for your some advices on how to change the code or configuration, Thx!
Where can i get the paper? I do not have access to IEEE
Hello, How can I get the results on the KITTI validation set for different distances?
Can you give me some suggestions? Thank you!
作者您好,如果已经配置好openpcdet环境了的话,运行这个程序还需要在配置好的openpcdet的环境里面进行develop嘛?运行的时候需要把相关的yaml文件等放到openpcdet那个项目里面嘛,因为运行时候的ROOT_DIR好像显示的是openpcdet所在的位置,运行的时候也会显示KeyError,应该怎么解决呢?
Hi, as i see in data format, it must have image 2D,
Does CasA support predict/train with cloudpoint only ?
Hello, I have made improvements to your baseline, and all three categories have improved on the validation set. However, the test results submitted to the kitti official website are much lower than your results. Could you please give me some suggestions?
We need to use a slightly lower score threshold for test split. Does the scor threshold stand for the value in POST_PEOCESSING or other configs? And why we need to use a lower one? I tried some lower value for POST_PEOCESSING's SCORE_THRESHOLD, and find increase in number of objects per sample. Is that the reason?
How to visualize the result please?
I used demo.py from openpcdet
Traceback (most recent call last):
File "/home/project/CasA/tools/demo.py", line 113, in <module>
main()
File "/home/project/CasA/tools/demo.py", line 99, in main
pred_dicts, _ = model.forward(data_dict)
ValueError: too many values to unpack (expected 2)
作者您好!我想要将发布的CasA-V模型转换到onnx格式做一些测试,把CasA-V.pth文件下载后放在了tools文件夹中,并且在tools/test.py文件中删除掉原来190-195行内容,并在189行后添加如下代码:
model.load_params_from_file(filename=args.ckpt, logger=logger, to_cpu=dist_test)
# set the model to inference mode
model.eval()
# Let's create a dummy input tensor
dummy_input = torch.randn(1, 3, 244, 244, requires_grad=True)
# Export the model
torch.onnx.export(model=model, # model being run
args=dummy_input, # model input (or a tuple for multiple inputs)
f="Output-CasA.onnx", # where to save the model
export_params=True, # store the trained parameter weights inside the model file
opset_version=10, # the ONNX version to export the model to
do_constant_folding=True, # whether to execute constant folding for optimization
input_names=['modelInput'], # the model's input names
output_names=['modelOutput'], # the model's output names
dynamic_axes={'modelInput': {0: 'batch_size'}, # variable length axes
'modelOutput': {0: 'batch_size'}}
)
pycharm报错信息:
Traceback (most recent call last):
File "/home/jlf/PycharmProjects/CasA/tools/pthToOnnx.py", line 148, in <module>
main()
File "/home/jlf/PycharmProjects/CasA/tools/pthToOnnx.py", line 123, in main
torch.onnx.export(model=model, # model being run
File "/home/jlf/miniconda3/lib/python3.8/site-packages/torch/onnx/__init__.py", line 203, in export
return utils.export(model, args, f, export_params, verbose, training,
File "/home/jlf/miniconda3/lib/python3.8/site-packages/torch/onnx/utils.py", line 86, in export
_export(model, args, f, export_params, verbose, training, input_names, output_names,
File "/home/jlf/miniconda3/lib/python3.8/site-packages/torch/onnx/utils.py", line 526, in _export
graph, params_dict, torch_out = _model_to_graph(model, args, verbose, input_names,
File "/home/jlf/miniconda3/lib/python3.8/site-packages/torch/onnx/utils.py", line 366, in _model_to_graph
graph, torch_out = _trace_and_get_graph_from_model(model, args)
File "/home/jlf/miniconda3/lib/python3.8/site-packages/torch/onnx/utils.py", line 319, in _trace_and_get_graph_from_model
torch.jit._get_trace_graph(model, args, strict=False, _force_outplace=False, _return_inputs_states=True)
File "/home/jlf/miniconda3/lib/python3.8/site-packages/torch/jit/__init__.py", line 338, in _get_trace_graph
outs = ONNXTracedModule(f, strict, _force_outplace, return_inputs, _return_inputs_states)(*args, **kwargs)
File "/home/jlf/miniconda3/lib/python3.8/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/jlf/miniconda3/lib/python3.8/site-packages/torch/jit/__init__.py", line 421, in forward
graph, out = torch._C._create_graph_by_tracing(
File "/home/jlf/miniconda3/lib/python3.8/site-packages/torch/jit/__init__.py", line 412, in wrapper
outs.append(self.inner(*trace_inputs))
File "/home/jlf/miniconda3/lib/python3.8/site-packages/torch/nn/modules/module.py", line 720, in _call_impl
result = self._slow_forward(*input, **kwargs)
File "/home/jlf/miniconda3/lib/python3.8/site-packages/torch/nn/modules/module.py", line 704, in _slow_forward
result = self.forward(*input, **kwargs)
File "/home/jlf/PycharmProjects/CasA/pcdet/models/detectors/voxel_rcnn.py", line 11, in forward
batch_dict = cur_module(batch_dict)
File "/home/jlf/miniconda3/lib/python3.8/site-packages/torch/nn/modules/module.py", line 720, in _call_impl
result = self._slow_forward(*input, **kwargs)
File "/home/jlf/miniconda3/lib/python3.8/site-packages/torch/nn/modules/module.py", line 704, in _slow_forward
result = self.forward(*input, **kwargs)
File "/home/jlf/PycharmProjects/CasA/pcdet/models/backbones_3d/vfe/mean_vfe.py", line 27, in forward
if 'semi_test' in batch_dict:
File "/home/jlf/miniconda3/lib/python3.8/site-packages/torch/tensor.py", line 502, in __contains__
raise RuntimeError(
RuntimeError: Tensor.__contains__ only supports Tensor or scalar, but you passed in a <class 'str'>.
Process finished with exit code 1
大佬您好!
请问您文章中Fig.5的数据是如何获取的呢?(就是不同距离的AP),渴望得到您的指导!
谢谢!
Hello. May I ask, how do I train my model when submitting it to the KITTI official website? Train 80 epochs with 80% data? What is the specific setting for the 'use slightly lower score threshold'? Thank you.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.