kitti_eval's People
Forkers
zdx3578 mudelin yheno starsmydestination hwd8868 ysfalo lihua213 q412611827 rjzz superxiaoying peiliangli bzdfzfer poodarchu hdjang zqdeepbluesky xinyugaotudelft lhyfst pyun-ram jlqzzz lkskstlr chaomath argosdh hexuanfang suljaxm twizwei jguertl liuyanghan zyg11 royzon 3dshijuezhazha shipingsong bilgecanpullinen yuichinagao luolizhuiyuyun720 kunjansingh ma-raca bilal-a-qureshi poincarelee tendernessmook lordoncn guangjinzheng daxtonyin littleflyfishkitti_eval's Issues
ERROR : NO results in plot and result.txt
The output :
Thank you for participating in our evaluation!
Loading detections...
number of files for evaluation: 1
Difference between COCO style AP at iou 0.5 and Kitti 2D results with iou set as 0.5?
I would expect similar values for both cases, but I do not get similar values. Is there any difference between these two?
ERROR: Couldn't read: 003323.txt of ground truth
How can I solve this problem?
Thank you for participating in our evaluation!
Loading detections...
number of files for evaluation: 3769
ERROR: Couldn't read: 003323.txt of ground truth. Please write me an email!
An error occured while processing your results.
Could you give some explanation about output?
First, thanks for your work. As a beginner, I don't know what is the output represent.
There are four txt for each class and related '.gp', I don't know the number in each row.
Could you give me some explanation or give me some Tutorials? Thanks.
stopped with segmentation fault
First, thanks for your amazing work!
While it's really helpful, I really need your help.
Here are the summary I'm currently seeing.
car_detection_AP : 85.031853 81.267220 74.636650
sh: 1: gnuplot: not found
sh: 1: gnuplot: not found
Error: /undefinedfilename in (car_detection_AP.eps)
Operand stack:Execution stack:
%interp_exit .runexec2 --nostringval-- --nostringval-- --nostringval-- 2 %stopped_push --nostringval-- --nostringval-- --nostringval-- false 1 %stopped_push
Dictionary stack:
--dict:961/1684(ro)(G)-- --dict:0/20(G)-- --dict:78/200(L)--
Current allocation mode is local
Last OS error: No such file or directory
GPL Ghostscript 9.26: Unrecoverable error, exit code 1
sh: 1: pdfcrop: not found
car_orientation_AOS : 84.987823 79.064232 69.322250
sh: 1: gnuplot: not found
sh: 1: gnuplot: not found
Error: /undefinedfilename in (car_orientation_AOS.eps)
Operand stack:Execution stack:
...
...
...
Execution stack:
%interp_exit .runexec2 --nostringval-- --nostringval-- --nostringval-- 2 %stopped_push --nostringval-- --nostringval-- --nostringval-- false 1 %stopped_push
Dictionary stack:
--dict:961/1684(ro)(G)-- --dict:0/20(G)-- --dict:78/200(L)--
Current allocation mode is local
Last OS error: No such file or directory
GPL Ghostscript 9.26: Unrecoverable error, exit code 1
sh: 1: pdfcrop: not found
When it starts evaluating bird's eye detection, a critical segmentation fault occurs, and it's all stopped.
It has been happening in other similar projects, but I can't figure out what the real issues are.
It will be so helpful if you could give me some piece of hints. Thanks in advance.
Apart from this issue, I have one more question.
How can we decide the 16th value 'score/confidence' in the label? This time, I just put any random number, but I have no idea of calculating the value. Does it come from 2D Detector which usually outputs confidence probability by softmax classification?
Again, thanks for you work, and looking forward to seeing your reply!
How are files organized?
Thank you for participating in our evaluation!
mkdir: cannot create directory ‘../../eval_kitti/results/exp1/data12/plot’: File exists
Loading detections...
number of files for evaluation: 0
why 0?
Criteria for filtering detections
Do the detections get filtered based on the size of 2D box or dimensions of the detected object?
number of files for evaluation: 0
The plot folder is generated, but it is empty,and number of files for evaluation: 0
why???
Question about eval_class
Hi, thanks for your hard work on implementing the evaluation code. I am looking into the code and have a hard time connecting it to the AP formula in the paper. Most confusion comes from eval_class, which calculates the AP of a single class. Since it's hard for me to organize my questions into one paragraph, I list all of them in the below:
-
Why use detection scores as recall? More specifically, why you use scores as threshold when computing statistics? As far as I know, there's no threshold when computing average precision, which only involves calculating precisions for discretized recalls.
-
Following the previous question, the discretized recalls are 0, 0.1,....,1 as described in the paper. Where do you use this information in eval_class
3.What does N_SAMPLE_PTS mean, and how do you come up with 41?
- Why we need getThresholds and how does it work?
To sum up, I don't quite understand how it connects to the AP formula, and would appreciate if you can help me out!!!
what is the meaning of '11'
kitti_eval/evaluate_object_3d_offline.cpp
Line 722 in 955c04c
Official KITTI 3D Object Detection Benchmark
Hi all,
just wanted to leave a note that thanks to the code provided by @prclibo, we released an official 3D object detection benchmark. Check it out at: http://www.cvlibs.net/datasets/kitti/eval_3dobject.php
2D AP vs COCO AP
Hi, I would expect the 2D AP to be somewhat close to the value I get when I evaluate it with COCO evaluation script.
Kitti 2D (iou=0.5) AP: 35.7
COCO AP(iou=0.5) : 56.5
Any idea why there is a disparity?
number of files for evaluation: 0
I have my result .txt files in my result_dir/
, and my validation .txt files in my 'ground_truth_dir/', but every time I will just get the results like this,
Thank you for participating in our evaluation!
Loading detections...
number of files for evaluation: 0
done.
Your evaluation results are available at:
../Stereo-RCNN/models_stereo/result
I put the label.txt in one folder and the result.txt in another folder, however it doesn't work
ERROR: Couldn't read: 000000.txt of ground truth. Please write me an email!
I put the training data's label_2 under the folder, but it can't works
comments can not be compatible with code
// this ground truth is not of the current or a neighboring class and therefore ignored if(ignored_gt[i]==-1) continue;
The case of neighboring class is not included when ignored_gt==-1.
May be some problems happen here
There is a error in the evaluate_object_3d_offline.cpp
car_detection_AP : 35.000000 79.857147 84.663734
sh: 1: gnuplot: not found
sh: 1: gnuplot: not found
Error: /undefinedfilename in (car_detection_AP.eps)
Operand stack:
Execution stack:
%interp_exit .runexec2 --nostringval-- --nostringval-- --nostringval-- 2 %stopped_push --nostringval-- --nostringval-- --nostringval-- false 1 %stopped_push
Dictionary stack:
--dict:960/1684(ro)(G)-- --dict:0/20(G)-- --dict:77/200(L)--
Current allocation mode is local
Last OS error: No such file or directory
GPL Ghostscript 9.26: Unrecoverable error, exit code 1
sh: 1: pdfcrop: not found
car_orientation_AOS : 18.006918 38.727802 40.937248
sh: 1: gnuplot: not found
sh: 1: gnuplot: not found
Error: /undefinedfilename in (car_orientation_AOS.eps)
Operand stack:
Execution stack:
%interp_exit .runexec2 --nostringval-- --nostringval-- --nostringval-- 2 %stopped_push --nostringval-- --nostringval-- --nostringval-- false 1 %stopped_push
Dictionary stack:
--dict:960/1684(ro)(G)-- --dict:0/20(G)-- --dict:77/200(L)--
Current allocation mode is local
Last OS error: No such file or directory
GPL Ghostscript 9.26: Unrecoverable error, exit code 1
sh: 1: pdfcrop: not found
car_detection_BEV_AP : 35.000000 82.357140 87.229523
sh: 1: gnuplot: not found
sh: 1: gnuplot: not found
Error: /undefinedfilename in (car_detection_BEV_AP.eps)
Operand stack:
I am looking forward to your reply!
Sanity check issue
When I give the ground truths with some score directly as results but always get 90.90 as the APs. Does anyone achieve AP of 100?
Segmentation fault Error.
car_detection AP: 0.417685 0.280079 0.230861
PDFCROP 1.38, 2012/11/02 - Copyright (c) 2002-2012 by Heiko Oberdiek.
==> 1 page written on `car_detection.pdf'.
save preds_val/plot/car_orientation.txt
car_orientation AP: 0.195098 0.127194 0.103741
PDFCROP 1.38, 2012/11/02 - Copyright (c) 2002-2012 by Heiko Oberdiek.
==> 1 page written on `car_orientation.pdf'.
Segmentation fault
in this file, evaluate_object_3d_offline.cpp.
Thanks.
better AP with moderate and hard than easy for BEV evaluation
Hello,
I see the following:
car_detection_ground AP: 63.954884 73.779037 74.295746
As I understand, the scores are ordered as easy, moderate, hard.
If this this correct then is there something wrong with above results?
How are hard samples having better score than easy ones?
Please help.
Best Regards
Sambit
Evaluate BEV detections
I'm using the KITTI cpp code (evaluate_object_3d_offline.cpp) to evaluate my birds eye view bounding box detections. The code filters the detections into easy, moderate and hard by the height of the detected object's bounding box in the image space (i.e. pixels). I do not have image detections at all and so all my detections are being ignored (lines 447-452). I'm experimenting solely on BEV data. Any idea how to evaluate ONLY BEV detections (without any corresponding image detections). Is there a reference somewhere for this?
Weird Result of running calculate_iou_partly
Running the following script,
`
**import numpy as np
from det3d.datasets.utils.eval import calculate_iou_partly
fake_gt_annos = []
anno = {}
anno.update(
{
"name": np.array(['Car']),
"location": np.array([[0.0, 0.0, 0.0]]),
"dimensions": np.array([[1.0, 1.0, 1.0]]),
"rotation_y": np.array([np.pi/8]),
}
)
fake_gt_annos.append(anno)
rets = calculate_iou_partly(
fake_gt_annos, fake_gt_annos, metric=1, num_parts=1, z_axis=2, z_center=0.5
)
print(rets)**
I got:
([array([[7.17528081e-09]])],
[array([[7.17528081e-09]])],
array([1]),
array([1]))
`
Why? I use the same parameters of dt_annos and gt_annos when calling calculate_iou_partly, and I expect to get the overlap result = 1.
Is this a bug? Or I misunderstand something?
I want to know AP3d index is AP3d|R40|IOU0.7>=0.7 or just AP3d|IOU>0.7
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.