confusing about detection experiment

[Frd027]

Hi, I'm a little confused about the results of the object detection experiment. In your paper, you said the performance are compared by computing the average mAP over three classes(cars, cyclists and pedestrians) in the moderately difficult cases. I want to know under what settings(Confidence, recall) the calculation data was obtained. And There are (bbox, bev, 3d, aos)four kinds of AP values in one catagory, what kind of calculations did you make to get the AP value for that category?

[CSautier]

Hi,
We didn't write the metrics computation for object detection and instead relied entirely of the implementation in OpenPCDet. The main metric mAP over the 3 classes is the official metric of the KITTI benchmark and all the metric you mentioned are reported in the official KITTI benchmark https://www.cvlibs.net/datasets/kitti/eval_object.php?obj_benchmark=3d

[Frd027]

Thanks for your reply. I found these descriptions at :

Is that the settings in object detection?
while I ran the default evaluation code in OpenPCDet, I got some other result (different overlap ratio) like this:

Which made me confused.

[CSautier]

I don't remember exactly, the official benchmark would be the 3d one, and most likely non-R40. Probably in what you show here you also have different levels of difficulties. I'd advise to ask to see in the OpenPCDet codebase which one respects the definition of the official benchmark of KITTI.