Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

meaning of provided result files #145

Open
yrt1993 opened this issue Sep 5, 2023 · 2 comments
Open

meaning of provided result files #145

yrt1993 opened this issue Sep 5, 2023 · 2 comments

Comments

@yrt1993
Copy link

yrt1993 commented Sep 5, 2023

Hi, I want to know the detailed meaning of the result file in the resources provides in the link:
https://github.com/tusen-ai/SST/blob/main/docs/CTRL_instructions.md#results

specifically, how do you get these files? i.e., their difference?
fsd_base_ped_val.bin
fsd_base_ped_val_forward10.bin
ped_result.bin
ped_result_forward10_no_empty.bin

@Abyssaledge
Copy link
Collaborator

fsd_base_ped_val.bin: tracking/detection results of base detector
fsd_base_ped_val_forward10.bin: tracking/detection results of the base detector with filling missing boxes in forward extension.
ped_result.bin: CTRL baseline results on Pedestrian without bidirectional extension
ped_result_forward10_no_empty.bin: CTRL results on Pedestrian with forward extension and the empty boxes are removed.
forward10 means filling missing boxes in forward extension, filling at most 10 consecutively missing timestamps.

@yrt1993
Copy link
Author

yrt1993 commented Sep 6, 2023

I tried to use the four files to compute waymo metrics. The following is what I get.
It's really large just using fsd_base_ped_val.bin. Are you sure it's just the results of base detector?

And what's the detector?

fsd_base_ped_val.bin
OBJECT_TYPE_TYPE_PEDESTRIAN_LEVEL_1: [mAP 0.863252] [mAPH 0.838515]
OBJECT_TYPE_TYPE_PEDESTRIAN_LEVEL_2: [mAP 0.798981] [mAPH 0.774175]
fsd_base_ped_val_forward10.bin
OBJECT_TYPE_TYPE_PEDESTRIAN_LEVEL_1: [mAP 0.870338] [mAPH 0.845087]
OBJECT_TYPE_TYPE_PEDESTRIAN_LEVEL_2: [mAP 0.808826] [mAPH 0.783306]
ped_result.bin
OBJECT_TYPE_TYPE_PEDESTRIAN_LEVEL_1: [mAP 0.872514] [mAPH 0.845624]
OBJECT_TYPE_TYPE_PEDESTRIAN_LEVEL_2: [mAP 0.809663] [mAPH 0.782687]
ped_result_forward10_no_empty.bin
OBJECT_TYPE_TYPE_PEDESTRIAN_LEVEL_1: [mAP 0.887341] [mAPH 0.859652]
OBJECT_TYPE_TYPE_PEDESTRIAN_LEVEL_2: [mAP 0.828941] [mAPH 0.800911]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants