Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Precision calculations #23

Open
exodiasleftleg opened this issue Apr 3, 2024 · 0 comments
Open

Precision calculations #23

exodiasleftleg opened this issue Apr 3, 2024 · 0 comments

Comments

@exodiasleftleg
Copy link

exodiasleftleg commented Apr 3, 2024

Hello @tamerthamoqa,
I am using this repo on a custom dataset, but I encountered some weird behaviour, every other metric constantly changes during epochs, but Precision always stays the same at 0.5000+-0.5000. I have also defined a custom validation dataset for which I generated an equal amount of positive and negative pairs in total consisting of 422 pairs, here's an example on one of the epochs:

100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 100/100 [00:22<00:00, 4.45it/s]
Epoch 137: Number of valid training triplets in epoch: 4
Validating on LFW! ...
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:01<00:00, 2.65it/s]
Accuracy on LFW: 0.8818+-0.0417 Precision 0.5000+-0.5000 Recall 0.4364+-0.4368 ROC Area Under Curve: 0.1977 Best distance threshold: 1.16+-0.03 TAR: 0.2068+-0.2145 @ FAR: 0.0000

I tried reducing the range of threshold from the default to:
thresholds_roc = np.arange(0.5, 0.8, 0.1)
thresholds_val = np.arange(0.5, 0.8, 0.1)
But the precision stays the same. My question is what's going on with the precision calculations as far as I have reviewed the calculation logic checks out?
Thank you in advance.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant