Fine-tuned fast_base detection model is picking up noise

[hanshupe007]

After fine-tuning the fast_base detection model to overcome some minor detection issues, it starts to detect very tiny pieces of noise, like dots as text, typically within another box. I already tried already to tune the synthetic data or tune hyper-parameters, but can't really solve the problem. Any suggestions I can try?

[felixdittrich92]

Hi @hanshupe007 👋,

If the only problem are such super small detections i would suggest to play a bit around with the binarization threshold and box threshold values before continue any training.
See: https://mindee.github.io/doctr/using_doctr/using_models.html#advanced-options

The default value for fast_base is really small 0.1 for both so try to increase the values a bit (between 0.1 and 0.9)

A sec option would be to plug a hook into the pipeline where you could filter such boxes by it's area before passing it to the recognition_predictor
See: same link

Could you describe a bit more how you trained the model (fine tuned with --pretrained) or from scratch ?
About the mentioned hyperparameter search how was that done ? By hand or used something like opunta to run several trials ?

Best,
Felix

[hanshupe007]

Thanks that's useful. Can the binarization threshold only be set for inference, or also during training (in train_pytorch.py), so that it impacts the loss? What's the difference between box and binary threshold?

I fine-tuned it with a few thousand samples and --pretrained, also tried freeze-backbone and from scratch, but was slightly worse. Hyperparameter search was done by hand, increasing batch size up to 16 made it usually worse, tried a wide range of learning rates, and epochs, but couldn't get rid of the tiny boxes.

As mentioned once before, I don't see the tiny false positive boxes reflected in any metric (including the loss), training runs with false positive or false negative boxes result in better metrics than the ones without (but slightly misaligned boxes), so I do currently a visual validation.

[felixdittrich92]

You can but both are parameters for the post-processing so it would show only an impact on recall, precision and miou

Could you share the logs from your "best" run ?
Does the loss start to stuck after some epochs ?

I say several times that Adam as optimizer seems to be a bit to "aggresive" in some cases maybe RMSProp would be an alternative and if the loss starts to stuck after ~3 epochs i would try to switch to a StepLR-Scheduler or do a test with a constant lr without a scheduler

[felixdittrich92]

In general i think it would be super usefull to have something like --hyper-search with optuna for example to find the best values .. it's on my todo list but other things have a bit higher priority actually 😅

[hanshupe007]

Ok, will test the threshold parameters, maybe it solves the issue.

Yes, I had the feeling validation loss was lowest at the very first epochs even with small lr, I can check later in more detail.