Epistemic Uncertainty Quantification in MONAI

[jadie1]

Hello, I am a developer who has done some work in epistemic uncertainty quantification in CNNs. Specifically, I did a benchmark of 8 UQ approaches in organ segmentation using MONAI: https://github.com/jadie1/MedSegUQ
I think some of this work could be good to bring into the MONAI project, but I'm unsure the best way to approach contributing since it spans many APIs. Potential contributions include:

• Concrete dropout and batch ensemble for convolution layers
• Loss functions
• Training pipelines for methods such as SWAG
• Functionality for getting predicted uncertainty maps from various methods
• Metrics for UQ calibration assessment

Any advice on the best way to approach contributing would be appreciated!

[KumoLiu]

Hi @jadie1,

It's great to hear about your work in epistemic uncertainty quantification for CNNs, and your benchmark of different UQ approaches in organ segmentation using MONAI! Your contributions could certainly be valuable for the MONAI community.
I would suggest breaking them down into manageable parts and contributing them incrementally
You can refer to this contribution guide: https://github.com/Project-MONAI/MONAI/blob/dev/CONTRIBUTING.md
Also you can add a tutorial in the tutorial repo to show how to use the components.

We are looking forward to your contributions to MONAI!
Thanks.

[jadie1]

@KumoLiu Thanks for your response! Creating a tutorial makes a lot of sense. I may even start there with the approaches that don't require new functionality like naive ensembling and MC dropout. I could make a version a tutorial like spleen_segmentation_3d that demonstrates how to quantify and visualize uncertainty.

From there I could work on incrementally contributing the other approaches and incrementally update the tutorial accordingly.