Brain Tumor MRI Classification | Step 2 Cross Validation Techniques and Hyperparameter Tuning

[theiturhs]

Description

This contains cross validation techniques and hyperparameter tuning. This is how these are implemented

Cross Validation Technique:

Different techniques for distributing dataset into training and testing dataset are implemented:

• Hold-Out CV: 0.9380
• K-Fold Cross Validation: 0.9604
• Repeated K-Fold: 0.9585
• Stratified K-Fold: 0.9548

Along with average overall accuracies, class wise accuracies are calculated.

Out of these techniques, K-Fold CV technique gives better overall accuracy and class wise accuracy.

Hyper-parameter Tuning

For this Learner module, FastAI, has been considered which provides a way to create and fine-tune CNN models. We can specify a pre-trained model architecture and fine-tune it on the dataset.

These implementation are done:

• Random Search optimization algorithm - Run 1: 95.27%
• Random Search optimization algorithm - Run 2: 95.53%
• Hyperparameter Optimization with Optuna's Successive Halving Pruner: 98.51%

Related Issues

Fixes #96

Testing Instructions

Necessary modules are already specified in the notebook itself. The work is carried out on Kaggle with accelarator set to GPU T4 x2.

Checklist

Make sure to check off all the items before submitting. Mark with [x] if done.

• I have performed a self-review of my code
• I have commented my code, particularly in hard-to-understand areas
• My changes generate no new warnings
• I am working on this issue under GSSOC

[github-actions]

Great job, @theiturhs! 🎉 Thank you for submitting your pull request. Your contribution is valuable and we appreciate your efforts to improve our project.

We will promptly review your changes and offer feedback. Keep up the excellent work! Kindly remember to check our contributing guidelines

[SrijanShovit]

@theiturhs Looks like you have used very less number of epochs. Why so?

[theiturhs]

Yes. The number of epochs for deciding the cross validation technique is set to 3 just for comparison purposes and find the best technique. Where in hyper-parameter tuning, you can find best parameters which includes epochs too where the range provided was 5-15. The obtained results are not the final ones. These were for comparison purposes and selecting the best method that we will perform at the end with entire dataset while training our model.

@SrijanShovit

[SrijanShovit]

Hmmm...still good work. I appreciate. Let me merge this. Go ahead with same thing for prediction and plots.