Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Metabolic Syndrome Prediction | 5. Model prediction and evaluation #156

Merged
merged 3 commits into from
Aug 9, 2024
+293,748 −13
Merged

Metabolic Syndrome Prediction | 5. Model prediction and evaluation #156

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
ae65702
initial work
Arihant-Bhandari Jun 2, 2024
2b1953c
Added models with proper metrics in place
Arihant-Bhandari Jun 20, 2024
464f550
Added models, ANN model implementations with saved models, tabulated …
Arihant-Bhandari Jun 24, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 44 additions & 0 deletions Metabolic Syndrome Prediction/METRICS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
# Performance Metrics for Evaluated Models:

We employed GridSearchCV along with K-fold cross-validation to identify and assess the performance of various models. Using the best hyperparameters identified through GridSearchCV, we implemented the models to determine the optimal metrics. Below are the comprehensive metrics gathered during the evaluation:

| Model | Data Type | Mean Accuracy | Std Dev Accuracy | Mean Precision | Std Dev Precision | Mean Recall | Std Dev Recall | Mean F1-score | Std Dev F1-score | Mean ROC AUC | Std Dev ROC AUC | Accuracy | Precision | Recall | F1 Score | ROC AUC Score | Confusion Matrix |
|-------------------------|--------------|-----------------|------------------|-----------------|--------------------|----------------|-----------------|-----------------|------------------|-----------------|-----------------|----------------|----------------|----------------|----------------|----------------|--------------------------|
| XGB Classifier | Normalized | 0.8929719917 | 0.0192048008 | 0.8536406786 | 0.0457545010 | 0.8334704672 | 0.0369092534 | 0.8422469002 | 0.0270992804 | 0.8787484562 | 0.0197900479 | 0.8818635607 | 0.8556149733 | 0.7843137255 | 0.8184143223 | 0.8581518249 | [[370, 27], [44, 160]] |
| | Standardized | 0.8929719917 | 0.0192048008 | 0.8536406786 | 0.0457545010 | 0.8334704672 | 0.0369092534 | 0.8422469002 | 0.0270992804 | 0.8787484562 | 0.0197900479 | 0.8818635607 | 0.8556149733 | 0.7843137255 | 0.8184143223 | 0.8581518249 | [[370, 27], [44, 160]] |
| Random Forest Classifier| Normalized | 0.8813087828 | 0.0121453899 | 0.8452559083 | 0.0342530423 | 0.8030120482 | 0.0453500693 | 0.8221130377 | 0.0200200422 | 0.8625678640 | 0.0171193742 | 0.8752079867 | 0.8486486486 | 0.7696078431 | 0.8071979434 | 0.8495394379 | [[369, 28], [47, 157]] |
| | Standardized | 0.8813087828 | 0.0121453899 | 0.8452559083 | 0.0342530423 | 0.8030120482 | 0.0453500693 | 0.8221130377 | 0.0200200422 | 0.8625678640 | 0.0171193742 | 0.8752079867 | 0.8486486486 | 0.7696078431 | 0.8071979434 | 0.8495394379 | [[369, 28], [47, 157]] |
| Decision Tree Classifier| Normalized | 0.8608869295 | 0.0165113765 | 0.7850972759 | 0.0329836768 | 0.8199970614 | 0.0243939124 | 0.8016412899 | 0.0210224940 | 0.8511014897 | 0.0157131535 | 0.8635607321 | 0.7961165049 | 0.8039215686 | 0.8000000000 | 0.8490640589 | [[355, 42], [40, 164]] |
| | Standardized | 0.8608869295 | 0.0165113765 | 0.7850972759 | 0.0329836768 | 0.8199970614 | 0.0243939124 | 0.8016412899 | 0.0210224940 | 0.8511014897 | 0.0157131535 | 0.8635607321 | 0.7961165049 | 0.8039215686 | 0.8000000000 | 0.8490640589 | [[355, 42], [40, 164]] |
| AdaBoost Classifier | Normalized | 0.8725518672 | 0.0150556447 | 0.8250753635 | 0.0348545791 | 0.7993682045 | 0.0355904361 | 0.8110587434 | 0.0216346715 | 0.8550436927 | 0.0170365789 | 0.8618968386 | 0.8134715026 | 0.7696078431 | 0.7909319899 | 0.8394638712 | [[361, 36], [47, 157]] |
| | Standardized | 0.8725518672 | 0.0150556447 | 0.8250753635 | 0.0348545791 | 0.7993682045 | 0.0355904361 | 0.8110587434 | 0.0216346715 | 0.8550436927 | 0.0170365789 | 0.8618968386 | 0.8134715026 | 0.7696078431 | 0.7909319899 | 0.8394638712 | [[361, 36], [47, 157]] |
| LGBM Classifier | Normalized | 0.8850535961 | 0.0142850569 | 0.8412698301 | 0.0299211820 | 0.8213341170 | 0.0479278391 | 0.8298603267 | 0.0235242528 | 0.8698261329 | 0.0200431423 | 0.8918469218 | 0.8638743455 | 0.8088235294 | 0.8354430380 | 0.8716661728 | [[371, 26], [39, 165]] |
| | Standardized | 0.8850535961 | 0.0142850569 | 0.8412698301 | 0.0299211820 | 0.8213341170 | 0.0479278391 | 0.8298603267 | 0.0235242528 | 0.8698261329 | 0.0200431423 | 0.8918469218 | 0.8638743455 | 0.8088235294 | 0.8354430380 | 0.8716661728 | [[371, 26], [39, 165]] |
| Logistic Regression | Normalized | 0.8384128631 | 0.0288940906 | 0.8072888487 | 0.0550171345 | 0.6959006759 | 0.0443465784 | 0.7468449823 | 0.0442479641 | 0.8042572798 | 0.0313473166 | 0.8202995008 | 0.7637362637 | 0.6813725490 | 0.7202072539 | 0.7865301032 | [[354, 43], [65, 139]] |
| | Standardized | 0.8384128631 | 0.0288940906 | 0.8072888487 | 0.0550171345 | 0.6959006759 | 0.0443465784 | 0.7468449823 | 0.0442479641 | 0.8042572798 | 0.0313473166 | 0.8202995008 | 0.7637362637 | 0.6813725490 | 0.7202072539 | 0.7865301032 | [[354, 43], [65, 139]] |
| Extra Trees Classifier | Normalized | 0.8688053250 | 0.0107353815 | 0.8494329074 | 0.0200132459 | 0.7505730238 | 0.0376898522 | 0.7962051073 | 0.0208023255 | 0.8404562289 | 0.0162864382 | 0.8535773710 | 0.8152173913 | 0.7352941176 | 0.7731958763 | 0.8248259001 | [[363, 34], [54, 150]] |
| | Standardized | 0.8688053250 | 0.0107353815 | 0.8494329074 | 0.0200132459 | 0.7505730238 | 0.0376898522 | 0.7962051073 | 0.0208023255 | 0.8404562289 | 0.0162864382 | 0.8535773710 | 0.8152173913 | 0.7352941176 | 0.7731958763 | 0.8248259001 | [[363, 34], [54, 150]] |
| HistGradBoosting Classifier | Normalized | 0.8917202628 | 0.0144775080 | 0.8445167635 | 0.0343586829 | 0.8407728475 | 0.0379181516 | 0.8416215259 | 0.0213085293 | 0.8795454982 | 0.0169850948 | 0.8951747088 | 0.8691099476 | 0.8137254902 | 0.8405063291 | 0.8753765990 | [[372, 25], [38, 166]] |
| | Standardized | 0.8917202628 | 0.0144775080 | 0.8445167635 | 0.0343586829 | 0.8407728475 | 0.0379181516 | 0.8416215259 | 0.0213085293 | 0.8795454982 | 0.0169850948 | 0.8951747088 | 0.8691099476 | 0.8137254902 | 0.8405063291 | 0.8753765990 | [[372, 25], [38, 166]] |
| Gradient Boosting Classifier | Normalized | 0.8892219917 | 0.0142510869 | 0.8494764200 | 0.0284972512 | 0.8237143697 | 0.0367301674 | 0.8356492645 | 0.0221124433 | 0.8735499205 | 0.0176714259 | 0.8818635607 | 0.8518518519 | 0.7892156863 | 0.8193384224 | 0.8593433595 | [[369, 28], [43, 161]] |
| | Standardized | 0.8892219917 | 0.0142510869 | 0.8494764200 | 0.0284972512 | 0.8237143697 | 0.0367301674 | 0.8356492645 | 0.0221124433 | 0.8735499205 | 0.0176714259 | 0.8818635607 | 0.8518518519 | 0.7892156863 | 0.8193384224 | 0.8593433595 | [[369, 28], [43, 161]] |
| K-Nearest Neighbors | Normalized | 0.8146749654 | 0.0283457368 | 0.7763764918 | 0.0493835548 | 0.6461357626 | 0.0552980301 | 0.7042253299 | 0.0472877233 | 0.7743054851 | 0.0330882711 | 0.8119800333 | 0.7791411043 | 0.6225490196 | 0.6920980926 | 0.7659344594 | [[361, 36], [77, 127]] |
| | Standardized | 0.8146749654 | 0.0283457368 | 0.7763764918 | 0.0493835548 | 0.6461357626 | 0.0552980301 | 0.7042253299 | 0.0472877233 | 0.7743054851 | 0.0330882711 | 0.8119800333 | 0.7791411043 | 0.6225490196 | 0.6920980926 | 0.7659344594 | [[361, 36], [77, 127]] |
| Stacking Classifier | Normalized | 0.81591459197787 | 0.030349300788210683 | 0.7568530963284796 | 0.0505783644964972 | 0.6837790185130767 | 0.0555930314207229 | 0.7174435978837421 | 0.04666704783684153 | 0.7842663535683984 | 0.034526039966267436 | 0.8219633943427621 | 0.7487179487179487 | 0.7156862745098039 | 0.731829573934837 | 0.7961302909072949 | [[348, 49], [58, 146]] |
| | Standardized | 0.81591459197787 | 0.030349300788210683 | 0.7568530963284796 | 0.0505783644964972 | 0.6837790185130767 | 0.0555930314207229 | 0.7174435978837421 | 0.04666704783684153 | 0.7842663535683984 | 0.034526039966267436 | 0.8219633943427621 | 0.7487179487179487 | 0.7156862745098039 | 0.731829573934837 | 0.7961302909072949 | [[348, 49], [58, 146]] |
| Voting Classifier | Normalized | 0.8758921161825727 | 0.017201464058433345 | 0.8277859055185062 | 0.03253056125722735 | 0.8066852776961504 | 0.03596903454697755 | 0.816445823644554 | 0.02551527303659126 | 0.8593351406621522 | 0.019796830876094428 | 0.8801996672212978 | 0.8473684210526315 | 0.7892156862745098 | 0.8172588832487309 | 0.8580839136662222 | [[368, 29], [43, 161]] |
| | Standardized | 0.8758921161825727 | 0.017201464058433345 | 0.8277859055185062 | 0.03253056125722735 | 0.8066852776961504 | 0.03596903454697755 | 0.816445823644554 | 0.02551527303659126 | 0.8593351406621522 | 0.019796830876094428 | 0.8801996672212978 | 0.8473684210526315 | 0.7892156862745098 | 0.8172588832487309 | 0.8580839136662222 | [[368, 29], [43, 161]] |
| Bagging Classifier | Normalized | 0.8929719917012449 | 0.016585366209661924 | 0.8524499345881044 | 0.03963276780343318 | 0.8346605935938879 | 0.04164150530913857 | 0.8421942043182058 | 0.024609787061739656 | 0.8790250480667978 | 0.019001104560078273 | 0.8885191347753744 | 0.8663101604278075 | 0.7941176470588235 | 0.8286445012787724 | 0.8655726774336938 | [[372, 25], [42, 162]] |
| | Standardized | 0.8929719917012449 | 0.016585366209661924 | 0.8524499345881044 | 0.03963276780343318 | 0.8346605935938879 | 0.04164150530913857 | 0.8421942043182058 | 0.024609787061739656 | 0.8790250480667978 | 0.019001104560078273 | 0.8885191347753744 | 0.8663101604278075 | 0.7941176470588235 | 0.8286445012787724 | 0.8655726774336938 | [[372, 25], [42, 162]] |
| CatBoost Classifier | Normalized | 0.8925570539419088 | 0.01736280999628729 | 0.8581554633489977 | 0.0462767969531329 | 0.8261533940640611 | 0.03781186410566183 | 0.8404969985160087 | 0.02465285113204594 | 0.8766661511963456 | 0.01835801213701771 | 0.8968386023294509 | 0.8585858585858586 | 0.8333333333333334 | 0.8457711442786069 | 0.8814021830394627 | [[369, 28], [34, 170]] |
| | Standardized | 0.8925570539419088 | 0.01736280999628729 | 0.8581554633489977 | 0.0462767969531329 | 0.8261533940640611 | 0.03781186410566183 | 0.8404969985160087 | 0.02465285113204594 | 0.8766661511963456 | 0.01835801213701771 | 0.8968386023294509 | 0.8585858585858586 | 0.8333333333333334 | 0.8457711442786069 | 0.8814021830394627 | [[369, 28], [34, 170]] |
| LinearSVC | Normalized | 0.6396680497925311 | 0.12724229244007448 | 0.5719002790868557 | 0.17060296499376207 | 0.7531442844548927 | 0.28991380948387724 | 0.5653112640891156 | 0.16878833530224618 | 0.6670280803069433 | 0.08612601249207863 | 0.8103161397670549 | 0.8308823529411765 | 0.553921568627451 | 0.6647058823529413 | 0.747993529905665 | [[374, 23], [91, 113]] |
| | Standardized| 0.6396680497925311 | 0.12724229244007448 | 0.5719002790868557 | 0.17060296499376207 | 0.7531442844548927 | 0.28991380948387724 | 0.5653112640891156 | 0.16878833530224618 | 0.6670280803069433 | 0.08612601249207863 | 0.8103161397670549 | 0.8308823529411765 | 0.553921568627451 | 0.6647058823529413 | 0.747993529905665 | [[374, 23], [91, 113]] |
| SVC (Linear) | Normalized | 0.6576434993084371 | 0.001416490093586219 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.5 | 0.0 | 0.6605657237936772 | 0.0 | 0.0 | 0.0 | 0.5 | [[397, 0], [204, 0]] |
| | Standardized | 0.6576434993084371 | 0.001416490093586219 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.5 | 0.0 | 0.6605657237936772 | 0.0 | 0.0 | 0.0 | 0.5 | [[397, 0], [204, 0]] |
| Perceptron | Normalized | 0.5869381051175657 | 0.1532782626832634 | 0.5424396959818047 | 0.17290092302933616 | 0.7031148986188657 | 0.31746538293787024 | 0.5166216267007258 | 0.11741940352196295 | 0.61482036957066 | 0.07747266852928748 | 0.6888519134775375 | 0.84 | 0.10294117647058823 | 0.1834061135371179 | 0.5464328048599794 | [[393, 4], [183, 21]] |
| | Standardized | 0.5869381051175657 | 0.1532782626832634 | 0.5424396959818047 | 0.17290092302933616 | 0.7031148986188657 | 0.31746538293787024 | 0.5166216267007258 | 0.11741940352196295 | 0.61482036957066 | 0.07747266852928748 | 0.6888519134775375 | 0.84 | 0.10294117647058823 | 0.1834061135371179 | 0.5464328048599794 | [[393, 4], [183, 21]] |
| Multilayer Perceptron | Normalized | 0.6584768326417704 | 0.003066947596561596 | 0.1 | 0.30000000000000004 | 0.0024390243902439024 | 0.007317073170731709 | 0.0047619047619047615 | 0.014285714285714285 | 0.501219512195122 | 0.0036585365853658573 | 0.6638935108153078 | 1.0 | 0.00980392156862745 | 0.01941747572815534 | 0.5049019607843137 | [[397, 0], [202, 2]] |
| | Standardized | 0.6584768326417704 | 0.003066947596561596 | 0.1 | 0.30000000000000004 | 0.0024390243902439024 | 0.007317073170731709 | 0.0047619047619047615 | 0.014285714285714285 | 0.501219512195122 | 0.0036585365853658573 | 0.6638935108153078 | 1.0 | 0.00980392156862745 | 0.01941747572815534 | 0.5049019607843137 | [[397, 0], [202, 2]] |
| ANN | Normalized | 0.8125916320885201 | 0.018187147044301306 | 0.7740722666721951 | 0.07639600048705199 | 0.662973846605936 | 0.09265153718002667 | 0.705410204420142 | 0.03544368787170356 | 0.7767118688806508 | 0.02711535539028044 | 0.8432732316227461 | 0.8235294117647058 | 0.6857142857142857 | 0.7483296213808464 | 0.8050420168067227 | [[440, 36], [77, 168]] |
| | Standardized | 0.8050968188105116 | 0.027681631103985953 | 0.7697294221926265 | 0.07957343064229522 | 0.6424037613870116 | 0.11746871345143681 | 0.6885453094621883 | 0.055852798365961576 | 0.766061995181936 | 0.04008998439080089 | 0.8474341192787794 | 0.7824267782426778 | 0.763265306122449 | 0.7727272727272727 | 0.8270108043217287 | [[424, 52], [58, 187]] |
293,717 changes: 293,704 additions & 13 deletions Metabolic Syndrome Prediction/metabolic_syndrome_predict.ipynb
View file
Open in desktop

Large diffs are not rendered by default.

Binary file added Metabolic Syndrome Prediction/normal_model.keras
View file
Open in desktop
Binary file not shown.
Binary file added Metabolic Syndrome Prediction/standard_model.keras
View file
Open in desktop
Binary file not shown.