Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Unit and Integration Testing #155

Open
reluzita opened this issue Jan 30, 2024 · 1 comment
Open

Unit and Integration Testing #155

reluzita opened this issue Jan 30, 2024 · 1 comment
Labels
Base Regards the base package enhancement New feature or request

Comments

@reluzita
Copy link
Collaborator

The aim of this issue is to develop a robust and effective unit and integration testing strategy.

We need to:

  • Ensure comprehensive testing for all features and functionalities
  • Develop integration tests to validate interactions between key components
  • Create documentation to provide clear and concise guidelines for writing and executing tests (include examples and best practices to assist developers in understanding testing procedures).
  • Ensure testing approach aligns with established testing best practices (ex: fairlean uses pytest)
  • Automatically run unit and integration tests with each code change
@reluzita reluzita added the enhancement New feature or request label Jan 30, 2024
@sgpjesus sgpjesus added the Base Regards the base package label Feb 12, 2024
@valmik-patel
Copy link
Contributor

#175

I created this draft PR to add testing to the library using pytest. Currently, the script will run tests defined in the tests folder and output a coverage report. This script can be easily added to GitHub actions to ensure that all tests are passing before a PR is merged.

PS: I had to update the version of one of the libraries because the tests were failing on the previous version.

Current coverage report:

---------- coverage: platform darwin, python 3.9.16-final-0 ----------
Name                                                                        Stmts   Miss  Cover
-----------------------------------------------------------------------------------------------
src/aequitas/__init__.py                                                        3      0   100%
src/aequitas/audit.py                                                         108     84    22%
src/aequitas/bias.py                                                          263    201    24%
src/aequitas/fairness.py                                                      104     22    79%
src/aequitas/flow/__init__.py                                                   2      2     0%
src/aequitas/flow/datasets/__init__.py                                          4      4     0%
src/aequitas/flow/datasets/baf.py                                             107    107     0%
src/aequitas/flow/datasets/dataset.py                                          50     50     0%
src/aequitas/flow/datasets/folktables.py                                      127    127     0%
src/aequitas/flow/datasets/generic.py                                         100    100     0%
src/aequitas/flow/evaluation/__init__.py                                        8      8     0%
src/aequitas/flow/evaluation/fairness.py                                       22     22     0%
src/aequitas/flow/evaluation/performance.py                                     9      9     0%
src/aequitas/flow/experiment/__init__.py                                        3      3     0%
src/aequitas/flow/experiment/_configs.py                                       25     25     0%
src/aequitas/flow/experiment/default.py                                        39     39     0%
src/aequitas/flow/experiment/experiment.py                                    155    155     0%
src/aequitas/flow/methods/__init__.py                                           4      4     0%
src/aequitas/flow/methods/base_estimator/__init__.py                            5      5     0%
src/aequitas/flow/methods/base_estimator/base_estimator.py                     18     18     0%
src/aequitas/flow/methods/base_estimator/lightgbm.py                           26     26     0%
src/aequitas/flow/methods/base_estimator/logistic_regression.py                37     37     0%
src/aequitas/flow/methods/base_estimator/neural_network.py                     84     84     0%
src/aequitas/flow/methods/base_estimator/random_forest.py                      37     37     0%
src/aequitas/flow/methods/inprocessing/__init__.py                              3      3     0%
src/aequitas/flow/methods/inprocessing/fairgbm.py                              16     16     0%
src/aequitas/flow/methods/inprocessing/fairlearn_classifier.py                 53     53     0%
src/aequitas/flow/methods/inprocessing/inprocessing.py                         10     10     0%
src/aequitas/flow/methods/postprocessing/__init__.py                            5      5     0%
src/aequitas/flow/methods/postprocessing/balanced_group_threshold.py           67     67     0%
src/aequitas/flow/methods/postprocessing/group_threshold.py                    23     23     0%
src/aequitas/flow/methods/postprocessing/identity.py                           10     10     0%
src/aequitas/flow/methods/postprocessing/postprocessing.py                     10     10     0%
src/aequitas/flow/methods/postprocessing/threshold.py                          50     50     0%
src/aequitas/flow/methods/preprocessing/__init__.py                             3      3     0%
src/aequitas/flow/methods/preprocessing/correlation_suppression.py             48     48     0%
src/aequitas/flow/methods/preprocessing/data_repairer.py                       43     43     0%
src/aequitas/flow/methods/preprocessing/feature_importance_suppression.py      48     48     0%
src/aequitas/flow/methods/preprocessing/identity.py                            12     12     0%
src/aequitas/flow/methods/preprocessing/label_flipping.py                     104    104     0%
src/aequitas/flow/methods/preprocessing/massaging.py                           45     45     0%
src/aequitas/flow/methods/preprocessing/preprocessing.py                       16     16     0%
src/aequitas/flow/methods/preprocessing/prevalence_sample.py                   81     81     0%
src/aequitas/flow/optimization/__init__.py                                    157    157     0%
src/aequitas/flow/plots/__init__.py                                             0      0   100%
src/aequitas/flow/plots/bootstrap/__init__.py                                   2      2     0%
src/aequitas/flow/plots/bootstrap/plot.py                                      42     42     0%
src/aequitas/flow/plots/bootstrap/visualize.py                                 48     48     0%
src/aequitas/flow/plots/pareto/__init__.py                                      2      2     0%
src/aequitas/flow/plots/pareto/plot.py                                        125    125     0%
src/aequitas/flow/plots/pareto/visualize.py                                    62     62     0%
src/aequitas/flow/utils/__init__.py                                             6      6     0%
src/aequitas/flow/utils/colab.py                                               33     33     0%
src/aequitas/flow/utils/config.py                                              28     28     0%
src/aequitas/flow/utils/evaluation.py                                          88     88     0%
src/aequitas/flow/utils/imports.py                                             16     16     0%
src/aequitas/flow/utils/labeled_frame.py                                       19     19     0%
src/aequitas/flow/utils/logging.py                                             59     59     0%
src/aequitas/flow/utils/metrics.py                                              6      6     0%
src/aequitas/flow/utils/results.py                                             45     45     0%
src/aequitas/group.py                                                         127     44    65%
src/aequitas/plot/__init__.py                                                   5      0   100%
src/aequitas/plot/bubble_concatenation_chart.py                                26     12    54%
src/aequitas/plot/bubble_disparity_chart.py                                   144    117    19%
src/aequitas/plot/bubble_metric_chart.py                                      119     95    20%
src/aequitas/plot/commons/__init__.py                                           0      0   100%
src/aequitas/plot/commons/helpers.py                                           21     12    43%
src/aequitas/plot/commons/initializers.py                                      76     64    16%
src/aequitas/plot/commons/labels.py                                             7      0   100%
src/aequitas/plot/commons/legend.py                                            27     18    33%
src/aequitas/plot/commons/metrics.py                                            2      0   100%
src/aequitas/plot/commons/scales.py                                            33     25    24%
src/aequitas/plot/commons/style/__init__.py                                     0      0   100%
src/aequitas/plot/commons/style/classes.py                                     86      0   100%
src/aequitas/plot/commons/style/color.py                                       19      0   100%
src/aequitas/plot/commons/style/sizes.py                                       39      0   100%
src/aequitas/plot/commons/style/text.py                                         8      0   100%
src/aequitas/plot/commons/tooltips.py                                          18     14    22%
src/aequitas/plot/commons/validators.py                                        69     57    17%
src/aequitas/plot/summary_chart.py                                            131    104    21%
src/aequitas/plot/xy_metrics_chart.py                                          91     73    20%
src/aequitas/plotting.py                                                      653    598     8%
src/aequitas/preprocessing.py                                                  50     23    54%
src/aequitas/squarify_flipped.py                                              119    102    14%
src/aequitas/version.py                                                         1      0   100%
-----------------------------------------------------------------------------------------------
TOTAL                                                                        4596   3912    15%

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Base Regards the base package enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@reluzita @sgpjesus @valmik-patel