Fairness-Definitions-Comparison.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "cc719301",
   "metadata": {},
   "source": [
    "## VARIOUS DEFINITIONS OF FAIRNESS COMPARED"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bad5178c",
   "metadata": {},
   "source": [
    "Here we will compare the aspects of the different popular [definitions of fairness](Fair-Classifier.ipynb) on the [Adult Dataset](https://archive.ics.uci.edu/ml/datasets/adult).  "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b8201de1",
   "metadata": {},
   "source": [
    "### IMPORTING THE NECESSARY LIBRARIES"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fae3b122",
   "metadata": {},
   "source": [
    "We will be using the open-source [AIF360](https://github.com/Trusted-AI/AIF360) package."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 164,
   "id": "dcd9aae8",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "np.set_printoptions(suppress = True)\n",
    "import pandas as pd\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# Importing the Dataset\n",
    "from aif360.datasets import AdultDataset\n",
    "from aif360.algorithms.preprocessing.optim_preproc_helpers.data_preproc_functions import load_preproc_data_adult\n",
    "\n",
    "from aif360.datasets import BinaryLabelDataset\n",
    "from aif360.metrics import BinaryLabelDatasetMetric, ClassificationMetric\n",
    "\n",
    "from aif360.metrics.utils import compute_boolean_conditioning_vector\n",
    "from common_utils import compute_metrics\n",
    "\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "from sklearn.model_selection import train_test_split"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a1a22b9c",
   "metadata": {},
   "source": [
    "### DATASET"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e257cd3f",
   "metadata": {},
   "source": [
    "The following shows the raw data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 117,
   "id": "074ce56f",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:root:Missing Data: 3620 rows removed from AdultDataset.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>age</th>\n",
       "      <th>education-num</th>\n",
       "      <th>race</th>\n",
       "      <th>sex</th>\n",
       "      <th>capital-gain</th>\n",
       "      <th>capital-loss</th>\n",
       "      <th>hours-per-week</th>\n",
       "      <th>income-per-year</th>\n",
       "      <th>workclass=Federal-gov</th>\n",
       "      <th>workclass=Local-gov</th>\n",
       "      <th>workclass=Private</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>25.0</td>\n",
       "      <td>7.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>40.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>38.0</td>\n",
       "      <td>9.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>50.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>28.0</td>\n",
       "      <td>12.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>40.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>44.0</td>\n",
       "      <td>10.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>7688.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>40.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>34.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>30.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>48837</th>\n",
       "      <td>27.0</td>\n",
       "      <td>12.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>38.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>48838</th>\n",
       "      <td>40.0</td>\n",
       "      <td>9.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>40.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>48839</th>\n",
       "      <td>58.0</td>\n",
       "      <td>9.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>40.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>48840</th>\n",
       "      <td>22.0</td>\n",
       "      <td>9.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>20.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>48841</th>\n",
       "      <td>52.0</td>\n",
       "      <td>9.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>15024.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>40.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>45222 rows × 11 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "        age  education-num  race  sex  capital-gain  capital-loss  \\\n",
       "0      25.0            7.0   0.0  1.0           0.0           0.0   \n",
       "1      38.0            9.0   1.0  1.0           0.0           0.0   \n",
       "2      28.0           12.0   1.0  1.0           0.0           0.0   \n",
       "3      44.0           10.0   0.0  1.0        7688.0           0.0   \n",
       "5      34.0            6.0   1.0  1.0           0.0           0.0   \n",
       "...     ...            ...   ...  ...           ...           ...   \n",
       "48837  27.0           12.0   1.0  0.0           0.0           0.0   \n",
       "48838  40.0            9.0   1.0  1.0           0.0           0.0   \n",
       "48839  58.0            9.0   1.0  0.0           0.0           0.0   \n",
       "48840  22.0            9.0   1.0  1.0           0.0           0.0   \n",
       "48841  52.0            9.0   1.0  0.0       15024.0           0.0   \n",
       "\n",
       "       hours-per-week  income-per-year  workclass=Federal-gov  \\\n",
       "0                40.0              0.0                    0.0   \n",
       "1                50.0              0.0                    0.0   \n",
       "2                40.0              1.0                    0.0   \n",
       "3                40.0              1.0                    0.0   \n",
       "5                30.0              0.0                    0.0   \n",
       "...               ...              ...                    ...   \n",
       "48837            38.0              0.0                    0.0   \n",
       "48838            40.0              1.0                    0.0   \n",
       "48839            40.0              0.0                    0.0   \n",
       "48840            20.0              0.0                    0.0   \n",
       "48841            40.0              1.0                    0.0   \n",
       "\n",
       "       workclass=Local-gov  workclass=Private  \n",
       "0                      0.0                1.0  \n",
       "1                      0.0                1.0  \n",
       "2                      1.0                0.0  \n",
       "3                      0.0                1.0  \n",
       "5                      0.0                1.0  \n",
       "...                    ...                ...  \n",
       "48837                  0.0                1.0  \n",
       "48838                  0.0                1.0  \n",
       "48839                  0.0                1.0  \n",
       "48840                  0.0                1.0  \n",
       "48841                  0.0                0.0  \n",
       "\n",
       "[45222 rows x 11 columns]"
      ]
     },
     "execution_count": 117,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_adult = AdultDataset()\n",
    "pd.DataFrame(data_adult.metadata['params']['df']).iloc[:,0:11] # Some columns are skipped to prevent print overflow"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "2ec5ff4b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['age', 'education-num', 'race', 'sex', 'capital-gain', 'capital-loss', 'hours-per-week', 'workclass=Federal-gov', 'workclass=Local-gov', 'workclass=Private', 'workclass=Self-emp-inc', 'workclass=Self-emp-not-inc', 'workclass=State-gov', 'workclass=Without-pay', 'education=10th', 'education=11th', 'education=12th', 'education=1st-4th', 'education=5th-6th', 'education=7th-8th', 'education=9th', 'education=Assoc-acdm', 'education=Assoc-voc', 'education=Bachelors', 'education=Doctorate', 'education=HS-grad', 'education=Masters', 'education=Preschool', 'education=Prof-school', 'education=Some-college', 'marital-status=Divorced', 'marital-status=Married-AF-spouse', 'marital-status=Married-civ-spouse', 'marital-status=Married-spouse-absent', 'marital-status=Never-married', 'marital-status=Separated', 'marital-status=Widowed', 'occupation=Adm-clerical', 'occupation=Armed-Forces', 'occupation=Craft-repair', 'occupation=Exec-managerial', 'occupation=Farming-fishing', 'occupation=Handlers-cleaners', 'occupation=Machine-op-inspct', 'occupation=Other-service', 'occupation=Priv-house-serv', 'occupation=Prof-specialty', 'occupation=Protective-serv', 'occupation=Sales', 'occupation=Tech-support', 'occupation=Transport-moving', 'relationship=Husband', 'relationship=Not-in-family', 'relationship=Other-relative', 'relationship=Own-child', 'relationship=Unmarried', 'relationship=Wife', 'native-country=Cambodia', 'native-country=Canada', 'native-country=China', 'native-country=Columbia', 'native-country=Cuba', 'native-country=Dominican-Republic', 'native-country=Ecuador', 'native-country=El-Salvador', 'native-country=England', 'native-country=France', 'native-country=Germany', 'native-country=Greece', 'native-country=Guatemala', 'native-country=Haiti', 'native-country=Holand-Netherlands', 'native-country=Honduras', 'native-country=Hong', 'native-country=Hungary', 'native-country=India', 'native-country=Iran', 'native-country=Ireland', 'native-country=Italy', 'native-country=Jamaica', 'native-country=Japan', 'native-country=Laos', 'native-country=Mexico', 'native-country=Nicaragua', 'native-country=Outlying-US(Guam-USVI-etc)', 'native-country=Peru', 'native-country=Philippines', 'native-country=Poland', 'native-country=Portugal', 'native-country=Puerto-Rico', 'native-country=Scotland', 'native-country=South', 'native-country=Taiwan', 'native-country=Thailand', 'native-country=Trinadad&Tobago', 'native-country=United-States', 'native-country=Vietnam', 'native-country=Yugoslavia']\n"
     ]
    }
   ],
   "source": [
    "print(data_adult.feature_names, sep=\",\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3cf52071",
   "metadata": {},
   "source": [
    "The above data is not the preprocessed or cleaned data. There are 100 features! AIF360 presents us with a function that loads the cleaned version of the Adult dataset and it is much better to handle it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "59e76efa",
   "metadata": {},
   "outputs": [],
   "source": [
    "priv_group = [{'sex':1}]\n",
    "unpriv_group = [{'sex':0}]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "61829850",
   "metadata": {},
   "source": [
    "We will address the problem of fairness for gender here. So, we choose the protected group as sex with the males indicated by 1 as the privileged group and the females indicated by 0 as the unprivileged group as shown above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 118,
   "id": "ecb4b423",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>race</th>\n",
       "      <th>sex</th>\n",
       "      <th>Income Binary</th>\n",
       "      <th>Age (decade)=10</th>\n",
       "      <th>Age (decade)=20</th>\n",
       "      <th>Age (decade)=30</th>\n",
       "      <th>Age (decade)=40</th>\n",
       "      <th>Age (decade)=50</th>\n",
       "      <th>Age (decade)=60</th>\n",
       "      <th>Age (decade)=&gt;=70</th>\n",
       "      <th>Education Years=6</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>48837</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>48838</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>48839</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>48840</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>48841</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>48842 rows × 11 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       race  sex  Income Binary  Age (decade)=10  Age (decade)=20  \\\n",
       "0       0.0  1.0            0.0              0.0              1.0   \n",
       "1       1.0  1.0            0.0              0.0              0.0   \n",
       "2       1.0  1.0            1.0              0.0              1.0   \n",
       "3       0.0  1.0            1.0              0.0              0.0   \n",
       "4       1.0  0.0            0.0              1.0              0.0   \n",
       "...     ...  ...            ...              ...              ...   \n",
       "48837   1.0  0.0            0.0              0.0              1.0   \n",
       "48838   1.0  1.0            1.0              0.0              0.0   \n",
       "48839   1.0  0.0            0.0              0.0              0.0   \n",
       "48840   1.0  1.0            0.0              0.0              1.0   \n",
       "48841   1.0  0.0            1.0              0.0              0.0   \n",
       "\n",
       "       Age (decade)=30  Age (decade)=40  Age (decade)=50  Age (decade)=60  \\\n",
       "0                  0.0              0.0              0.0              0.0   \n",
       "1                  1.0              0.0              0.0              0.0   \n",
       "2                  0.0              0.0              0.0              0.0   \n",
       "3                  0.0              1.0              0.0              0.0   \n",
       "4                  0.0              0.0              0.0              0.0   \n",
       "...                ...              ...              ...              ...   \n",
       "48837              0.0              0.0              0.0              0.0   \n",
       "48838              0.0              1.0              0.0              0.0   \n",
       "48839              0.0              0.0              1.0              0.0   \n",
       "48840              0.0              0.0              0.0              0.0   \n",
       "48841              0.0              0.0              1.0              0.0   \n",
       "\n",
       "       Age (decade)=>=70  Education Years=6  \n",
       "0                    0.0                0.0  \n",
       "1                    0.0                0.0  \n",
       "2                    0.0                0.0  \n",
       "3                    0.0                0.0  \n",
       "4                    0.0                0.0  \n",
       "...                  ...                ...  \n",
       "48837                0.0                0.0  \n",
       "48838                0.0                0.0  \n",
       "48839                0.0                0.0  \n",
       "48840                0.0                0.0  \n",
       "48841                0.0                0.0  \n",
       "\n",
       "[48842 rows x 11 columns]"
      ]
     },
     "execution_count": 118,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_adult = load_preproc_data_adult(['sex'])\n",
    "pd.DataFrame(data_adult.metadata['params']['df']).iloc[:,0:11] # Some columns are skipped to prevent print overflow"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "3a321aa7",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['race', 'sex', 'Age (decade)=10', 'Age (decade)=20', 'Age (decade)=30', 'Age (decade)=40', 'Age (decade)=50', 'Age (decade)=60', 'Age (decade)=>=70', 'Education Years=6', 'Education Years=7', 'Education Years=8', 'Education Years=9', 'Education Years=10', 'Education Years=11', 'Education Years=12', 'Education Years=<6', 'Education Years=>12']\n"
     ]
    }
   ],
   "source": [
    "print(data_adult.feature_names, sep=\", \")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "af182ce8",
   "metadata": {},
   "source": [
    "The number of features in this cleaned pre-processed dataset is far less than the number of features in the original dataset. It is more managable."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "202e3cfa",
   "metadata": {},
   "source": [
    "### EVALUATION OF FAIRNESS"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4c56027a",
   "metadata": {},
   "source": [
    "Here, we will try to find out how fair is the data with respect to the protected group i.e. sex as per the various definitions of fairness."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8ba214b0",
   "metadata": {},
   "source": [
    "#### CLEANED DATA FAIRNESS"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "99dddc05",
   "metadata": {},
   "source": [
    "We look in the following histogram the proportion of the protected privileged and unprivileged groups proportion along with the class labels. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "id": "ce84c09a",
   "metadata": {},
   "outputs": [],
   "source": [
    "def plot_prop_prtced_grp_lab(data, title=\"\", figsize=(10,8)):\n",
    "    feature_idx = np.where(np.array(data.feature_names) == data.protected_attribute_names[0])[0][0]\n",
    "    n_p_fav = len(np.where(np.logical_and(data.features[:,feature_idx] == data.privileged_protected_attributes[0],data.labels == data.favorable_label) == True)[0])\n",
    "    n_unp_fav = len(np.where(np.logical_and(data.features[:,feature_idx] == data.unprivileged_protected_attributes[0],data.labels == data.favorable_label) == True)[0])\n",
    "    n_p_unfav = len(np.where(np.logical_and(data.features[:,feature_idx] == data.privileged_protected_attributes[0],data.labels == data.unfavorable_label) == True)[0])\n",
    "    n_unp_unfav = len(np.where(np.logical_and(data.features[:,feature_idx] == data.unprivileged_protected_attributes[0],data.labels == data.unfavorable_label) == True)[0])\n",
    "    plt.figure(figsize=figsize)\n",
    "    xlocs = [i for i in range(4)]\n",
    "    y = [n_p_fav/(n_p_fav+n_p_unfav), n_unp_fav/(n_unp_fav+n_unp_unfav), n_p_unfav/(n_p_fav+n_p_unfav), n_unp_unfav/(n_unp_fav+n_unp_unfav)]\n",
    "    plt.bar(['priv grp,fav label','unpriv grp,fav label', 'priv grp,unfav label', 'unpriv grp,unfav label'], y)\n",
    "    for i, v in enumerate(y):\n",
    "        plt.text(xlocs[i] - 0.05, v + 0.01, str(round(v,2)))\n",
    "    plt.title(title)\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cc84cb11",
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_prop_prtced_grp_lab(data_adult, \"Proportion of Groups and Labels on Entire Data\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "e6dd7e14",
   "metadata": {},
   "outputs": [],
   "source": [
    "ad_metrics = BinaryLabelDatasetMetric(data_adult,\n",
    "                                      unprivileged_groups=unpriv_group,\n",
    "                                     privileged_groups=priv_group)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "d65c5c87",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Statistical Parity Difference: -0.19451574596420296\n"
     ]
    }
   ],
   "source": [
    "print(\"Statistical Parity Difference:\",ad_metrics.statistical_parity_difference())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "64454055",
   "metadata": {},
   "source": [
    "We observe quite a bit of deviation from 0 which is in favor of the male group. Hence, data is not fair as per this definition. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "da7981f9",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Disparate Impact: 0.3596552625800337\n"
     ]
    }
   ],
   "source": [
    "print(\"Disparate Impact:\",ad_metrics.disparate_impact())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "33dd3b82",
   "metadata": {},
   "source": [
    "We see that this value is clearly not close to 1 rather it is close to 0 again showing that the male individuals are favored in the data compared to females."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "642051cb",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Consistency: 0.7566602514229388\n"
     ]
    }
   ],
   "source": [
    "print(\"Consistency:\",ad_metrics.consistency()[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ca72164b",
   "metadata": {},
   "source": [
    "The value is not very close to 1, so somewhat the individual fairness is also not exhibited properly by the data."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0057c5ee",
   "metadata": {},
   "source": [
    "#### FAIRNESS OF TEST DATA"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eb2b0e21",
   "metadata": {},
   "source": [
    "To train our model we will split our data into 2 parts i.e. a training data and a test data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "171e7850",
   "metadata": {},
   "outputs": [],
   "source": [
    "dset_raw_trn, dset_raw_tst = data_adult.split([0.7], shuffle=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3b4cc059",
   "metadata": {},
   "source": [
    "Taking a look at the fairness metrics for the test data and the proportion of protected group and labels, we get:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "id": "f120ca25",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlMAAAHiCAYAAADMP0mlAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAqGklEQVR4nO3de7hdVX3u8e8LKSqiohJsSYIBA0rwAp5AtbYWLyBoBbWgwd5QC/ZR1OMFpa2mFE8VK9W2Aq1YOVgvBLC2pBJBrYJIVRIuRyWIRG5JtBgQtLEqBn7njzk3rGzWzt7J2El2wvfzPPvJmnOONeaYc40997vGHFkrVYUkSZI2znZbugGSJElbM8OUJElSA8OUJElSA8OUJElSA8OUJElSA8OUJElSA8OUtIkl+b0kn98C+31WkhuSrEnyks29/61FkkuS/PEk13lMkq9u7udK2jIMU9oqJbk5yc/6oHBbkrOT7DQF2jU7SSWZNrKuqj5ZVYdsgeacDJxWVTtV1b8NK5BkfpJvJPlpkh/2j1+XJJu3qVNTkpOSfGJLt2NzSbJ7/zs18lN93xhZ/q2NqPPmJM9fz/aDktw7sI+VSc5LcsAG7ONB9Tpp6jFMaWv24qraCXg6MA945+gCg6FmU9uc+5qgxwPXjrUxyVuBvwPeD/wq8DjgT4BnATuM8ZztJ7+Zmiqq6tY+fO/U/24BPG1g3WWbaNff7/f3COAZwHeAy5I8bxPtT5pUhilt9apqFfA54MkA/bvp1ye5AbihX3dskuVJfpRkUZLdRp7fl39jkhuT3J7k/Um267dtl+SdSW7pR27+Ocmj+m0jo1CvSXIr8CXgK321d/Xvsp85+rZNkt9IsiTJj/t/f2Ng2yVJ3p3k8iT/neTzSXYZ69jHOq4k3wP2BP69b8dDRj3vUXQjV6+rqk9X1X9X5+qq+r2q+kVf7uwk/5BkcZKfAs9Jsk/fzruSXJvk8FHt/+OB5dHHvr5zPSfJpf15uT3Jues57vOT/Fdf9itJ9h3YdnaS05Nc2J/DbyR5wsD2g5N8p3/uacBGjcIlOTHJ9/p9LEvy0gcWyWn9fr4zGAySPCrJR5P8IMmqJP9nWFBN54N93/tJkm8lefIY7dmt7wM/6vvEsQPbTko32vPPfXuvTTJvA4/3IUlOTXJrutHgf0zysH7bLkk+2/eJHyW5LN3vzseB3bm/H759ffvo++DKqloA/BPwvoH9/12SFf15uDL9KFmSQ4E/A17R7+P/9etfleS6/nhvTPLaDTleaUMYprTVSzILeCFw9cDqlwC/DsxN8lzgvcDLgV8DbgEWjqrmpXSjW08HjgBe3a8/pv95Dl042Qk4bdRzfxvYB3gB8Ox+3c79O/mvjWrrY4ALgb8HHgt8ALgwyWMHir0SeBWwK90I0dvGOO4xj6uqngDcSj96NxKOBjwTeAhwwbC6R3kl8Fd0owbfAP4d+HzfvjcAn0zyxAnUM2Ksc/3uvt5HAzOBD62njs8Be/VtuAr45Kjt84G/7Ota3refPph+hm4Ucxfge3QjcRvje8BvAY/q9/WJJL82sP3X+zK7AH8BfKZ//QHOBtYCc4D9gUOAYfO2DqHrU3v3+3k5cMcY7VkIrAR2A44E3tP3kRGH92V2BhbxwH48nlP6duzXt3sGsKDf9tZ+39PpRjj/jC4b/QHr9sO/3oD9fQZ4epKH98tL+n0/BvgUcH6Sh1bVRcB7gHP7fTytL/9D4HeAR9L9Pn0wydM38JilCTFMaWv2b0nuAr4KXEp3QR3x3qr6UVX9DPg94KyquqoPFX8KPDPJ7IHy7+vL3wr8LXB0v/73gA9U1Y1VtaZ/7vyse0vvpKr6ab+v8bwIuKGqPl5Va6vqHLpbGi8eKPN/q+q7fX3n0f0BGWYixzWWXYDbq2rtyIok/9mPLPwsybMHyl5QVZdX1b19W3YCTqmqu6vqS8Bnuf98TcRY5/qXdLcmd6uqn1fVmJOwq+qsfjTtF8BJwNPSjxj2/rWqruiP75Pcfw5fCFzbj8b9st//f21A2wfbcH5Vfb+q7q2qc+lGQQ8cKPJD4G+r6pf99uuBFyV5XN+O/933mx8CH6QLgKP9ki7EPglIVV1XVT8YXah/Q/Es4B39ubuGbmTnDweKfbWqFlfVPcDHgaeNrmcsSQIcB7y5f+3+m+73baTNv6QL9I/vj/eyav/i1+/TjRruDFBVn6iqO/rfm7+hezMwZoivqgur6nv9aNeldEF9g+d8SRNhmNLW7CVVtXNVPb6qXjcqzKwYeLwb3agNAH0ouoPunfWw8rf0z3nAc/vH0+jefQ977nhG1zdS52BbBv+4/w9deBm3rjGOayx3ALtk3Ynyv1FVO/fbBq8No8/lij5YjdX+8Yx1rt9O98fziv421Ksf8Ey6eVtJTulvsf0EuLnfNHg7dKxzuNvg/vs/+Bvy+g224w+TXNMH0LvobjMPtmHVqEAxcqyPB34F+MHAcz9MN8q2jj6sngacDvwwyZlJHjmkObsBIyFncH/r61cPzcTn+U0HdgSuHGjzRf166ObdLQc+399SO3GC9a7PDKCAuwCSvK2/bffjfv+PYt3zvY4khyX5en/b8S66ADtmeamFYUrbqsE/Yt+n+wMGQH/b4LHAqoEyswYe794/5wHP7betBW4bY1/jvRsfXd9InauGlB3PRI5rLF8DfkF3m208o8/lrPTznHqD7f8p3R/dEb86pL6h57qq/quqjq2q3YDXAmckmTPk+a/s2/18uj+os/v1E5n79IPB/fcjLrPGLj5ckscDHwGOBx7bh9Bvj2rDjL7+ESPHuoLu3O/SvxnYuaoeWVX7MkRV/X1V/S9gLt1tthOGFPs+8Jgkjxi1v43pV8PcDvwM2HegzY/qJ43TjxK+tar2pLud+JaBOWIbO0L1UuCqqvppPz/q7XS3OR/dn+8fc//5Xmcf6eYI/gtwKvC4vvxiJtZHpA1mmNKDwTnAq5Ls119k3wN8o6puHihzQpJH97dL3gScO/DcNyfZI91HL4zMzVjLcKuBe+nmVw2zGNg7ySuTTEvyCro/kp/dRMc1VFXdRTfP54wkRyZ5RD9heD/g4et56jfoRjXenuRXkhxEd4tyZA7aNcDLkuzYB6HXDKlj6LlOclSSmX2ZO+n+QN475PmPoAsjd9AFt/cMKTOWC4F9k7ysH5V5I8MD36Dtkjx04OchdOeo6F5vkryK/j9ADNgVeGN/no6im1e3uL9N93ngb5I8sj/vT0jy26N3nOSAJL+e5FfogurPGXJOqmoF8J/Ae/s2PpXu3E/KxwX0I5EfoZt3tGvfthlJXtA//p10/4EgdCHnnoF23sbYvw/rSGdGkr+gm0P2Z/2mR9C9iVkNTEuygG4u1IjbgNkDIX8HutuAq4G1SQ6jm38mbRKGKW3zquqLwLvo3qn+AHgCD5yfcgFwJV0YuBD4aL/+LLr5JV8BbqL7Y/aG9ezrf+gmO1/e3w55xqjtd9BNin0rXRh4O/A7VXX7Jjqu9T3/r4G39G24rf/5MPAOuj/Mw55zN114OoxutOIM4A+r6jt9kQ8Cd/d1fYwHTgyHsc/1AcA3kqyhmyD9pqq6ccjz/5nuFtYqYBnw9Q045tuBo+gmU99BN4n98nGedjTdqMzIz/eqahnwN3QjfLcBTxlSzzf6+m+n6xNH9q8/dHOZdujbfyfwabo5R6M9ki7E3El3zHfQ3VIbq52z6Uap/hX4i76PTJZ30N3K+3p/e/WL3D9naa9+eQ3dOTmjqr7cb3sv8M7+92Hof6YAdutf9zV0E82fAhxUVSMfdnsx3W3F79Kdh5+z7u3Z8/t/70hyVX+78410cw7vpBvNXLTRRy6NI+1zBKWtW5IC9qqq5Vu6Lds6z7WkbZEjU5IkSQ0MU5IkSQ28zSdJktTAkSlJkqQGhilJkqQGW+xb7nfZZZeaPXv2ltq9JEnShF155ZW3V9X0Ydu2WJiaPXs2S5cu3VK7lyRJmrAko78K7D7e5pMkSWpgmJIkSWpgmJIkSWpgmJIkSWpgmJIkSWpgmJKkbcBFF13EE5/4RObMmcMpp5zygO1vfvOb2W+//dhvv/3Ye++92Xnnne/bduutt3LIIYewzz77MHfuXG6++ebN13BtMvaJzWeLfZ3MvHnzyo9GkKR299xzD3vvvTdf+MIXmDlzJgcccADnnHMOc+fOHVr+Qx/6EFdffTVnnXUWAAcddBB//ud/zsEHH8yaNWvYbrvt2HHHHTfnIWiS2ScmX5Irq2resG2OTEnSVu6KK65gzpw57Lnnnuywww7Mnz+fCy64YMzy55xzDkcffTQAy5YtY+3atRx88MEA7LTTTg/6P5rbAvvE5mWYkqSt3KpVq5g1a9Z9yzNnzmTVqlVDy95yyy3cdNNNPPe5zwXgu9/9LjvvvDMve9nL2H///TnhhBO45557Nku7tenYJzYvw5QkPYgsXLiQI488ku233x6AtWvXctlll3HqqaeyZMkSbrzxRs4+++wt20htVvaJdoYpSdrKzZgxgxUrVty3vHLlSmbMmDG07MKFC++7nQPdiMV+++3HnnvuybRp03jJS17CVVddtcnbrE3LPrF5GaYkaSt3wAEHcMMNN3DTTTdx9913s3DhQg4//PAHlPvOd77DnXfeyTOf+cx1nnvXXXexevVqAL70pS+NOUlZWw/7xOZlmJKkrdy0adM47bTTeMELXsA+++zDy1/+cvbdd18WLFjAokWL7iu3cOFC5s+fT5L71m2//faceuqpPO95z+MpT3kKVcWxxx67JQ5Dk8g+sXn50QiSJEnj8KMRJEmSNhHDlCRJUgPDlCRJUgPDlCRJUgPDlCRJUoNpW7oBkrSlzT7xwi3dBE3Qzae8aLPty36x9dic/WIYR6YkSZIaGKYkSZIaGKYkSZIaGKYkSZIaGKYkSZIaGKYkSZIaGKYkSZIaGKYkSZIaGKYkSZIaGKYkSZIaGKYkSZIaGKYkSZIaGKYkSZIaGKYkSZIaGKYkSZIaGKYkSZIaGKYkSZIaGKYkSZIaGKYkSZIaGKYkSZIaGKYkSZIaGKYkSZIaGKYkSZIaGKYkSZIaGKYkSZIaGKYkSZIaGKYkSZIaTChMJTk0yfVJlic5ccj2Dya5pv/5bpK7Jr2lkiRJU9C08Qok2R44HTgYWAksSbKoqpaNlKmqNw+UfwOw/yZoqyRJ0pQzkZGpA4HlVXVjVd0NLASOWE/5o4FzJqNxkiRJU91EwtQMYMXA8sp+3QMkeTywB/Cl9qZJkiRNfZM9AX0+8OmqumfYxiTHJVmaZOnq1asnedeSJEmb30TC1Cpg1sDyzH7dMPNZzy2+qjqzquZV1bzp06dPvJWSJElT1ETC1BJgryR7JNmBLjAtGl0oyZOARwNfm9wmSpIkTV3jhqmqWgscD1wMXAecV1XXJjk5yeEDRecDC6uqNk1TJUmSpp5xPxoBoKoWA4tHrVswavmkyWuWJEnS1sFPQJckSWpgmJIkSWpgmJIkSWpgmJIkSWpgmJIkSWpgmJIkSWpgmJIkSWpgmJIkSWpgmJIkSWpgmJIkSWpgmJIkSWpgmJIkSWpgmJIkSWpgmJIkSWpgmJIkSWpgmJIkSWpgmJIkSWpgmJIkSWpgmJIkSWpgmJIkSWpgmJIkSWpgmJIkSWpgmJIkSWpgmJIkSWpgmJIkSWpgmJIkSWpgmJIkSWpgmJIkSWpgmJIkSWpgmJIkSWpgmJIkSWpgmJIkSWpgmJIkSWpgmJIkSWpgmJIkSWpgmJIkSWpgmJIkSWpgmJIkSWpgmJIkSWpgmJIkSWpgmJIkSWpgmJIkSWpgmJIkSWpgmJIkSWpgmJIkSWowoTCV5NAk1ydZnuTEMcq8PMmyJNcm+dTkNlOSJGlqmjZegSTbA6cDBwMrgSVJFlXVsoEyewF/Cjyrqu5MsuumarAkSdJUMpGRqQOB5VV1Y1XdDSwEjhhV5ljg9Kq6E6Cqfji5zZQkSZqaJhKmZgArBpZX9usG7Q3sneTyJF9PcuhkNVCSJGkqG/c23wbUsxdwEDAT+EqSp1TVXYOFkhwHHAew++67T9KuJUmStpyJjEytAmYNLM/s1w1aCSyqql9W1U3Ad+nC1Tqq6syqmldV86ZPn76xbZYkSZoyJhKmlgB7JdkjyQ7AfGDRqDL/RjcqRZJd6G773Th5zZQkSZqaxg1TVbUWOB64GLgOOK+qrk1ycpLD+2IXA3ckWQZ8GTihqu7YVI2WJEmaKiY0Z6qqFgOLR61bMPC4gLf0P5IkSQ8afgK6JElSA8OUJElSA8OUJElSA8OUJElSA8OUJElSA8OUJElSA8OUJElSA8OUJElSA8OUJElSA8OUJElSA8OUJElSA8OUJElSA8OUJElSA8OUJElSA8OUJElSA8OUJElSA8OUJElSA8OUJElSA8OUJElSA8OUJElSA8OUJElSA8OUJElSA8OUJElSA8OUJElSA8OUJElSA8OUJElSA8OUJElSA8OUJElSA8OUJElSA8OUJElSA8OUJElSA8OUJElSA8OUJElSA8OUJElSA8OUJElSA8OUJElSA8OUJElSA8OUJElSA8OUJElSA8OUJElSA8OUJElSA8OUJElSA8OUJElSA8OUJElSA8OUJElSA8OUJElSgwmFqSSHJrk+yfIkJw7ZfkyS1Umu6X/+ePKbKkmSNPVMG69Aku2B04GDgZXAkiSLqmrZqKLnVtXxm6CNkiRJU9ZERqYOBJZX1Y1VdTewEDhi0zZLkiRp6zCRMDUDWDGwvLJfN9rvJvlmkk8nmTUprZMkSZriJmsC+r8Ds6vqqcAXgI8NK5TkuCRLkyxdvXr1JO1akiRpy5lImFoFDI40zezX3aeq7qiqX/SL/wT8r2EVVdWZVTWvquZNnz59Y9orSZI0pUwkTC0B9kqyR5IdgPnAosECSX5tYPFw4LrJa6IkSdLUNe7/5quqtUmOBy4GtgfOqqprk5wMLK2qRcAbkxwOrAV+BByzCdssSZI0ZYwbpgCqajGweNS6BQOP/xT408ltmiRJ0tTnJ6BLkiQ1MExJkiQ1MExJkiQ1MExJkiQ1MExJkiQ1MExJkiQ1MExJkiQ1MExJkiQ1MExJkiQ1MExJkiQ1MExJkiQ1MExJkiQ1MExJkiQ1MExJkiQ1MExJkiQ1MExJkiQ1MExJkiQ1MExJkiQ1MExJkiQ1MExJkiQ1MExJkiQ1MExJkiQ1MExJkiQ1MExJkiQ1MExJkiQ1MExJkiQ1MExJkiQ1MExJkiQ1MExJkiQ1MExJkiQ1MExJkiQ1MExJkiQ1MExJkiQ1MExJkiQ1MExJkiQ1MExJkiQ1MExJkiQ1MExJkiQ1MExJkiQ1MExJkiQ1MExJkiQ1MExJkiQ1MExJkiQ1MExJkiQ1MExJkiQ1MExJkiQ1mFCYSnJokuuTLE9y4nrK/W6SSjJv8pooSZI0dY0bppJsD5wOHAbMBY5OMndIuUcAbwK+MdmNlCRJmqomMjJ1ILC8qm6sqruBhcARQ8q9G3gf8PNJbJ8kSdKUNpEwNQNYMbC8sl93nyRPB2ZV1YXrqyjJcUmWJlm6evXqDW6sJEnSVNM8AT3JdsAHgLeOV7aqzqyqeVU1b/r06a27liRJ2uImEqZWAbMGlmf260Y8AngycEmSm4FnAIuchC5Jkh4MJhKmlgB7JdkjyQ7AfGDRyMaq+nFV7VJVs6tqNvB14PCqWrpJWixJkjSFjBumqmotcDxwMXAdcF5VXZvk5CSHb+oGSpIkTWXTJlKoqhYDi0etWzBG2YPamyVJkrR18BPQJUmSGhimJEmSGhimJEmSGhimJEmSGhimJEmSGhimJEmSGhimJEmSGhimJEmSGhimJEmSGhimJEmSGhimJEmSGhimJEmSGhimJEmSGhimJEmSGhimJEmSGhimJEmSGhimJEmSGhimJEmSGhimJEmSGhimJEmSGhimJEmSGhimJEmSGhimJEmSGhimJEmSGhimJEmSGhimJEmSGhimJEmSGhimJEmSGhimJEmSGhimJEmSGhimJEmSGhimJEmSGhimJEmSGhimJEmSGhimJEmSGhimJEmSGhimJEmSGhimJEmSGhimJEmSGhimJEmSGhimJEmSGhimJEmSGhimJEmSGhimJEmSGhimJEmSGkwoTCU5NMn1SZYnOXHI9j9J8q0k1yT5apK5k99USZKkqWfcMJVke+B04DBgLnD0kLD0qap6SlXtB/w18IHJbqgkSdJUNJGRqQOB5VV1Y1XdDSwEjhgsUFU/GVh8OFCT10RJkqSpa9oEyswAVgwsrwR+fXShJK8H3gLsADx3UlonSZI0xU3aBPSqOr2qngC8A3jnsDJJjkuyNMnS1atXT9auJUmStpiJhKlVwKyB5Zn9urEsBF4ybENVnVlV86pq3vTp0yfcSEmSpKlqImFqCbBXkj2S7ADMBxYNFkiy18Dii4AbJq+JkiRJU9e4c6aqam2S44GLge2Bs6rq2iQnA0urahFwfJLnA78E7gT+aFM2WpIkaaqYyAR0qmoxsHjUugUDj980ye2SJEnaKvgJ6JIkSQ0MU5IkSQ0MU5IkSQ0MU5IkSQ0MU5IkSQ0MU5IkSQ0MU5IkSQ0MU5IkSQ0MU5IkSQ0MU5IkSQ0MU5IkSQ0MU5IkSQ0MU5IkSQ0MU5IkSQ0MU1PcRRddxBOf+ETmzJnDKaec8oDtH/jAB5g7dy5PfepTed7znsctt9yyzvaf/OQnzJw5k+OPP35zNVmbmH1CkqYWw9QUds899/D617+ez33ucyxbtoxzzjmHZcuWrVNm//33Z+nSpXzzm9/kyCOP5O1vf/s629/1rnfx7Gc/e3M2W5uQfUKSph7D1BR2xRVXMGfOHPbcc0922GEH5s+fzwUXXLBOmec85znsuOOOADzjGc9g5cqV92278sorue222zjkkEM2a7u16dgnJGnqMUxNYatWrWLWrFn3Lc+cOZNVq1aNWf6jH/0ohx12GAD33nsvb33rWzn11FM3eTu1+dgnJGnqmbalG6DJ8YlPfIKlS5dy6aWXAnDGGWfwwhe+kJkzZ27hlmlLsU9I0uZhmJrCZsyYwYoVK+5bXrlyJTNmzHhAuS9+8Yv81V/9FZdeeikPechDAPja177GZZddxhlnnMGaNWu4++672WmnnYZOWNbWwz4hSVOPYWoKO+CAA7jhhhu46aabmDFjBgsXLuRTn/rUOmWuvvpqXvva13LRRRex66673rf+k5/85H2Pzz77bJYuXeofzW2AfUKSph7nTE1h06ZN47TTTuMFL3gB++yzDy9/+cvZd999WbBgAYsWLQLghBNOYM2aNRx11FHst99+HH744Vu41dqU7BOSNPWkqrbIjufNm1dLly7dIvuWpEGzT7xwSzdBE3TzKS/abPuyX2w9Nke/SHJlVc0bts2RKUmSpAaGKUmSpAaGKUmSpAaGKUmSpAbb9EcjOHlw6+GkUg2zOfuFJG0sR6YkSZIaGKYkSZIaGKYkSZIaGKYkSZIaGKYkSZIaGKYkSZIaGKYkSZIaGKYkSZIaGKYkSZIaGKYkSZIaGKYkSZIaGKYkSZIaGKYkSZIaGKYkSZIaGKYkSZIaGKYkSZIaGKYkSZIaGKYkSZIaTChMJTk0yfVJlic5ccj2tyRZluSbSf4jyeMnv6mSJElTz7hhKsn2wOnAYcBc4Ogkc0cVuxqYV1VPBT4N/PVkN1SSJGkqmsjI1IHA8qq6saruBhYCRwwWqKovV9X/9ItfB2ZObjMlSZKmpomEqRnAioHllf26sbwG+FxLoyRJkrYW0yazsiS/D8wDfnuM7ccBxwHsvvvuk7lrSZKkLWIiI1OrgFkDyzP7detI8nzgz4HDq+oXwyqqqjOral5VzZs+ffrGtFeSJGlKmUiYWgLslWSPJDsA84FFgwWS7A98mC5I/XDymylJkjQ1jRumqmotcDxwMXAdcF5VXZvk5CSH98XeD+wEnJ/kmiSLxqhOkiRpmzKhOVNVtRhYPGrdgoHHz5/kdkmSJG0V/AR0SZKkBoYpSZKkBoYpSZKkBoYpSZKkBoYpSZKkBoYpSZKkBoYpSZKkBoYpSZKkBoYpSZKkBoYpSZKkBoYpSZKkBoYpSZKkBoYpSZKkBoYpSZKkBoYpSZKkBoYpSZKkBoYpSZKkBoYpSZKkBoYpSZKkBoYpSZKkBoYpSZKkBoYpSZKkBoYpSZKkBoYpSZKkBoYpSZKkBoYpSZKkBoYpSZKkBoYpSZKkBoYpSZKkBoYpSZKkBoYpSZKkBoYpSZKkBoYpSZKkBoYpSZKkBoYpSZKkBoYpSZKkBoYpSZKkBoYpSZKkBoYpSZKkBoYpSZKkBoYpSZKkBoYpSZKkBoYpSZKkBoYpSZKkBoYpSZKkBhMKU0kOTXJ9kuVJThyy/dlJrkqyNsmRk99MSZKkqWncMJVke+B04DBgLnB0krmjit0KHAN8arIbKEmSNJVNm0CZA4HlVXUjQJKFwBHAspECVXVzv+3eTdBGSZKkKWsit/lmACsGllf26yRJkh70NusE9CTHJVmaZOnq1as3564lSZI2iYmEqVXArIHlmf26DVZVZ1bVvKqaN3369I2pQpIkaUqZSJhaAuyVZI8kOwDzgUWbtlmSJElbh3HDVFWtBY4HLgauA86rqmuTnJzkcIAkByRZCRwFfDjJtZuy0ZIkSVPFRP43H1W1GFg8at2CgcdL6G7/SZIkPaj4CeiSJEkNDFOSJEkNDFOSJEkNDFOSJEkNDFOSJEkNDFOSJEkNDFOSJEkNDFOSJEkNDFOSJEkNDFOSJEkNDFOSJEkNDFOSJEkNDFOSJEkNDFOSJEkNDFOSJEkNDFOSJEkNDFOSJEkNDFOSJEkNDFOSJEkNDFOSJEkNDFOSJEkNDFOSJEkNDFOSJEkNDFOSJEkNDFOSJEkNDFOSJEkNDFOSJEkNDFOSJEkNDFOSJEkNDFOSJEkNDFOSJEkNDFOSJEkNDFOSJEkNDFOSJEkNDFOSJEkNDFOSJEkNDFOSJEkNDFOSJEkNDFOSJEkNDFOSJEkNDFOSJEkNDFOSJEkNDFOSJEkNDFOSJEkNDFOSJEkNDFOSJEkNJhSmkhya5Poky5OcOGT7Q5Kc22//RpLZk95SSZKkKWjcMJVke+B04DBgLnB0krmjir0GuLOq5gAfBN432Q2VJEmaiiYyMnUgsLyqbqyqu4GFwBGjyhwBfKx//GngeUkyec2UJEmamiYSpmYAKwaWV/brhpapqrXAj4HHTkYDJUmSprJpm3NnSY4DjusX1yS5fnPufxuyC3D7lm7EZIo3hlttc30C7BeTYJvrF/aJZttcn4DN1i8eP9aGiYSpVcCsgeWZ/bphZVYmmQY8CrhjdEVVdSZw5gT2qfVIsrSq5m3pdmjqsE9oGPuFRrNPbBoTuc23BNgryR5JdgDmA4tGlVkE/FH/+EjgS1VVk9dMSZKkqWnckamqWpvkeOBiYHvgrKq6NsnJwNKqWgR8FPh4kuXAj+gClyRJ0jZvQnOmqmoxsHjUugUDj38OHDW5TdN6eKtUo9knNIz9QqPZJzaBeDdOkiRp4/l1MpIkSQ0MUxshyclJnr8F939UkuuSfLmxntlJvj1OmYOSfHYD670kif9bpJdkcZKdt+D+35/k2iTvb6znmCSnjVPmpCRv28B617S0ayrY0teEjWG/2HK29DVhQ/VfGffFJNckeUVjXWcnOXKcMhv0N2Rj/k5Nts36OVPbgiTbD84Xm8R6p/UfeDoRrwGOraqvTnY7NHn6bwFIVb1wE9S9If3lOOAxVXXPZLdDU+aasDHsF5vZFLombKj9Aapqv01U/1bPkaleP0rznSSf7Ed9Pp1kx37bzUnel+Qq4KiRZN1/AfT5A3UMTcdJXtjXfWWSvx8p079b+3iSy+n+N+QxSS7oU/kNSf5iSF0LgN8EPtq/s5yd5LIkV/U/v9GXW5jkRQPPW++7gbHq6T0yyYX9l13/Y5Lt+ucckuRrffnzk+y0gad9Shg9QpfkbUlO6h9f0r/2VyT5bpLf6tcPfa36uq5P8s/At4FZff/ZJckpSV4/sJ+h79aTvKuv46tJzhkp0+/rb5MsBd7Uv6b/mGRp37bfGVLXImAn4Mokr0jy4nRfRn51/07zcUm269u488DzbkjyuPWcswfUM7D5aX2/uCHJsQPPOSHJkiTfTPKX470uW9rWck3on7dm4PGRSc7uH5/d1/+fSW4cuQbYL9ZvG78mrNMnk5yW5Jj+8c1J/jLdNf1bSZ6UZFfgE8AB6UamnpBkQf+afTvJmek8KckVo87ht8Y5zw+oZ2DzH/T7+3aSA/vyD09yVn/ur04y+qvttpyq8qebhD8bKOBZ/fJZwNv6xzcDbx8oezbd52lNA24FHt6v/wfg90fV+1C6r9rZo18+B/hs//gk4ErgYf3yMcAP6L6K52F0v3jzhrT1kpH1wI7AQ/vHe9F9XAXAS4GP9Y936NvwsCHH/O1x6jkI+DmwJ91HY3yhP/ZdgK8MHPs7gAWj27c1/Ayeh375bcBJA8fyN/3jFwJfXN9r1dd1L/CMgfpu7s/X/sClA+uXAbNGteUA4Jq+3zwCuGGgH14CnDGqH15E96ZoL7qvenrokONbM/D40dz/H0/+eODY/g54Vf/410eOc1Q9xwCnjVPPScD/68/JLn2/2w04hO5/EaVv72eBZ49u31T6Yeu6Jgy+xkcCZw+06/z+nM+l+55V+8XEXvtt8ppAd03/7MDyacAxA+16Q//4dcA/jfGcxww8/jjw4v7xNdzfr98BvHPIuT0bOHKcei4BPtI/fjb3/516D/3vE7Az8F3g4aPbtyV+HJla14qqurx//Am6EaAR544uXN2Q6kXAi9N98vuLgAtGFXsScGNV3dQvnzNq+6Kq+tnA8heq6o5+3WdGtWGYXwE+0r8DOJ/uggnwOeA5SR4CHAZ8ZdR+JloPwBXVfdH1PX37fxN4Rl/m8iTX0H1o65gftb+V+0z/75V0F8YRY71Wt1TV10dXUlVXA7sm2S3J04A7q2rFqGLPAi6oqp9X1X8D/z5q++h+eF5V3VtVNwA30vW39ZkJXNy/zicA+w7UOzIXYv6Q/Uy0Hvr2/6yqbge+TPdl6Yf0P1cDV/Xt3GucfUwFW+M1YbR/6/vIMmCsUSX7xYbZlq4JEz22Qc/pRyC/BTyX+1/n87i/v7xiSNsmWg/0vxdV9RW6uyM70/WVE/u/OZfQBczdJ3RUm5hzptY1+nMiBpd/OsZzFgLH031Y6dK+s2+I0fWurw3DvBm4DXga3buRn0P32V9JLgFeQNepF25MPetpU+guHEePU+/WYC3r3vJ+6Kjtv+j/vYd1f2fGeq3G6ivQBdUjgV9l/AvNMK395UPAB6pqUZKD6EYMAL4GzEkyHXgJ8H82sp6x2hTgvVX14XHqnWq2lmvC4Lqx+i90r8Mw9ot1bcvXhI09NgCSPBQ4g26EdEV/+3OkjnOB85N8Bqg+0A01Tj1jHUeA362qdb7Xd323njcXR6bWtXuSZ/aPXwlMZIL3pcDTgWMZHliuB/ZMMrtfHu9/Qhyc5DFJHkZ38bocIMl/JJkxpPyjgB9U1b3AH9DdihtxLvAq4Lfo3i2vz/rqOTDd1wlt17f/q8DXgWclmdO37+FJ9h5nH1PVbXTvDh/bj+Q9YJ7BGIa+VuM4l+4d/pF0F9HRLqcb1Xhoujlo47XlqHRzW55Adyv2+iQzkvzHGOUfxf3frflHIyurGzf/V+ADwHVV9YDv1pxIPb0j+vY/lm74fQndNyi8uj8m+jbuOs4+poKt5ZpwW5J9+t/Rl06gjaPZL9a1LV8TbgHmpvsfejsDz5tAGweNBJ7b+/bcNxe3qr5HF8LexfjBcMx6eq8ASPKbwI+r6sd0/eUNI3Orkuy/gW3fZByZWtf1wOuTnEV37/ofxntCVd2TbjLfMTzw4kFV/SzJ64CLkvyU7gKyPlcA/0I3XP6JqlraXyDn0L3THe0M4F+S/CFdYBp8l/J5uvvQF1TV3ePsd331LKG7rz6Hbnj+X6vq3nSTFs/pLzYA76S7h71Vqapfpvt6pCvo/hB8Z4JPHfZazR5nX9cmeQSwqqp+MGT7knSTg79Jd0H/FvDj9VR5a9+ORwJ/0o9I/hrdu89hTqJ753gn8CVgj4Ft59K91ses7xgmUM836frJLsC7q+r7wPeT7AN8rb8OrgF+H/jhBPa1JW0t14QT6eYbrQaW0k0u3xAnYb+4z7Z8TehHgc6jm9N1E90t1gmrqruSfKR//n/xwP57LvB+1n3tN6aenye5mm4Kyqv7de8G/hb4Zv87cBMTD7qblJ+A3us7/Ger6smboO6dqmpNn6ZPB26oqg8OKXcM3ZDn8aPWPxl4dVW9ZbLbpo0z1ms1SXWP9Jcd6Sb5H1dVVw0pdzZdn/30qPXHA7dW972Z2kheE7QhvCY8uDkytXkcm+SP6P5X3dXABs0PqKpvA140HzzOTDKXbhj8Y8MumutTVev9AEVNCV4TtCG8JkxxjkxJkiQ1cAK6JElSA8OUJElSA8OUJElSA8OUJElSA8OUJElSA8OUJElSg/8P2F7d1YfEpNcAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 720x576 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_prop_prtced_grp_lab(dset_raw_tst, \"Proportion of Groups and Labels on Test Data\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "41768e7f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Statistical Parity Difference: -0.1867972998771832\n",
      "Disparate Impact: 0.3835940007074953\n",
      "Consistency: 0.7237971746400178\n"
     ]
    }
   ],
   "source": [
    "dset_raw_tst_metrics = BinaryLabelDatasetMetric(dset_raw_tst,\n",
    "                                      unprivileged_groups=unpriv_group,\n",
    "                                     privileged_groups=priv_group)\n",
    "print(\"Statistical Parity Difference:\",dset_raw_tst_metrics.statistical_parity_difference())\n",
    "print(\"Disparate Impact:\",dset_raw_tst_metrics.disparate_impact())\n",
    "print(\"Consistency:\",dset_raw_tst_metrics.consistency()[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "80ff2750",
   "metadata": {},
   "source": [
    "##### FITTING MODEL"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "47a4b410",
   "metadata": {},
   "source": [
    "Now, we fit a Logistic Regression Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "fb8981e1",
   "metadata": {},
   "outputs": [],
   "source": [
    "scaler = StandardScaler()\n",
    "dset_trn = scaler.fit_transform(dset_raw_trn.features)\n",
    "y_trn = dset_raw_trn.labels.ravel()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "id": "d64cf9bf",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LogisticRegression()"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lmod= LogisticRegression(penalty='l2')\n",
    "lmod.fit(dset_trn, y_trn)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "id": "41ee7756",
   "metadata": {},
   "outputs": [],
   "source": [
    "dset_tst_pred = dset_raw_tst.copy(deepcopy=True)\n",
    "dset_tst = scaler.transform(dset_tst_pred.features)\n",
    "dset_tst_pred.labels = lmod.predict(dset_tst)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "id": "d8e160d2",
   "metadata": {},
   "outputs": [],
   "source": [
    "dset_tst_prob = lmod.predict_proba(dset_tst)[:,1]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "73725a41",
   "metadata": {},
   "source": [
    "#### FAIRNESS OF MODEL"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b7781414",
   "metadata": {},
   "source": [
    "Taking a look at the fairness metris for the model's predictions on the test data, we get:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "10db3569",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Statistical Parity Difference: -0.21217055052406636\n",
      "Disparate Impact: 0.0\n",
      "Consistency: 0.9998498600969085\n"
     ]
    }
   ],
   "source": [
    "dset_tst_pred_metrics = BinaryLabelDatasetMetric(dset_tst_pred,\n",
    "                                      unprivileged_groups=unpriv_group,\n",
    "                                     privileged_groups=priv_group)\n",
    "print(\"Statistical Parity Difference:\",dset_tst_pred_metrics.statistical_parity_difference())\n",
    "print(\"Disparate Impact:\",dset_tst_pred_metrics.disparate_impact())\n",
    "print(\"Consistency:\",dset_tst_pred_metrics.consistency())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "318c69c5",
   "metadata": {},
   "source": [
    "We see training without any fairness intervention made the fairness issue with the dataset even worse. Except for individual fairness the fairness between classes of protected group is poor. Taking a look at the proportion of unprivileged group individuals who were given the favorable label by the model and privileged group proportion for the same, we see the model exhibits fairness problem severely.It classified no protected privileged group individual to favorable label and classified all unprivileged group individual to unfavorable label."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "id": "6d8086ac",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlMAAAHiCAYAAADMP0mlAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAr10lEQVR4nO3de5xdVX338c8PYkCuEQkqSSDiACXwILQBQYUiKJeooe0TJLReIghaQfuiVqCPNkXUltanUmvwgsUi0CZcqk0qIQhIoKIYwuVBidBEAiYjIiBQ8cIl/J4/9ppwcnImM8mazEwmn/frdV5zzt5rr732Puvs+Z691zknMhNJkiRtmC2GugGSJEmbMsOUJElSBcOUJElSBcOUJElSBcOUJElSBcOUJElSBcOUNjsR8ScR8a0hWO8bImJpRDwdEX8w2OvfVETEwoh43wDXOSMivjPYy66jzoiIf4mIJyJi0UDW3cd6L4mIT/Wz7IMR8eaN3SZpJDBMaUCUA+9vSlB4pBy0txsG7ZoYERkRo3qmZea/ZubRQ9Cc84BZmbldZv5HpwIRMT0ivh8Rv4qIn5f7H4yIGNymDk8RcW5EXD7U7RgAbwTeAozPzIPbZ5YAlxFxQdv048v0SwapnRtVOV703F5oOYY8HRF/sgH1rTOItxwPetbxSER8MyLesh7rGPBwrU2fYUoD6e2ZuR3wu8Bk4OPtBVpDzcY2mOvqp92Be3ubGREfAT4HfAZ4JfAK4APAG4DRvSyz5cA3U4Ngd+DBzPzVOsr8GHhHWz9+D/DfG7Vlg6i8sdiuHDd+QjmGlNu/bsRVjynrfC1wPfCNiJixEdenEc4wpQGXmd3AtcB+AOWd4OkRsRRYWqadGhHLIuIXETEvInbtWb6U/3BEPBARj0XEZyJiizJvi4j4eEQ8VM7cXBoRO5Z5Pe86T4mInwDfBm4p1T5Z3oke2v7OMiJeHxG3R8RT5e/rW+YtjIhPRsStEfHLiPhWROzc27b3tl0R8WNgD+A/Szu2altuR5ozVx/MzKsz85fZuCsz/yQznynlLomIL0bE/Ij4FfCmiNintPPJiLg3Iqa2tf99LY/bt31d+7orIm4u++WxiLhiHdt9VUT8rJS9JSL2bZl3SURcGBHXlH34/Yh4Tcv8t0TEfWXZWcAGnYWLiHMi4sdlHUsi4g/XLhKzynrui4ijWmbsGBEXR8TDEdEdEZ/qFFSjcUHpe/8TET+IiP16ac+upQ/8ovSJU8v0U4B/Bg4tfeETvWzSz4AfAMeU5XYCXg/Ma1vP1PK8P1me731a5h0YEXeWfXIFsHXbsm+LiLvLst+NiP17aUv7tu1YXnuPRvNa/HhLv5kREd+JiP8bzWXM5RFxXH/qbal/i5bn8/GIuLJsPxGxdURcXqY/Gc1r9hUR8WngMGBW2a+z+lpPZv4sMz8HnAv8Xcs2dOxLZd9+iRefuyfL9LdGxF2lT6yIiHPXZ3s1AmSmN2/VN+BB4M3l/gSaMzCfLI+T5t3fTsBLgSOBx2jOYG0FfB64paWuBG4q5XejeSf+vjLvZGAZTTDZDvg6cFmZN7EseymwbVlXz7RRLfXPAL5T7u8EPAG8CxgFnFQev7zMX0hzhmCvUt9C4Pxe9kFf27V6H3VY9ljg+dZ29lLuEuApmrNVWwDbl/3xf2jOXh0J/BLYu6X97+u07f3Y17OBj5X1bA28cR3tOrm0ZSvgH4G729r8OHBw2cf/Cswp83Yu7Z0GvAQ4s+yH9/WynnOBy3uZdwKwa2nvicCvgFe1bPfzpf6XlPlPATuV+d8Avlz6zS7AIuD9HfrLMcAdwBia0LdPzzo6tOcW4Atl3x0APAoc2el56LDsDOA7wB8DV5RpHyxt/BRwSZm2V9nOt5TtOqv0h9Hl9lDLNk8DngM+VZY9EPg58DpgS5qzXg8CW/Wjv14KzC3P+cTSb05paftzwKml3j8FfgrEehxD/gy4DRhP06e+DMwu894P/CewTan/94AdOvX3DuuYSNvxoEzfo0zfp5996Tttyx8B/K9Sfn/gEeAPNvZx19vwuQ15A7yNjFs5ED4NPFkO4F8AXlrmZc8/kfL4YuDvWx5vVw6+E1vKH9sy/4PAjeX+jTRnb3rm7V2WHdVyoNyjZf5aB0/W/Of4LmBR27Z8D5hR7i8EPt7WlgW97IO+tmv1P4sOy74T+FnbtO+W/fkb4PAy7RLg0pYyh9GcwdiiZdps4NyW9vcVpnrb15cCF9GM61mfvjCm1LtjS5v/uWX+FOC+cv/dwG0t8wJYyQaEqQ5l7waOb9nuNf6h0wSmd9FcTn2G0l/LvJOAmzr0lyNpgsMhrfu8w7onAKuA7Vum/S0vhqA1nocOy8+gCVMvpfnHvCNNuHgDa4apvwKubFluC6Cb5p/74R22+bu8GKa+SHnD0zL/fuD319VfaQLMs8CklmnvBxa2tH1Zy7xtSn94ZR/P1+r1AT8CjmqZ9ypefJ2fXLZj/w51LOyt75T5E+kcprYu09/Qz77U63NXyvwjcMH6vG68bdo3L/NpIP1BZo7JzN0z84OZ+ZuWeSta7u9KE7gAyMynac5cjOul/ENlmbWWLfdH0fxD7LRsX9rr66mztS0/a7n/a5qQ1GddvWxXbx4Hdo41B8q/PjPHlHmtr9X2fbkiM19YR/v70tu+Posm3Cwql5FO7rRwRGwZEeeXyyL/Q/NPEZqzTj1624e7tq4/M5P1e/5a2/HulktWT9JcZm5tQ3epv0fPtu5Oc+bm4ZZlv0xzhmoNmfltYBZwIfDziLgoInbo0JxdgV9k5i/b1rc+zwvlNXQNzfjDl2fmrR3W09rnXqDZf+PKvE7b3GN34CM921y2ewIvPv+92Zlmf7W/Dju+ZjLz1+Xu+nwgZXeacUw97foRTTh9BXAZcB0wJyJ+GhF/HxEvWY+6O+lp+y+gX31pDRHxuoi4qVz2fIpmrGOv5TXyGKY0WFoP6D+lOVgCEBHbAi+neUfdY0LL/d3KMmstW+Y9T/PuvdO6Wu930l5fT53dHcr2pT/b1Zvv0ZwdOb4fZdv35YSesR5Fa/t/RXNmoMcrO9TXcV9nM57k1MzclebMwxcioqvD8n9c2v1mmjMoE8v0/ox9erh1/RERbe3pl4jYHfgKcAZN6BgD/LCtDeNK/T16tnUFzb7fubwZGJOZO2TmvnSQmf+Umb8HTKK5zPbRDsV+CuwUEdu3rW9D+tWlwEeATp9ibO9zPfuvm2bfdtrmHiuAT7ds85jM3CYzZ/fRnsdozhK1vw43ZNt6swI4rq1tW2dmd2Y+l5mfyMxJNGPI3kZzhhP6fr335g9pLnne34++1Gkd/0Yzlm1CZu5IM67KT+BuRgxTGgqzgfdGxAHRDMT+G+D7mflgS5mPRsTLImICzfiJK1qWPTMiXh3NVy/8Dc2Ykud7WdejwAs0YyI6mQ/sFRF/HBGjIuJEmn+S39xI29VRZj4JfIImsEyLiO3LINwDaMbx9Ob7NGd6zoqIl0TEEcDbgTll/t3AH0XENiUIndKhjo77OiJOiIjxpcwTNP9EXuiw/PY0YeRxmuD2N31tb4trgH0j4o/KWbkP0znwtdqiDELuuW1Fs4+S5vkmIt5L+QBEi12AD5f9dALNeKf5mfkw8C3gHyJih7LfXxMRv9++4og4qJyFeAlNUP0tHfZJZq6guRT1t6WN+9Ps+w35WoebacZEfb7DvCuBt0bEUaVNH6F5Lr5LE9Cfb9nmP6IZt9bjK8AHyvZERGxbBlJv376Stm1bVdb76dJPdwf+fAO3rTdfKvXvDhARYyPi+HL/TRHxv6L5gMD/0AS7nufgEXp/ra8lmoHrZwB/DfxlObPXV196BBgfEa2fsN2e5kzkbyPiYJo3GNqMGKY06DLzBpqxHv9O8+75NcD0tmJzaQb63k3zD/fiMv2rNKf5bwGW0/wz+9A61vVr4NPAreWU/SFt8x+neWf7EZowcBbwtsx8bCNt17qW/3uaf0pn0RywH6G53HQ2zT/HTss8SxOejqM5Y/AF4N2ZeV8pcgHN+JZHgK/RDP5u19u+Pgj4fkQ8TfOu+88y84EOy19Kc5mnG1hCM7anv9v8GM1g3/Np9v+eQPulrHYn0Ywj67n9ODOXAP9AEyAeoRkM3F7P90v9j9H0iWnl+YfmzMbo0v4ngKtpxum024EmhDxBs82P03yVRW/tnEhz9ugbwF+XPrJesnFjZv6iw7z7acbbfb5s19tpvl7g2dI3/ohmjM8vaAZSf71l2cU0g8Rnle1ZVsr2x4dowuQDNGO7/o3mtTlQPkfT574VEb+k6VOvK/NeSfP8/A/N5b+baY4JPctNi+ZThP+0jvqfjObTsD+gGcN3QmZ+FaAffenbNB+w+VlE9BwnPgicV9o6kyZsajMSa15Ol4ZeRCSwZ2YuG+q2jHTua0mq55kpSZKkCoYpSZKkCl7mkyRJquCZKUmSpAqGKUmSpAqj+i6ycey88845ceLEoVq9JElSv91xxx2PZebYTvOGLExNnDiRxYsXD9XqJUmS+i0i2n96bDUv80mSJFUwTEmSJFUwTEmSJFUwTEmSJFUwTEmSJFUwTEmSJFUwTEmSJFUwTEmSNAKdfPLJ7LLLLuy3334d52cmH/7wh+nq6mL//ffnzjvvHOQWjhyGKUmSRqAZM2awYMGCXudfe+21LF26lKVLl3LRRRfxp3/6p4PYupHFMCVJ0gh0+OGHs9NOO/U6f+7cubz73e8mIjjkkEN48sknefjhhwexhSOHYUqSpM1Qd3c3EyZMWP14/PjxdHd3D2GLNl2GKUmSpAqGKUmSNkPjxo1jxYoVqx+vXLmScePGDWGLNl2GKUmSNkNTp07l0ksvJTO57bbb2HHHHXnVq1411M3aJI0a6gZIkqSBd9JJJ7Fw4UIee+wxxo8fzyc+8Qmee+45AD7wgQ8wZcoU5s+fT1dXF9tssw3/8i//MsQt3nRFZq67QMRXgbcBP8/Mtb6sIiIC+BwwBfg1MCMz+/yyismTJ+fixYs3qNGSJEmDKSLuyMzJneb15zLfJcCx65h/HLBnuZ0GfHF9GyhJkrSp6jNMZeYtwC/WUeR44NJs3AaMiQgvukqSpM3CQAxAHwesaHm8skyTJEka8QZ1AHpEnEZzKZDddtttMFctSdJ6mXjONUPdBPXTg+e/dUjXPxBnprqBCS2Px5dpa8nMizJzcmZOHjt27ACsWpIkaWgNRJiaB7w7GocAT2WmP+4jSZI2C31e5ouI2cARwM4RsRL4a+AlAJn5JWA+zdciLKP5aoT3bqzGSpIkDTd9hqnMPKmP+QmcPmAtkiRJ2oT4czKSJEkVDFOSJEkVDFOSJEkVDFOSJEkVDFOSJEkVDFOSJEkVDFOSJEkVDFOSJEkVDFOSJEkVDFOSJEkVDFOSJEkVDFOSJEkVDFOSJEkVDFOSJEkVDFOSJEkVDFOSJEkVDFOSJEkVDFOSJEkVDFOSJEkVDFOSJEkVDFOSJEkVDFOSJEkVDFOSJEkVDFOSJEkVDFOSJEkVDFOSJEkVDFOSJEkVDFOSJEkVDFOSJEkVDFOSJEkVDFOSJEkVDFOSJEkVDFOSJEkVDFOSJEkVDFOSJEkVDFOSJEkVDFOSJEkVDFOSJEkVDFOSJEkVDFOSJEkVDFOSJEkVDFOSJEkVDFOSJEkVDFOSJEkVDFOSJEkVDFOSJEkVDFOSJEkVDFOSJEkVDFOSJEkVDFOSJEkVDFOSJEkVDFOSJEkVDFOSJEkVDFOSJEkVDFOSJEkVDFOSJEkVDFOSJEkVDFOSJEkVDFOSJEkVDFOSJEkVDFOSJEkVDFOSJEkVDFOSJEkVDFOSJEkVDFOSJEkVDFOSJEkVDFOSJEkVDFOSJEkVDFOSJEkVDFOSJEkVDFOSJEkVDFOSNAIsWLCAvffem66uLs4///y15p955pkccMABHHDAAey1116MGTNm9byzzz6b/fbbj/32248rrrhiEFstjQyjhroBkqQ6q1at4vTTT+f6669n/PjxHHTQQUydOpVJkyatLnPBBResvv/5z3+eu+66C4BrrrmGO++8k7vvvptnnnmGI444guOOO44ddthh0LdD2lR5ZkqSNnGLFi2iq6uLPfbYg9GjRzN9+nTmzp3ba/nZs2dz0kknAbBkyRIOP/xwRo0axbbbbsv+++/PggULBqvp0ojQrzAVEcdGxP0RsSwizukwf7eIuCki7oqIeyJiysA3VZLUSXd3NxMmTFj9ePz48XR3d3cs+9BDD7F8+XKOPPJIAF772teyYMECfv3rX/PYY49x0003sWLFikFptzRS9HmZLyK2BC4E3gKsBG6PiHmZuaSl2MeBKzPzixExCZgPTNwI7ZUkVZgzZw7Tpk1jyy23BODoo4/m9ttv5/Wvfz1jx47l0EMPXT1PUv/058zUwcCyzHwgM58F5gDHt5VJoOcC+47ATweuiZKkdRk3btwaZ5NWrlzJuHHjOpadM2fO6kt8PT72sY9x9913c/3115OZ7LXXXhu1vdJI058wNQ5oPee7skxrdS7wzohYSXNW6kOdKoqI0yJicUQsfvTRRzeguZKkdgcddBBLly5l+fLlPPvss8yZM4epU6euVe6+++7jiSee4NBDD109bdWqVTz++OMA3HPPPdxzzz0cffTRg9Z2aSQYqE/znQRckpn/EBGHApdFxH6Z+UJrocy8CLgIYPLkyTlA65akzdqoUaOYNWsWxxxzDKtWreLkk09m3333ZebMmUyePHl1sJozZw7Tp08nIlYv+9xzz3HYYYcBsMMOO3D55ZczapQf9JbWR2SuO9OUcHRuZh5THv8lQGb+bUuZe4FjM3NFefwAcEhm/ry3eidPnpyLFy+u3wJJkjaCiedcM9RNUD89eP5bN/o6IuKOzJzcaV5/LvPdDuwZEa+OiNHAdGBeW5mfAEeVle0DbA14HU+SJI14fYapzHweOAO4DvgRzaf27o2I8yKi56L8R4BTI+L/AbOBGdnXKS9JkqQRoF8XxjNzPs3A8tZpM1vuLwHeMLBNkyRJGv4cZShps+fYmE3HYIyNkdaXPycjSZJUwTAlSZJUwTAlSZJUwTAlSZJUwTAlSZJUwTAlSZJUwTAlSZJUwTAlSZJUwTAlSZJUwTAlSZJUwTAlSZJUwTAlSZJUwTAlSZJUwTAlSZJUwTAlSZJUwTAlSZJUwTAlSZJUwTAlSZJUwTAlSZJUwTAlSZJUwTAlSZJUwTAlSZJUwTAlSZJUwTAlSZJUwTAlSZJUwTAlSZJUwTAlSZJUwTAlSZJUwTAlSZJUwTAlSZJUwTAlSZJUwTAlSZJUwTAlSZJUwTAlSZJUwTAlSZJUwTAlSZJUwTAlSZJUwTAlSZJUwTAlSZJUwTAlSZJUwTAlSZJUwTAlSZJUwTAlSZJUwTAlSZJUwTAlSZJUwTAlSZJUwTAlSZJUwTAlSZJUwTAlSZJUwTAlSZJUwTAlSZJUwTAlSZJUwTAlSZJUwTAlSZJUwTAlSZJUwTAlSZJUwTAlSZJUwTAlSZJUwTAlSZJUwTAlSZJUwTAlSZJUwTAlSZJUwTAlSZJUwTAlSZJUwTAlSZJUwTAlSZJUwTAlSZJUwTAlSZJUwTAlSZJUwTAlSZJUwTAlSZJUwTAlSZJUwTAlSZJUoV9hKiKOjYj7I2JZRJzTS5l3RMSSiLg3Iv5tYJspSZI0PI3qq0BEbAlcCLwFWAncHhHzMnNJS5k9gb8E3pCZT0TELhurwZIkScNJf85MHQwsy8wHMvNZYA5wfFuZU4ELM/MJgMz8+cA2U5IkaXjqT5gaB6xoebyyTGu1F7BXRNwaEbdFxLGdKoqI0yJicUQsfvTRRzesxZIkScPIQA1AHwXsCRwBnAR8JSLGtBfKzIsyc3JmTh47duwArVqSJGno9CdMdQMTWh6PL9NarQTmZeZzmbkc+G+acCVJkjSi9SdM3Q7sGRGvjojRwHRgXluZ/6A5K0VE7Exz2e+BgWumJEnS8NRnmMrM54EzgOuAHwFXZua9EXFeREwtxa4DHo+IJcBNwEcz8/GN1WhJkqThos+vRgDIzPnA/LZpM1vuJ/Dn5SZJkrTZ8BvQJUmSKhimJEmSKhimJEmSKhimJEmSKhimJEmSKhimJEmSKhimJEmSKhimJEmSKhimJEmSKhimJEmSKhimJEmSKhimJEmSKhimJEmSKhimJEmSKhimJEmSKhimJEmSKhimJEmSKhimJEmSKhimJEmSKhimJEmSKhimJEmSKhimJEmSKhimJEmSKhimJEmSKhimJEmSKhimJEmSKhimJEmSKhimJEmSKhimJEmSKhimJEmSKhimJEmSKhimJEmSKhimJEmSKhimJEmSKhimJEmSKhimJEmSKhimJEmSKhimJEmSKhimJEmSKhimJEmSKhimJEmSKhimJEmSKhimJEmSKhimJEmSKhimJEmSKhimJEmSKhimJEmSKhimJEmSKhimJEmSKhimJEmSKhimJEmSKhimJEmSKhimJEmSKhimJEmSKhimJEmSKhimJEmSKhimJEmSKhimJEmSKhimJEmSKhimJEmSKhimJEmSKhimJEmSKhimJEmSKhimJEmSKhimJEmSKhimJEmSKhimJEmSKhimJEmSKhimJEmSKhimJEmSKhimJEmSKhimJEmSKhimJEmSKvQrTEXEsRFxf0Qsi4hz1lHuf0dERsTkgWuiJEnS8NVnmIqILYELgeOAScBJETGpQ7ntgT8Dvj/QjZQkSRqu+nNm6mBgWWY+kJnPAnOA4zuU+yTwd8BvB7B9kiRJw1p/wtQ4YEXL45Vl2moR8bvAhMy8ZgDbJkmSNOxVD0CPiC2AzwIf6UfZ0yJicUQsfvTRR2tXLUmSNOT6E6a6gQktj8eXaT22B/YDFkbEg8AhwLxOg9Az86LMnJyZk8eOHbvhrZYkSRom+hOmbgf2jIhXR8RoYDowr2dmZj6VmTtn5sTMnAjcBkzNzMUbpcWSJEnDSJ9hKjOfB84ArgN+BFyZmfdGxHkRMXVjN1CSJGk4G9WfQpk5H5jfNm1mL2WPqG+WJEnSpsFvQJckSapgmJIkSapgmJIkSapgmJIkSapgmJIkSapgmJIkSapgmJIkSapgmJIkSapgmJIkSapgmJIkSapgmJIkSapgmJIkSapgmJIkSapgmJIkSapgmJIkSapgmJIkSapgmJIkSapgmJIkSapgmJIkSapgmJIkSapgmJIkSapgmJIkSapgmJIkSapgmJIkSapgmJIkSapgmJIkSapgmJIkSapgmJIkSapgmJIkSapgmJIkSapgmJIkSapgmJIkSapgmJIkSapgmJIkSapgmJIkSapgmJIkSapgmJIkSapgmJIkSapgmJIkSapgmJIkSapgmJIkSapgmJIkSapgmJIkSapgmJIkSapgmJIkSapgmJIkSapgmJIkSapgmJIkSapgmJIkSapgmJIkSapgmJIkSapgmJIkSapgmJIkSapgmJIkSapgmJIkSapgmJIkSapgmJIkSapgmJIkSapgmJIkSapgmJIkSapgmJIkSapgmBrmFixYwN57701XVxfnn3/+WvM/+9nPMmnSJPbff3+OOuooHnroodXzjj32WMaMGcPb3va2wWyyJEmbFcPUMLZq1SpOP/10rr32WpYsWcLs2bNZsmTJGmUOPPBAFi9ezD333MO0adM466yzVs/76Ec/ymWXXTbYzZYkabNimBrGFi1aRFdXF3vssQejR49m+vTpzJ07d40yb3rTm9hmm20AOOSQQ1i5cuXqeUcddRTbb7/9oLZZkqTNjWFqGOvu7mbChAmrH48fP57u7u5ey1988cUcd9xxg9E0SZJUjBrqBmhgXH755SxevJibb755qJsiSdJmxTA1jI0bN44VK1asfrxy5UrGjRu3VrkbbriBT3/609x8881stdVWg9lESZI2e17mG8YOOuggli5dyvLly3n22WeZM2cOU6dOXaPMXXfdxfvf/37mzZvHLrvsMkQtlSRp82WYGsZGjRrFrFmzOOaYY9hnn314xzvewb777svMmTOZN28e0Hxi7+mnn+aEE07ggAMOWCNsHXbYYZxwwgnceOONjB8/nuuuu26oNkWSpBHLy3zD3JQpU5gyZcoa084777zV92+44YZel/2v//qvjdYuSZLU8MyUJElSBcOUJElSBcOUJElShRE9ZmriOdcMdRPUTw+e/9ahboIkSRukX2emIuLYiLg/IpZFxDkd5v95RCyJiHsi4saI2H3gmypJkjT89BmmImJL4ELgOGAScFJETGordhcwOTP3B64G/n6gGypJkjQc9efM1MHAssx8IDOfBeYAx7cWyMybMvPX5eFtwPiBbaYkSdLw1J8wNQ5Y0fJ4ZZnWm1OAa2saJUmStKkY0AHoEfFOYDLw+73MPw04DWC33XYbyFVLkiQNif6cmeoGJrQ8Hl+mrSEi3gx8DJiamc90qigzL8rMyZk5eezYsRvSXkmSpGGlP2HqdmDPiHh1RIwGpgPzWgtExIHAl2mC1M8HvpmSJEnDU59hKjOfB84ArgN+BFyZmfdGxHkR0fOrup8BtgOuioi7I2JeL9VJkiSNKP0aM5WZ84H5bdNmttx/8wC3S5IkaZPgz8lIkiRVMExJkiRVMExJkiRVMExJkiRVMExJkiRVMExJkiRVMExJkiRVMExJkiRVMExJkiRVMExJkiRVMExJkiRVMExJkiRVMExJkiRVMExJkiRVMExJkiRVMExJkiRVMExJkiRVMExJkiRVMExJkiRVMExJkiRVMExJkiRVMExJkiRVMExJkiRVMExJkiRVMExJkiRVMExJkiRVMExJkiRVMExJkiRVMExJkiRVMExJkiRVMExJkiRVMExJkiRVMExJkiRVMExJkiRVMExJkiRVMExJkiRVMExJkiRVMExJkiRVMExJkiRVMExJkiRVMExJkiRVMExJkiRVMExJkiRVMExJkiRVMExJkiRVMExJkiRVMExJkiRVMExJkiRVMExJkiRVMExJkiRVMExJkiRVMExJkiRVMExJkiRVMExJkiRVMExJkiRVMExJkiRVMExJkiRVMExJkiRVMExJkiRVMExJkiRVMExJkiRVMExJkiRVMExJkiRVMExJkiRVMExJkiRVMExJkiRVMExJm5gFCxaw995709XVxfnnn7/W/GeeeYYTTzyRrq4uXve61/Hggw8OfiMlaTNimJI2IatWreL000/n2muvZcmSJcyePZslS5asUebiiy/mZS97GcuWLePMM8/k7LPPHqLWStLmwTAlbUIWLVpEV1cXe+yxB6NHj2b69OnMnTt3jTJz587lPe95DwDTpk3jxhtvJDOHormStFkwTEmbkO7ubiZMmLD68fjx4+nu7u61zKhRo9hxxx15/PHHB7WdkrQ5MUxJkiRVMExJm5Bx48axYsWK1Y9XrlzJuHHjei3z/PPP89RTT/Hyl798UNspSZsTw5S0CTnooINYunQpy5cv59lnn2XOnDlMnTp1jTJTp07la1/7GgBXX301Rx55JBExFM2VpM3CqKFugKT+GzVqFLNmzeKYY45h1apVnHzyyey7777MnDmTyZMnM3XqVE455RTe9a530dXVxU477cScOXOGutmSNKL1K0xFxLHA54AtgX/OzPPb5m8FXAr8HvA4cGJmPjiwTZUEMGXKFKZMmbLGtPPOO2/1/a233pqrrrpqsJslSZutPi/zRcSWwIXAccAk4KSImNRW7BTgiczsAi4A/m6gGypJkjQc9WfM1MHAssx8IDOfBeYAx7eVOR74Wrl/NXBUOEhDkiRtBvoTpsYBK1oeryzTOpbJzOeBpwA/PiRJkka8QR2AHhGnAaeVh09HxP2Duf4RZGfgsaFuxEAKLwzXGnF9QgNixPULjxXVRlyfgEHrF7v3NqM/YaobmNDyeHyZ1qnMyogYBexIMxB9DZl5EXBRP9apdYiIxZk5eajboeHDPqFO7BdqZ5/YOPpzme92YM+IeHVEjAamA/PayswD3lPuTwO+nf4YmCRJ2gz0eWYqM5+PiDOA62i+GuGrmXlvRJwHLM7MecDFwGURsQz4BU3gkiRJGvH6NWYqM+cD89umzWy5/1vghIFtmtbBS6VqZ59QJ/YLtbNPbATh1ThJkqQN52/zSZIkVTBMbYCIOC8i3jyE6z8hIn4UETdV1jMxIn7YR5kjIuKb61nvwojw0yJFRMyPiDFDuP7PRMS9EfGZynpmRMSsPsqcGxF/sZ71Pl3TruFgqI8JG8J+MXSG+piwviJiq4i4ISLujogTK+u6JCKm9VFmvf6HbMj/qYHmDx2vp4jYsnW82ADWO6p84Wl/nAKcmpnfGeh2aOCUXwGIzJzSZ+H1r3t9+stpwE6ZuWqg26Fhc0zYEPaLQTaMjgnr60CAzDxgI9W/yfPMVFHO0twXEf9azvpcHRHblHkPRsTfRcSdwAk9yToijo2Iq1rq6JiOI2JKqfuOiPinnjLl3dplEXErzachZ0TE3JLKl0bEX3eoaybwRuDi8s5yYkT8V0TcWW6vL+XmRMRbW5Zb57uB3uopdoiIayLi/oj4UkRsUZY5OiK+V8pfFRHbreduHxbaz9BFxF9ExLnl/sLy3C+KiP+OiMPK9I7PVanr/oi4FPghMKH0n50j4vyIOL1lPR3frUfEX5U6vhMRs3vKlHX9Y0QsBv6sPKdfiojFpW1v61DXPGA74I6IODEi3h4R34+Iu8o7zVdExBaljWNallsaEa9Yxz5bq56W2a8t/WJpRJzassxHI+L2iLgnIj7R1/My1DaVY0JZ7umW+9Mi4pJy/5JS/3cj4oGeY4D9Yt1G+DFhjT4ZEbMiYka5/2BEfCKaY/oPIuJ3ImIX4HLgoGjOTL0mImaW5+yHEXFRNH4nIha17cMf9LGf16qnZfa7yvp+GBEHl/LbRsRXy76/KyLaf9pu6GSmt2YQ/kQggTeUx18F/qLcfxA4q6XsJTTfpzUK+AmwbZn+ReCdbfVuTfNTO68uj2cD3yz3zwXuAF5aHs8AHqb5KZ6X0rzwJndo68Ke6cA2wNbl/p40X1cB8IfA18r90aUNL+2wzT/so54jgN8Ce9B8Ncb1Zdt3Bm5p2fazgZnt7dsUbq37oTz+C+Dclm35h3J/CnDDup6rUtcLwCEt9T1Y9teBwM0t05cAE9rachBwd+k32wNLW/rhQuALbf1wAc2boj1pfupp6w7b93TL/Zfx4gdP3teybZ8D3lvuv65nO9vqmQHM6qOec4H/V/bJzqXf7QocTfMpoijt/SZweHv7htONTeuY0PocTwMuaWnXVWWfT6L5nVX7Rf+e+xF5TKA5pn+z5fEsYEZLuz5U7n8Q+Odeltmp5f5lwNvL/bt5sV+fDXy8w769BJjWRz0Lga+U+4fz4v+pv6G8noAxwH8D27a3byhunpla04rMvLXcv5zmDFCPK9oLZ3NKdQHw9mi++f2twNy2Yr8DPJCZy8vj2W3z52Xmb1oeX5+Zj5dpX29rQycvAb5S3gFcRXPABLgWeFNEbAUcB9zStp7+1gOwKJsful5V2v9G4JBS5taIuJvmS1t7/ar9TdzXy987aA6MPXp7rh7KzNvaK8nMu4BdImLXiHgt8ERmrmgr9gZgbmb+NjN/Cfxn2/z2fnhlZr6QmUuBB2j627qMB64rz/NHgX1b6u0ZCzG9w3r6Ww+l/b/JzMeAm2h+LP3ocrsLuLO0c88+1jEcbIrHhHb/UfrIEqC3s0r2i/Uzko4J/d22Vm8qZyB/ABzJi8/zlbzYX07s0Lb+1gPldZGZt9BcHRlD01fOKf9zFtIEzN36tVUbmWOm1tT+PRGtj3/VyzJzgDNovqx0cens66O93nW1oZMzgUeA19K8G/ktNN/9FRELgWNoOvWcDalnHW0KmgPHSX3Uuyl4njUveW/dNv+Z8ncVa75menuueusr0ATVacAr6ftA00ltf/k88NnMnBcRR9CcMQD4HtAVEWOBPwA+tYH19NamAP42M7/cR73DzaZyTGid1lv/heZ56MR+saaRfEzY0G0DICK2Br5Ac4Z0Rbn82VPHFcBVEfF1IEug66iPenrbjgD+d2au8bu+67r0PFg8M7Wm3SLi0HL/j4H+DPC+Gfhd4FQ6B5b7gT0iYmJ53NcnId4SETtFxEtpDl63AkTEjRExrkP5HYGHM/MF4F00l+J6XAG8FziM5t3yuqyrnoOj+TmhLUr7vwPcBrwhIrpK+7aNiL36WMdw9QjNu8OXlzN5a40z6EXH56oPV9C8w59GcxBtdyvNWY2toxmD1ldbTohmbMtraC7F3h8R4yLixl7K78iLv635np6J2Zw3/wbwWeBHmbnWb2v2p57i+NL+l9Ocfr+d5hcUTi7bRGnjLn2sYzjYVI4Jj0TEPuU1+of9aGM7+8WaRvIx4SFgUjSf0BsDHNWPNrbqCTyPlfasHoubmT+mCWF/Rd/BsNd6ihMBIuKNwFOZ+RRNf/lQz9iqiDhwPdu+0Xhmak33A6dHxFdprl1/sa8FMnNVNIP5ZrD2wYPM/E1EfBBYEBG/ojmArMsi4N9pTpdfnpmLywGyi+adbrsvAP8eEe+mCUyt71K+RXMdem5mPtvHetdVz+0019W7aE7PfyMzX4hm0OLscrAB+DjNNexNSmY+F83PIy2i+UdwXz8X7fRcTexjXfdGxPZAd2Y+3GH+7dEMDr6H5oD+A+CpdVT5k9KOHYAPlDOSr6J599nJuTTvHJ8Avg28umXeFTTP9Yx1bUM/6rmHpp/sDHwyM38K/DQi9gG+V46DTwPvBH7ej3UNpU3lmHAOzXijR4HFNIPL18e52C9WG8nHhHIW6EqaMV3LaS6x9ltmPhkRXynL/4y1++8VwGdY87nfkHp+GxF30QxBOblM+yTwj8A95TWwnP4H3Y3Kb0AvSof/ZmbutxHq3i4zny5p+kJgaWZe0KHcDJpTnme0Td8PODkz/3yg26YN09tzNUB19/SXbWgG+Z+WmXd2KHcJTZ+9um36GcBPsvndTG0gjwlaHx4TNm+emRocp0bEe2g+VXcXsF7jAzLzh4AHzc3HRRExieY0+Nc6HTTXJTPX+QWKGhY8Jmh9eEwY5jwzJUmSVMEB6JIkSRUMU5IkSRUMU5IkSRUMU5IkSRUMU5IkSRUMU5IkSRX+P/3r2Cg6CpQLAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 720x576 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_prop_prtced_grp_lab(dset_tst_pred, \"Proportion of Groups and Labels of Model on Test Data\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "27fc3729",
   "metadata": {},
   "source": [
    "We will now evaluate the fairness of our model using several of the definitions for model fairness we saw [before](Fair-Classifier.ipynb)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "id": "f2e58054",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy: 0.8044086535180509\n",
      "Predictive Parity Difference: -0.6613908872901678\n",
      "FNR Difference: 0.4630624580255205\n",
      "Accuracy Difference: -0.1183125130653383\n",
      "Average Positive Class Difference: -0.25324331468908357\n",
      "Average Negative Class Difference: -0.13724064726347057\n"
     ]
    }
   ],
   "source": [
    "mod_metrics = ClassificationMetric(dset_raw_tst, dset_tst_pred,\n",
    "                                            unprivileged_groups=unpriv_group,\n",
    "                                            privileged_groups=priv_group)\n",
    "print(\"Accuracy:\", mod_metrics.accuracy())\n",
    "print(\"Predictive Parity Difference:\", mod_metrics.positive_predictive_value(False)-mod_metrics.positive_predictive_value(True))\n",
    "print(\"FNR Difference:\", mod_metrics.false_negative_rate_difference())\n",
    "print(\"Accuracy Difference:\", mod_metrics.error_rate_difference())\n",
    "print(\"Average Positive Class Difference:\", np.mean(dset_tst_prob[(dset_raw_tst.labels.ravel() == 1.)&(dset_raw_tst.features[:,1].ravel() == 0.)])\n",
    "                                        - np.mean(dset_tst_prob[(dset_raw_tst.labels.ravel() == 1.)&(dset_raw_tst.features[:,1].ravel() == 1.)]))\n",
    "print(\"Average Negative Class Difference:\", np.mean(dset_tst_prob[(dset_raw_tst.labels.ravel() == 0.)&(dset_raw_tst.features[:,1].ravel() == 0.)])\n",
    "                                        - np.mean(dset_tst_prob[(dset_raw_tst.labels.ravel() == 0.)&(dset_raw_tst.features[:,1].ravel() == 1.)]))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f680056b",
   "metadata": {},
   "source": [
    "We see above that:\n",
    "- Even though the model accuracy is good, it severely shows fairness issue as none of the difference values are close to 0. \n",
    "- Again, the same trend is seen that the unprivileged group gets the unfavorable label much more than the privileged group.\n",
    "- A high value of FNR difference indicates the model wrongly labels unprivileged group individuals to unfavorable label.\n",
    "- On an average our model predicts favorable outcome 25% more for privileged group compared to that of unprivileged group, where the individuals of both groups actually have a favorable outcome.\n",
    "- On an average our model predicts favorable outcome 13.7% more for privileged group compared to that of unprivileged group, where the individuals of both groups actually have a unfavorable outcome."
   ]
  }
 ],
 "metadata": {
  "authors": [
   {
    "name": "Rishi Dey Chowdhury"
   }
  ],
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  },
  "title": "Fairness Definitions Comparison"
 },
 "nbformat": 4,
 "nbformat_minor": 5
}