notebooks/trees_sol_01.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# \ud83d\udcc3 Solution for Exercise M5.01\n",
    "\n",
    "In the previous notebook, we showed how a tree with a depth of 1 level was\n",
    "working. The aim of this exercise is to repeat part of the previous\n",
    "experiment for a depth with 2 levels to show how the process of partitioning\n",
    "is repeated over time.\n",
    "\n",
    "Before to start, we will:\n",
    "\n",
    "* load the dataset;\n",
    "* split the dataset into training and testing dataset;\n",
    "* define the function to show the classification decision function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "\n",
    "penguins = pd.read_csv(\"../datasets/penguins_classification.csv\")\n",
    "culmen_columns = [\"Culmen Length (mm)\", \"Culmen Depth (mm)\"]\n",
    "target_column = \"Species\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"admonition note alert alert-info\">\n",
    "<p class=\"first admonition-title\" style=\"font-weight: bold;\">Note</p>\n",
    "<p class=\"last\">If you want a deeper overview regarding this dataset, you can refer to the\n",
    "Appendix - Datasets description section at the end of this MOOC.</p>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "data, target = penguins[culmen_columns], penguins[target_column]\n",
    "data_train, data_test, target_train, target_test = train_test_split(\n",
    "    data, target, random_state=0\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create a decision tree classifier with a maximum depth of 2 levels and fit\n",
    "the training data. Once this classifier trained, plot the data and the\n",
    "decision boundary to see the benefit of increasing the depth. To plot the\n",
    "decision boundary, you should import the class `DecisionBoundaryDisplay`\n",
    "from the module `helpers.plotting` as shown in the previous course notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# solution\n",
    "from sklearn.tree import DecisionTreeClassifier\n",
    "\n",
    "tree = DecisionTreeClassifier(max_depth=2)\n",
    "tree.fit(data_train, target_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "solution"
    ]
   },
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "\n",
    "from helpers.plotting import DecisionBoundaryDisplay\n",
    "\n",
    "palette = [\"tab:red\", \"tab:blue\", \"black\"]\n",
    "DecisionBoundaryDisplay.from_estimator(\n",
    "    tree, data_train, response_method=\"predict\", cmap=\"RdBu\", alpha=0.5\n",
    ")\n",
    "ax = sns.scatterplot(data=penguins, x=culmen_columns[0], y=culmen_columns[1],\n",
    "                     hue=target_column, palette=palette)\n",
    "plt.legend(bbox_to_anchor=(1.05, 1), loc='upper left')\n",
    "_ = plt.title(\"Decision boundary using a decision tree\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Did we make use of the feature \"Culmen Length\"?\n",
    "Plot the tree using the function `sklearn.tree.plot_tree` to find out!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# solution\n",
    "from sklearn.tree import plot_tree\n",
    "\n",
    "_, ax = plt.subplots(figsize=(16, 12))\n",
    "_ = plot_tree(tree, feature_names=culmen_columns,\n",
    "              class_names=tree.classes_, impurity=False, ax=ax)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "solution"
    ]
   },
   "source": [
    "The resulting tree has 7 nodes: 3 of them are \"split nodes\" and 4\n",
    "are \"leaf nodes\" (or simply \"leaves\"), organized in 2 levels.\n",
    "We see that the second tree level used the \"Culmen Length\" to make\n",
    "two new decisions. Qualitatively, we saw that such a simple tree was enough\n",
    "to classify the penguins' species."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Compute the accuracy of the decision tree on the testing data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# solution\n",
    "test_score = tree.fit(data_train, target_train).score(data_test, target_test)\n",
    "print(f\"Accuracy of the DecisionTreeClassifier: {test_score:.2f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "solution"
    ]
   },
   "source": [
    "At this stage, we have the intuition that a decision tree is built by\n",
    "successively partitioning the feature space, considering one feature at a\n",
    "time.\n",
    "\n",
    "We predict an Adelie penguin if the feature value is below the threshold,\n",
    "which is not surprising since this partition was almost pure. If the feature\n",
    "value is above the threshold, we predict the Gentoo penguin, the class that\n",
    "is most probable."
   ]
  }
 ],
 "metadata": {
  "jupytext": {
   "main_language": "python"
  },
  "kernelspec": {
   "display_name": "Python 3",
   "name": "python3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}