notebooks/linear_models_sol_03.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# \ud83d\udcc3 Solution for Exercise M4.03\n",
    "\n",
    "In all previous notebooks, we only used a single feature in `data`. But we\n",
    "have already shown that we could add new features to make the model more\n",
    "expressive by deriving new features, based on the original feature.\n",
    "\n",
    "The aim of this notebook is to train a linear regression algorithm on a\n",
    "dataset with more than a single feature.\n",
    "\n",
    "We will load a dataset about house prices in California.\n",
    "The dataset consists of 8 features regarding the demography and geography of\n",
    "districts in California and the aim is to predict the median house price of\n",
    "each district. We will use all 8 features to predict the target, the median\n",
    "house price."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"admonition note alert alert-info\">\n",
    "<p class=\"first admonition-title\" style=\"font-weight: bold;\">Note</p>\n",
    "<p class=\"last\">If you want a deeper overview regarding this dataset, you can refer to the\n",
    "Appendix - Datasets description section at the end of this MOOC.</p>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.datasets import fetch_california_housing\n",
    "\n",
    "data, target = fetch_california_housing(as_frame=True, return_X_y=True)\n",
    "target *= 100  # rescale the target in k$\n",
    "data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now it is your turn to train a linear regression model on this dataset.\n",
    "First, create a linear regression model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# solution\n",
    "from sklearn.linear_model import LinearRegression\n",
    "\n",
    "linear_regression = LinearRegression()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Execute a cross-validation with 10 folds and use the mean absolute error\n",
    "(MAE) as metric. Be sure to *return* the fitted *estimators*."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# solution\n",
    "from sklearn.model_selection import cross_validate\n",
    "\n",
    "cv_results = cross_validate(linear_regression, data, target,\n",
    "                            scoring=\"neg_mean_absolute_error\",\n",
    "                            return_estimator=True, cv=10, n_jobs=2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Compute the mean and std of the MAE in thousands of dollars (k$)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# solution\n",
    "print(f\"Mean absolute error on testing set: \"\n",
    "      f\"{-cv_results['test_score'].mean():.3f} k$ +/- \"\n",
    "      f\"{cv_results['test_score'].std():.3f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lines_to_next_cell": 2
   },
   "source": [
    "Inspect the fitted model using a box plot to show the distribution of values\n",
    "for the coefficients returned from the cross-validation. Hint:\n",
    "use the function\n",
    "[`df.plot.box()`](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.plot.box.html)\n",
    "to create a box plot."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# solution\n",
    "import pandas as pd\n",
    "\n",
    "weights = pd.DataFrame(\n",
    "    [est.coef_ for est in cv_results[\"estimator\"]], columns=data.columns)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "solution"
    ]
   },
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "color = {\"whiskers\": \"black\", \"medians\": \"black\", \"caps\": \"black\"}\n",
    "weights.plot.box(color=color, vert=False)\n",
    "_ = plt.title(\"Value of linear regression coefficients\")"
   ]
  }
 ],
 "metadata": {
  "jupytext": {
   "main_language": "python"
  },
  "kernelspec": {
   "display_name": "Python 3",
   "name": "python3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}