Intro_to_ML_Python_Instructor.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "# Machine Learning with Python\n",
    "\n",
    "Now that we've learned how to acquire, clean, and visualize our data, let's start doing some machine learning. Throughout this workshop, we will be using a Python package called scikit-learn. It is surprisingly easy to implement machine learning algorithms using scikit-learn, and in fact much of the work is done for you. Often, it is the acquisition and structuring of the data itself that requires the most finesse. Because of this, we will be working with pre-processed data to illustrate just how simple machine learning can be using scikit-learn. \n",
    "\n",
    "Before we dive in, let's start with some background on machine learning. There are two main types of machine learning algorithms: **supervised** and **unsupervised**. The goal of **supervised learning** is to train a model to classify data against a set of labels. For instance, we can train a model to separate images according to whether or not the image contains a face. In this case, the two labels would be *face* and *no face*. The presence of labeled training data is a huge advantage in machine learning, however it is not always available.\n",
    "\n",
    "When labeled training data is unavailable, we must use **unsupervised learning**. The goal of unsupervised learning is to train a model to extract *meaningful features* from the data. These meaningful features are often common patterns which are useful for compressing the information contained in the data. For example, in speech recognition, individual phonemes serve as meaningful features from which it is possible to reconstruct words independently of their tone or pitch. In computer vision, oriented edge-detectors are often the best features."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Digit Recognition\n",
    "\n",
    "In the first half of this workshop, we'll be using a very important supervised machine learning algorithm called the **support vector machine** to classify handwritten digits. This is a very well-studied problem in the machine learning community, and serves as a great starting point. First, let's import scikit-learn and a couple other modules we'll need."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "from sklearn import datasets\n",
    "from sklearn import svm\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, sklearn comes with a few preloaded datasets, so let's load up the handwritten digits dataset. This is a list of pixel intensities corresponding to images of handwritten digits plus their associated labels (0-9)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[  0.   0.   5. ...,   0.   0.   0.]\n",
      " [  0.   0.   0. ...,  10.   0.   0.]\n",
      " [  0.   0.   0. ...,  16.   9.   0.]\n",
      " ..., \n",
      " [  0.   0.   1. ...,   6.   0.   0.]\n",
      " [  0.   0.   2. ...,  12.   0.   0.]\n",
      " [  0.   0.  10. ...,  12.   1.   0.]]\n"
     ]
    }
   ],
   "source": [
    "digits = datasets.load_digits()\n",
    "print(digits.data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's use matplotlib to see what one of these images looks like"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPgAAAD8CAYAAABaQGkdAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAACylJREFUeJzt3f+rlvUdx/HXa0ftTLMc2So8MmuU\nEItlOYc4guk2bEUFG0uhxmJwYFAUyaJGY9s/EO6HEYTVglzSrCBaXxaraIEzv+QqOzpMGp6sNPru\nSD353g/nFpw7230d7+vb/e75gIPnPt6cz/tGnl7Xuc99Xx9HhADk9IWmBwBQHQIHEiNwIDECBxIj\ncCAxAgcSI3AgMQIHEiNwILEpVXzTaT4pBjWjim/dqLHZ9T6mM898r7a13jwwq7a1BkcP17ZWHB6r\nba06faoDOhQH3e1+lQQ+qBn6ppdV8a0b9e4PFte63s9XrattrV9uubK2tc67+a3a1hp7+53a1qrT\nxvhLoftxig4kRuBAYgQOJEbgQGIEDiRG4EBiBA4kRuBAYoUCt73c9k7bu2zfWvVQAMrRNXDbA5J+\nJ+lSSedLWmn7/KoHA9C7IkfwRZJ2RcTuiDgkaZ2k+l7XCOCEFQl8jqQ9x9we7XwNQMsVebPJRO9Y\n+a+LqdseljQsSYOa3uNYAMpQ5Ag+KmnuMbeHJO09/k4RcVdELIyIhVN1UlnzAehBkcA3STrX9tm2\np0laIenRascCUIaup+gRMWb7eklPSRqQdE9EbK98MgA9K3TBh4h4XNLjFc8CoGS8kg1IjMCBxAgc\nSIzAgcQIHEiMwIHECBxIjMCBxCrZ2SSrOncakaQVM9+vba3Vsz6pba0/bX2qtrUu/vXPaltLkmbf\ntaHW9brhCA4kRuBAYgQOJEbgQGIEDiRG4EBiBA4kRuBAYgQOJFZkZ5N7bO+z/WodAwEoT5Ej+O8l\nLa94DgAV6Bp4RDwv6b0aZgFQMn4GBxIr7d1kbF0EtE9pR3C2LgLah1N0ILEivyZ7QNIGSfNtj9r+\nafVjAShDkb3JVtYxCIDycYoOJEbgQGIEDiRG4EBiBA4kRuBAYgQOJEbgQGJ9v3XR2NKLa1trxcxt\nta0lSZcuX1HbWqe+vKO2tX70wrLa1npvwWe1rSVJs2tdrTuO4EBiBA4kRuBAYgQOJEbgQGIEDiRG\n4EBiBA4kRuBAYgQOJFbkootzbT9re8T2dts31jEYgN4VeS36mKRVEbHV9kxJW2w/HRGvVTwbgB4V\n2ZvsrYjY2vn8Y0kjkuZUPRiA3k3q3WS250laIGnjBH/H1kVAyxR+ks32yZIeknRTRHx0/N+zdRHQ\nPoUCtz1V43GvjYiHqx0JQFmKPItuSXdLGomIO6ofCUBZihzBl0i6VtJS29s6H9+veC4AJSiyN9kL\nklzDLABKxivZgMQIHEiMwIHECBxIjMCBxAgcSIzAgcQIHEis7/cm+/S0+h7C7fsuqG0tSTpS435h\nddr0ylebHuFzgyM4kBiBA4kROJAYgQOJETiQGIEDiRE4kBiBA4kROJBYkYsuDtp+0fbfO1sX/aaO\nwQD0rsjrPA9KWhoRn3Qun/yC7Sci4m8VzwagR0UuuhiSPuncnNr5iCqHAlCOohsfDNjeJmmfpKcj\nYsKti2xvtr35sA6WPSeAE1Ao8Ij4LCIulDQkaZHtr01wH7YuAlpmUs+iR8QHkp6TtLySaQCUqsiz\n6KfbntX5/IuSviMp5xuVgWSKPIt+lqT7bA9o/D+EByPisWrHAlCGIs+iv6zxPcEB9BleyQYkRuBA\nYgQOJEbgQGIEDiRG4EBiBA4kRuBAYv2/ddGX6vs/au2GxbWtJUnn6cVa16vLlFMP1bbW2IfTalur\njTiCA4kROJAYgQOJETiQGIEDiRE4kBiBA4kROJAYgQOJFQ68c230l2xzPTagT0zmCH6jpJGqBgFQ\nvqI7mwxJukzSmmrHAVCmokfw1ZJukXSkwlkAlKzIxgeXS9oXEVu63I+9yYCWKXIEXyLpCttvSFon\naant+4+/E3uTAe3TNfCIuC0ihiJinqQVkp6JiGsqnwxAz/g9OJDYpK7oEhHPaXx3UQB9gCM4kBiB\nA4kROJAYgQOJETiQGIEDiRE4kBiBA4n1/dZFg+/X9wa3b1zwem1rSdKHNa415cwzalvr6vP/7/uW\nSvXgE9+qba024ggOJEbgQGIEDiRG4EBiBA4kRuBAYgQOJEbgQGIEDiRW6JVsnSuqfizpM0ljEbGw\nyqEAlGMyL1X9dkS8W9kkAErHKTqQWNHAQ9KfbW+xPVzlQADKU/QUfUlE7LX9ZUlP294REc8fe4dO\n+MOSNKjpJY8J4EQUOoJHxN7On/skPSJp0QT3YesioGWKbD44w/bMo59L+p6kV6seDEDvipyinyHp\nEdtH7/+HiHiy0qkAlKJr4BGxW9LXa5gFQMn4NRmQGIEDiRE4kBiBA4kROJAYgQOJETiQGIEDifX9\n1kWn7Kxvg59fDT1W21qS9OPhm2tba+pV+2tbq05n37ah6REaxREcSIzAgcQIHEiMwIHECBxIjMCB\nxAgcSIzAgcQIHEisUOC2Z9leb3uH7RHbi6seDEDvir5U9beSnoyIH9qeJnHhc6AfdA3c9imSLpH0\nE0mKiEOSDlU7FoAyFDlFP0fSfkn32n7J9prO9dEBtFyRwKdIukjSnRGxQNIBSbcefyfbw7Y32958\nWAdLHhPAiSgS+Kik0YjY2Lm9XuPB/we2LgLap2vgEfG2pD2253e+tEzSa5VOBaAURZ9Fv0HS2s4z\n6LslXVfdSADKUijwiNgmaWHFswAoGa9kAxIjcCAxAgcSI3AgMQIHEiNwIDECBxIjcCAxAgcS6/u9\nyY68vKO2ta6+c1Vta0nS7aseqG2t1a8vq22tTRcO1LbW5x1HcCAxAgcSI3AgMQIHEiNwIDECBxIj\ncCAxAgcSI3Agsa6B255ve9sxHx/ZvqmO4QD0putLVSNip6QLJcn2gKQ3JT1S8VwASjDZU/Rlkl6P\niH9WMQyAck32zSYrJE34Dgjbw5KGJWmQzUeBVih8BO9senCFpD9O9PdsXQS0z2RO0S+VtDUi3qlq\nGADlmkzgK/U/Ts8BtFOhwG1Pl/RdSQ9XOw6AMhXdm+xfkk6reBYAJeOVbEBiBA4kRuBAYgQOJEbg\nQGIEDiRG4EBiBA4k5ogo/5va+yVN9i2lsyW9W/ow7ZD1sfG4mvOViDi9250qCfxE2N4cEQubnqMK\nWR8bj6v9OEUHEiNwILE2BX5X0wNUKOtj43G1XGt+BgdQvjYdwQGUrBWB215ue6ftXbZvbXqeMtie\na/tZ2yO2t9u+semZymR7wPZLth9repYy2Z5le73tHZ1/u8VNz9SLxk/RO9da/4fGrxgzKmmTpJUR\n8Vqjg/XI9lmSzoqIrbZnStoi6ap+f1xH2b5Z0kJJp0TE5U3PUxbb90n6a0Ss6VxodHpEfND0XCeq\nDUfwRZJ2RcTuiDgkaZ2kKxueqWcR8VZEbO18/rGkEUlzmp2qHLaHJF0maU3Ts5TJ9imSLpF0tyRF\nxKF+jltqR+BzJO055vaokoRwlO15khZI2tjsJKVZLekWSUeaHqRk50jaL+nezo8fa2zPaHqoXrQh\ncE/wtTRP7ds+WdJDkm6KiI+anqdXti+XtC8itjQ9SwWmSLpI0p0RsUDSAUl9/ZxQGwIflTT3mNtD\nkvY2NEupbE/VeNxrIyLLFWmXSLrC9hsa/3Fqqe37mx2pNKOSRiPi6JnWeo0H37faEPgmSefaPrvz\npMYKSY82PFPPbFvjP8uNRMQdTc9Tloi4LSKGImKexv+tnomIaxoeqxQR8bakPbbnd760TFJfPyk6\n2b3JShcRY7avl/SUpAFJ90TE9obHKsMSSddKesX2ts7XfhERjzc4E7q7QdLazsFmt6TrGp6nJ43/\nmgxAddpwig6gIgQOJEbgQGIEDiRG4EBiBA4kRuBAYgQOJPZvavih6sahAwsAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x20a60eaef98>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "\n",
    "plt.imshow(digits.images[0])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "By default, matplotlib plots each value on a color scale. We can convert this to greyscale to get a better idea of the actual image."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPgAAAD8CAYAAABaQGkdAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAACtlJREFUeJzt3V9onfUdx/HPZ1HZ/FOsazekqYsB\nKchgtoaCFITVZdQpuospLShMBr1SlA2s7m53eiPuYghSdYKd0lQFEacTVJywOZO226ypo60dzapr\nyir+GaxUv7vIKXRdtjzp+T1/ztf3C4L5c8jve4jvPs85OXl+jggByOlLbQ8AoD4EDiRG4EBiBA4k\nRuBAYgQOJEbgQGIEDiRG4EBiZ9XxTZctWxYjIyN1fOtWHTt2rNH1ZmZmGltryZIlja01PDzc2FpD\nQ0ONrdWkgwcP6ujRo17odrUEPjIyosnJyTq+dasmJiYaXW/Lli2NrTU+Pt7YWvfdd19jay1durSx\ntZo0NjZW6XacogOJETiQGIEDiRE4kBiBA4kROJAYgQOJETiQWKXAbW+w/a7tfbbvqXsoAGUsGLjt\nIUm/kHStpMslbbJ9ed2DAehflSP4Wkn7IuJARByX9JSkG+sdC0AJVQJfIenQKR/P9D4HoOOqBD7f\nX6z818XUbW+2PWl7cnZ2tv/JAPStSuAzklae8vGwpMOn3ygiHo6IsYgYW758ean5APShSuBvSbrM\n9qW2z5G0UdJz9Y4FoIQF/x48Ik7Yvl3SS5KGJD0aEXtqnwxA3ypd8CEiXpD0Qs2zACiMV7IBiRE4\nkBiBA4kROJAYgQOJETiQGIEDiRE4kFgtO5tk1eROI5L03nvvNbZWk9syXXTRRY2ttX379sbWkqSb\nbrqp0fUWwhEcSIzAgcQIHEiMwIHECBxIjMCBxAgcSIzAgcQIHEisys4mj9o+YvvtJgYCUE6VI/gv\nJW2oeQ4ANVgw8Ih4XdI/GpgFQGE8BgcSKxY4WxcB3VMscLYuArqHU3QgsSq/JntS0u8krbI9Y/tH\n9Y8FoIQqe5NtamIQAOVxig4kRuBAYgQOJEbgQGIEDiRG4EBiBA4kRuBAYgO/ddHU1FRjazW5lZAk\n7d+/v7G1RkdHG1trfHy8sbWa/P9DYusiAA0icCAxAgcSI3AgMQIHEiNwIDECBxIjcCAxAgcSI3Ag\nsSoXXVxp+1Xb07b32L6zicEA9K/Ka9FPSPpJROy0fYGkKdsvR8Q7Nc8GoE9V9iZ7PyJ29t7/WNK0\npBV1Dwagf4t6DG57RNJqSW/O8zW2LgI6pnLgts+X9LSkuyLio9O/ztZFQPdUCtz22ZqLe1tEPFPv\nSABKqfIsuiU9Imk6Ih6ofyQApVQ5gq+TdKuk9bZ3996+V/NcAAqosjfZG5LcwCwACuOVbEBiBA4k\nRuBAYgQOJEbgQGIEDiRG4EBiBA4kNvB7kx07dqyxtdasWdPYWlKz+4U16corr2x7hC8MjuBAYgQO\nJEbgQGIEDiRG4EBiBA4kRuBAYgQOJEbgQGJVLrr4Zdt/sP3H3tZFP2tiMAD9q/JS1X9JWh8Rn/Qu\nn/yG7V9HxO9rng1An6pcdDEkfdL78OzeW9Q5FIAyqm58MGR7t6Qjkl6OCLYuAgZApcAj4rOIuELS\nsKS1tr85z23YugjomEU9ix4RH0p6TdKGWqYBUFSVZ9GX276w9/5XJH1H0t66BwPQvyrPol8s6XHb\nQ5r7B2F7RDxf71gASqjyLPqfNLcnOIABwyvZgMQIHEiMwIHECBxIjMCBxAgcSIzAgcQIHEiMrYsW\nYXx8vLG1MmvyZ7Z06dLG1uoijuBAYgQOJEbgQGIEDiRG4EBiBA4kRuBAYgQOJEbgQGKVA+9dG32X\nba7HBgyIxRzB75Q0XdcgAMqrurPJsKTrJG2tdxwAJVU9gj8o6W5Jn9c4C4DCqmx8cL2kIxExtcDt\n2JsM6JgqR/B1km6wfVDSU5LW237i9BuxNxnQPQsGHhH3RsRwRIxI2ijplYi4pfbJAPSN34MDiS3q\nii4R8ZrmdhcFMAA4ggOJETiQGIEDiRE4kBiBA4kROJAYgQOJETiQ2MBvXdTk1jRTU//3720GWpPb\nCU1OTja21s0339zYWl3EERxIjMCBxAgcSIzAgcQIHEiMwIHECBxIjMCBxAgcSKzSK9l6V1T9WNJn\nkk5ExFidQwEoYzEvVf12RBytbRIAxXGKDiRWNfCQ9BvbU7Y31zkQgHKqnqKvi4jDtr8m6WXbeyPi\n9VNv0At/syRdcsklhccEcCYqHcEj4nDvv0ckPStp7Ty3YesioGOqbD54nu0LTr4v6buS3q57MAD9\nq3KK/nVJz9o+eftfRcSLtU4FoIgFA4+IA5K+1cAsAArj12RAYgQOJEbgQGIEDiRG4EBiBA4kRuBA\nYgQOJDbwWxeNjo42tlaTW+5I0sTERMq1mrRly5a2R2gVR3AgMQIHEiNwIDECBxIjcCAxAgcSI3Ag\nMQIHEiNwILFKgdu+0PYO23ttT9u+qu7BAPSv6ktVfy7pxYj4ge1zJJ1b40wAClkwcNtLJF0t6YeS\nFBHHJR2vdywAJVQ5RR+VNCvpMdu7bG/tXR8dQMdVCfwsSWskPRQRqyV9Kume029ke7PtSduTs7Oz\nhccEcCaqBD4jaSYi3ux9vENzwf8Hti4CumfBwCPiA0mHbK/qfeoaSe/UOhWAIqo+i36HpG29Z9AP\nSLqtvpEAlFIp8IjYLWms5lkAFMYr2YDECBxIjMCBxAgcSIzAgcQIHEiMwIHECBxIjMCBxNibbBHu\nv//+xtaSmt1Xa2ysuRcqTk1NNbbWFx1HcCAxAgcSI3AgMQIHEiNwIDECBxIjcCAxAgcSI3AgsQUD\nt73K9u5T3j6yfVcTwwHoz4IvVY2IdyVdIUm2hyT9TdKzNc8FoIDFnqJfI2l/RPy1jmEAlLXYwDdK\nenK+L7B1EdA9lQPvbXpwg6SJ+b7O1kVA9yzmCH6tpJ0R8fe6hgFQ1mIC36T/cXoOoJsqBW77XEnj\nkp6pdxwAJVXdm+yfkr5a8ywACuOVbEBiBA4kRuBAYgQOJEbgQGIEDiRG4EBiBA4k5ogo/03tWUmL\n/ZPSZZKOFh+mG7LeN+5Xe74REQv+VVctgZ8J25MR0dwGWQ3Ket+4X93HKTqQGIEDiXUp8IfbHqBG\nWe8b96vjOvMYHEB5XTqCAyisE4Hb3mD7Xdv7bN/T9jwl2F5p+1Xb07b32L6z7ZlKsj1ke5ft59ue\npSTbF9reYXtv72d3Vdsz9aP1U/Tetdb/orkrxsxIekvSpoh4p9XB+mT7YkkXR8RO2xdImpL0/UG/\nXyfZ/rGkMUlLIuL6tucpxfbjkn4bEVt7Fxo9NyI+bHuuM9WFI/haSfsi4kBEHJf0lKQbW56pbxHx\nfkTs7L3/saRpSSvanaoM28OSrpO0te1ZSrK9RNLVkh6RpIg4PshxS90IfIWkQ6d8PKMkIZxke0TS\naklvtjtJMQ9KulvS520PUtiopFlJj/Uefmy1fV7bQ/WjC4F7ns+leWrf9vmSnpZ0V0R81PY8/bJ9\nvaQjETHV9iw1OEvSGkkPRcRqSZ9KGujnhLoQ+Iyklad8PCzpcEuzFGX7bM3FvS0islyRdp2kG2wf\n1NzDqfW2n2h3pGJmJM1ExMkzrR2aC35gdSHwtyRdZvvS3pMaGyU91/JMfbNtzT2Wm46IB9qep5SI\nuDcihiNiRHM/q1ci4paWxyoiIj6QdMj2qt6nrpE00E+KVrpscp0i4oTt2yW9JGlI0qMRsaflsUpY\nJ+lWSX+2vbv3uZ9GxAstzoSF3SFpW+9gc0DSbS3P05fWf00GoD5dOEUHUBMCBxIjcCAxAgcSI3Ag\nMQIHEiNwIDECBxL7NyyRs2/TGgiSAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x20a6579ef28>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.imshow(digits.images[0], cmap=plt.cm.gray_r)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPgAAAD8CAYAAABaQGkdAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAACwNJREFUeJzt3e1rnfUdx/HPZ1HZ6l2h7YY0dUdB\nCjJYKqEgBenqNuoUmwd70IJCZNBHimED0T1y/4CmD4YgVSvYKVu9QcTpBI1O2Jxpm23W1NGVjGbV\nNnXGu8FC9bsHOYWuZuRKz3WXL+8XBHOSQ37fQ3l7Xefk5Po5IgQgp681PQCA6hA4kBiBA4kROJAY\ngQOJETiQGIEDiRE4kBiBA4ldUMUPXb16dXQ6nSp+9FfMzc3Vso4knThxora1JOnDDz+sba2+vr7a\n1lq5cmVta61ataq2tSRpxYoVtawzNTWlU6dOebH7VRJ4p9PR+Ph4FT/6K6ampmpZR5JGR0drW0uS\n9uzZU9tadUY3NDRU21rDw8O1rSVJAwMDtawzODhY6H6cogOJETiQGIEDiRE4kBiBA4kROJAYgQOJ\nETiQWKHAbW+1/Z7tI7bvrXooAOVYNHDbfZJ+KekmSddK2mH72qoHA9C7IkfwjZKORMTRiJiT9JSk\nbdWOBaAMRQJfK+nYWbenu18D0HJFAl/oL1a+cjF12zttj9sen5mZ6X0yAD0rEvi0pHVn3e6XdPzc\nO0XEwxExGBGDa9asKWs+AD0oEvjbkq6xfZXtiyRtl/R8tWMBKMOifw8eEadt3ynpZUl9kh6NiEOV\nTwagZ4Uu+BARL0p6seJZAJSMd7IBiRE4kBiBA4kROJAYgQOJETiQGIEDiRE4kFglO5vUqc6dTcbG\nxmpbS5JGRkZqW2t2dra2tXbt2lXbWnXu2CLVt7NJURzBgcQIHEiMwIHECBxIjMCBxAgcSIzAgcQI\nHEiMwIHEiuxs8qjtk7bfqWMgAOUpcgTfI2lrxXMAqMCigUfEG5L+VcMsAErGc3AgsdICZ+sioH1K\nC5yti4D24RQdSKzIr8melPQHSettT9v+SfVjAShDkb3JdtQxCIDycYoOJEbgQGIEDiRG4EBiBA4k\nRuBAYgQOJEbgQGLLfuuizZs317bWxMREbWtJ0p49e2pb6/77769trcsvv7y2tYaGhmpbq404ggOJ\nETiQGIEDiRE4kBiBA4kROJAYgQOJETiQGIEDiRE4kFiRiy6us/2a7Unbh2zfXcdgAHpX5L3opyX9\nLCIO2L5U0n7br0TEuxXPBqBHRfYmez8iDnQ//1TSpKS1VQ8GoHdLeg5uuyNpg6S3FvgeWxcBLVM4\ncNuXSHpa0khEfHLu99m6CGifQoHbvlDzce+NiGeqHQlAWYq8im5Jj0iajIgHqh8JQFmKHME3Sbpd\n0hbbE92PH1U8F4ASFNmb7E1JrmEWACXjnWxAYgQOJEbgQGIEDiRG4EBiBA4kRuBAYgQOJLbs9ybL\n7Lnnnmt6hErUucdbp9Opba024ggOJEbgQGIEDiRG4EBiBA4kRuBAYgQOJEbgQGIEDiRW5KKLX7f9\nJ9t/7m5d9Is6BgPQuyJvVf2PpC0R8Vn38slv2v5tRPyx4tkA9KjIRRdD0mfdmxd2P6LKoQCUo+jG\nB322JySdlPRKRLB1EbAMFAo8Ir6IiAFJ/ZI22v7OAvdh6yKgZZb0KnpEzEoak7S1kmkAlKrIq+hr\nbK/sfv4NSd+XdLjqwQD0rsir6FdIetx2n+b/h/DriHih2rEAlKHIq+h/0fye4ACWGd7JBiRG4EBi\nBA4kRuBAYgQOJEbgQGIEDiRG4EBibF3UYqOjo7WtNTAwUNtaIyMjta2VdfunojiCA4kROJAYgQOJ\nETiQGIEDiRE4kBiBA4kROJAYgQOJFQ68e230g7a5HhuwTCzlCH63pMmqBgFQvqI7m/RLulnS7mrH\nAVCmokfwUUn3SPqywlkAlKzIxge3SDoZEfsXuR97kwEtU+QIvknSrbanJD0laYvtJ869E3uTAe2z\naOARcV9E9EdER9J2Sa9GxG2VTwagZ/weHEhsSVd0iYgxze8uCmAZ4AgOJEbgQGIEDiRG4EBiBA4k\nRuBAYgQOJEbgQGJsXdRinU6ntrUmJiZqW6vObZLGxsZqW0uSNm/eXOt6i+EIDiRG4EBiBA4kRuBA\nYgQOJEbgQGIEDiRG4EBiBA4kVuidbN0rqn4q6QtJpyNisMqhAJRjKW9V/V5EnKpsEgCl4xQdSKxo\n4CHpd7b3295Z5UAAylP0FH1TRBy3/U1Jr9g+HBFvnH2Hbvg7JenKK68seUwA56PQETwijnf/e1LS\ns5I2LnAfti4CWqbI5oMX2770zOeSfijpnaoHA9C7Iqfo35L0rO0z9/9VRLxU6VQASrFo4BFxVNJ3\na5gFQMn4NRmQGIEDiRE4kBiBA4kROJAYgQOJETiQGIEDiS37rYtmZ2drW+v111+vbS1J+uijj2pb\na3R0tLa1Pv7449rWmpqaqm2tNuIIDiRG4EBiBA4kRuBAYgQOJEbgQGIEDiRG4EBiBA4kVihw2ytt\n77N92Pak7eurHgxA74q+VXWXpJci4se2L5K0osKZAJRk0cBtXybpBknDkhQRc5Lmqh0LQBmKnKJf\nLWlG0mO2D9re3b0+OoCWKxL4BZKuk/RQRGyQ9Lmke8+9k+2dtsdtj8/MzJQ8JoDzUSTwaUnTEfFW\n9/Y+zQf/P9i6CGifRQOPiA8kHbO9vvulGyW9W+lUAEpR9FX0uyTt7b6CflTSHdWNBKAshQKPiAlJ\ngxXPAqBkvJMNSIzAgcQIHEiMwIHECBxIjMCBxAgcSIzAgcQIHEiMvcmW4MEHH6xtrcy2bdtW21rD\nw8O1rdVGHMGBxAgcSIzAgcQIHEiMwIHECBxIjMCBxAgcSIzAgcQWDdz2etsTZ318YnukjuEA9GbR\nt6pGxHuSBiTJdp+kf0p6tuK5AJRgqafoN0r6e0T8o4phAJRrqYFvl/TkQt9g6yKgfQoH3t304FZJ\nv1no+2xdBLTPUo7gN0k6EBEnqhoGQLmWEvgO/Z/TcwDtVChw2ysk/UDSM9WOA6BMRfcm+7ekVRXP\nAqBkvJMNSIzAgcQIHEiMwIHECBxIjMCBxAgcSIzAgcQcEeX/UHtG0lL/pHS1pFOlD9MOWR8bj6s5\n346IRf+qq5LAz4ft8YgYbHqOKmR9bDyu9uMUHUiMwIHE2hT4w00PUKGsj43H1XKteQ4OoHxtOoID\nKFkrAre91fZ7to/Yvrfpecpge53t12xP2j5k++6mZyqT7T7bB22/0PQsZbK90vY+24e7/3bXNz1T\nLxo/Re9ea/1vmr9izLSktyXtiIh3Gx2sR7avkHRFRBywfamk/ZKGlvvjOsP2TyUNSrosIm5pep6y\n2H5c0u8jYnf3QqMrImK26bnOVxuO4BslHYmIoxExJ+kpSdsanqlnEfF+RBzofv6ppElJa5udqhy2\n+yXdLGl307OUyfZlkm6Q9IgkRcTcco5bakfgayUdO+v2tJKEcIbtjqQNkt5qdpLSjEq6R9KXTQ9S\nsqslzUh6rPv0Y7fti5seqhdtCNwLfC3NS/u2L5H0tKSRiPik6Xl6ZfsWSScjYn/Ts1TgAknXSXoo\nIjZI+lzSsn5NqA2BT0tad9btfknHG5qlVLYv1HzceyMiyxVpN0m61faU5p9ObbH9RLMjlWZa0nRE\nnDnT2qf54JetNgT+tqRrbF/VfVFju6TnG56pZ7at+edykxHxQNPzlCUi7ouI/ojoaP7f6tWIuK3h\nsUoRER9IOmZ7ffdLN0pa1i+KFrpscpUi4rTtOyW9LKlP0qMRcajhscqwSdLtkv5qe6L7tZ9HxIsN\nzoTF3SVpb/dgc1TSHQ3P05PGf00GoDptOEUHUBECBxIjcCAxAgcSI3AgMQIHEiNwIDECBxL7L984\nul/SE2VPAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x20a6568da90>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.imshow(digits.images[13], cmap=plt.cm.gray_r)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that we've got a better idea of what our data looks like, let's begin organizing it. An important concept in machine learning is overfitting. Overfitting typically occurs when a model has a large number of parameters, but not much data. An overfit model will perform very well on its training data, but will be unable to generalize beyond this training data.\n",
    "\n",
    "To prevent overfitting, we will split our dataset into two groups: training data and test data. This will allow us to evaluate how our model performs on data it's never seen before. This procedure is called cross-validation and is extremely important in machine learning."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\Tyler\\Anaconda3\\lib\\site-packages\\sklearn\\cross_validation.py:41: DeprecationWarning: This module was deprecated in version 0.18 in favor of the model_selection module into which all the refactored classes and functions are moved. Also note that the interface of the new CV iterators are different from that of this module. This module will be removed in 0.20.\n",
      "  \"This module will be removed in 0.20.\", DeprecationWarning)\n"
     ]
    }
   ],
   "source": [
    "from sklearn.cross_validation import train_test_split\n",
    "\n",
    "data_train, data_test, labels_train, labels_test = train_test_split(digits.data,\n",
    "                                                                    digits.target,\n",
    "                                                                    test_size = 0.3,\n",
    "                                                                    random_state = 42)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Video describing SVM: https://www.youtube.com/watch?v=mA5nwGoRAOo\n",
    "\n",
    "From here, creating and training our classifier is relatively straightforward."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "SVC(C=100, cache_size=200, class_weight=None, coef0=0.0,\n",
       "  decision_function_shape='ovr', degree=3, gamma=0.001, kernel='rbf',\n",
       "  max_iter=-1, probability=False, random_state=None, shrinking=True,\n",
       "  tol=0.001, verbose=False)"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "clf = svm.SVC(gamma=0.001, C=100)\n",
    "clf.fit(data_train, labels_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's compute the accuracty of our classifier using the test data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.990740740741\n"
     ]
    }
   ],
   "source": [
    "accuracy = clf.score(data_test, labels_test)\n",
    "print(accuracy)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Video describing C: https://www.youtube.com/watch?v=joTa_FeMZ2s\n",
    "\n",
    "Video describing gamma: https://www.youtube.com/watch?v=m2a2K4lprQw\n",
    "\n",
    "That's pretty impressive. So what are those mysterious values gamma and C? The C parameter controls the penalty for misclassification of each example in the training data. Large values of C highly penalize misclassification, and thus will fit to the training data more exactly. However, this can lead to overfitting and trouble with outliers, in which case a smaller value of C should be chosen.\n",
    "\n",
    "The gamma parameter is somewhat more complicated, but it can be understood to be the radius of influence of the individual support vectors. More info can be found in the sklearn SVM documentation: http://scikit-learn.org/stable/auto_examples/svm/plot_rbf_parameters.html\n",
    "\n",
    "So let's try changing the values of C and gamma and see what we get."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.731481481481\n"
     ]
    }
   ],
   "source": [
    "clf = svm.SVC(gamma=0.01, C=100)\n",
    "clf.fit(data_train, labels_train)\n",
    "accuracy = clf.score(data_test, labels_test)\n",
    "print(accuracy)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can automate the process of finding the best values of gamma and C (also known as hyperparameters) by using sklearn's built in grid-search function. Note that a new classifier must be built, trained, and evaluated for each combination of hyperparameters, so this process can be time consuming."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\Tyler\\Anaconda3\\lib\\site-packages\\sklearn\\grid_search.py:42: DeprecationWarning: This module was deprecated in version 0.18 in favor of the model_selection module into which all the refactored classes and functions are moved. This module will be removed in 0.20.\n",
      "  DeprecationWarning)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'C': 10, 'gamma': 0.001}\n"
     ]
    }
   ],
   "source": [
    "from sklearn.grid_search import GridSearchCV\n",
    "\n",
    "param_grid = {\"C\": [1,10,100], \"gamma\": [0.1, 0.01, 0.001]}\n",
    "\n",
    "gridsearch = GridSearchCV(svm.SVC(), param_grid)\n",
    "\n",
    "gridsearch.fit(data_test, labels_test)\n",
    "\n",
    "print(gridsearch.best_params_)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Kaggle: Making a Submission\n",
    "\n",
    "Kaggle is an excellent platform for participating in data science competitions, accessing cool datasets, and getting involved in the data science community. For part two of this workshop, we will be using Kaggle's Biological Response competition data to create a Kaggle submission entry. This is part of Kaggle's Getting Started with Python tutorial, which can be found here: https://www.kaggle.com/wiki/GettingStartedWithPythonForDataScience\n",
    "\n",
    "First let's check out the dataset, which can be found here: https://www.kaggle.com/c/bioresponse/data. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Activity</th>\n",
       "      <th>D1</th>\n",
       "      <th>D2</th>\n",
       "      <th>D3</th>\n",
       "      <th>D4</th>\n",
       "      <th>D5</th>\n",
       "      <th>D6</th>\n",
       "      <th>D7</th>\n",
       "      <th>D8</th>\n",
       "      <th>D9</th>\n",
       "      <th>...</th>\n",
       "      <th>D1767</th>\n",
       "      <th>D1768</th>\n",
       "      <th>D1769</th>\n",
       "      <th>D1770</th>\n",
       "      <th>D1771</th>\n",
       "      <th>D1772</th>\n",
       "      <th>D1773</th>\n",
       "      <th>D1774</th>\n",
       "      <th>D1775</th>\n",
       "      <th>D1776</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.497009</td>\n",
       "      <td>0.10</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.132956</td>\n",
       "      <td>0.678031</td>\n",
       "      <td>0.273166</td>\n",
       "      <td>0.585445</td>\n",
       "      <td>0.743663</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>0.366667</td>\n",
       "      <td>0.606291</td>\n",
       "      <td>0.05</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.111209</td>\n",
       "      <td>0.803455</td>\n",
       "      <td>0.106105</td>\n",
       "      <td>0.411754</td>\n",
       "      <td>0.836582</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>0.033300</td>\n",
       "      <td>0.480124</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.209791</td>\n",
       "      <td>0.610350</td>\n",
       "      <td>0.356453</td>\n",
       "      <td>0.517720</td>\n",
       "      <td>0.679051</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.538825</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.5</td>\n",
       "      <td>0.196344</td>\n",
       "      <td>0.724230</td>\n",
       "      <td>0.235606</td>\n",
       "      <td>0.288764</td>\n",
       "      <td>0.805110</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>0.100000</td>\n",
       "      <td>0.517794</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.494734</td>\n",
       "      <td>0.781422</td>\n",
       "      <td>0.154361</td>\n",
       "      <td>0.303809</td>\n",
       "      <td>0.812646</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 1777 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   Activity        D1        D2    D3   D4        D5        D6        D7  \\\n",
       "0         1  0.000000  0.497009  0.10  0.0  0.132956  0.678031  0.273166   \n",
       "1         1  0.366667  0.606291  0.05  0.0  0.111209  0.803455  0.106105   \n",
       "2         1  0.033300  0.480124  0.00  0.0  0.209791  0.610350  0.356453   \n",
       "3         1  0.000000  0.538825  0.00  0.5  0.196344  0.724230  0.235606   \n",
       "4         0  0.100000  0.517794  0.00  0.0  0.494734  0.781422  0.154361   \n",
       "\n",
       "         D8        D9  ...    D1767  D1768  D1769  D1770  D1771  D1772  D1773  \\\n",
       "0  0.585445  0.743663  ...        0      0      0      0      0      0      0   \n",
       "1  0.411754  0.836582  ...        1      1      1      1      0      1      0   \n",
       "2  0.517720  0.679051  ...        0      0      0      0      0      0      0   \n",
       "3  0.288764  0.805110  ...        0      0      0      0      0      0      0   \n",
       "4  0.303809  0.812646  ...        0      0      0      0      0      0      0   \n",
       "\n",
       "   D1774  D1775  D1776  \n",
       "0      0      0      0  \n",
       "1      0      1      0  \n",
       "2      0      0      0  \n",
       "3      0      0      0  \n",
       "4      0      0      0  \n",
       "\n",
       "[5 rows x 1777 columns]"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "\n",
    "data = pd.read_csv(\"BioResponseTrain.csv\")\n",
    "data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Each row in this dataframe represents a specific molecule, and the column labeled \"Activity\" denotes whether the molecule elicits a specific biological response, (1), or not, (0). The descriptors D1, D2, ..., D1773 quantitatively describe relevant structural and chemical properties of the molecule. The goal is to use these descriptors as features and predict the activity of new molecules.\n",
    "\n",
    "First, let's split our data into training and test cases. Since the first column represents our labels, or target values, we must be sure to separate them from our features."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "targets = data[\"Activity\"]\n",
    "features = data.iloc[:,1:]\n",
    "\n",
    "features_train, features_test, targets_train, targets_test = train_test_split(\n",
    "    features, targets, test_size = 0.3, random_state = 42)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this section, we'll be using a random forest classifier. The random forest classifier can be thought of as an ensemble of individual decision tree classifiers. These decision tree classifiers are trained to extract the most relevant features and make a series of decisions based on given input using these features. While decision trees are prone to overfitting by themselves, as part of an ensemble they form a robust model.\n",
    "\n",
    "\n",
    "https://upload.wikimedia.org/wikipedia/commons/f/f3/CART_tree_titanic_survivors.png\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "RandomForestClassifier(bootstrap=True, class_weight=None, criterion='gini',\n",
       "            max_depth=None, max_features='auto', max_leaf_nodes=None,\n",
       "            min_impurity_decrease=0.0, min_impurity_split=None,\n",
       "            min_samples_leaf=1, min_samples_split=2,\n",
       "            min_weight_fraction_leaf=0.0, n_estimators=100, n_jobs=1,\n",
       "            oob_score=False, random_state=None, verbose=0,\n",
       "            warm_start=False)"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.ensemble import RandomForestClassifier\n",
    "\n",
    "rf = RandomForestClassifier(n_estimators=100) # number of trees = 100\n",
    "rf.fit(features_train,targets_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's use this decision tree classifier to make some predictions on our test data, and see how it holds up. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "accuracy = rf.score(features_test, targets_test)\n",
    "print(accuracy)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And that's an intro to Machine learning in Python!"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}