Spring-Symposium-Python-1-Instructor.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Python Basics"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data Science Tutorial"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that we've covered some Python basics, we will begin a tutorial going through many tasks a data scientist may perform.  We will obtain real world data and go through the process of auditing, analyzing, visualing, and building classifiers from the data.\n",
    "\n",
    "We will use a database of selected disease statistics of various contries from the Global Health Observatory. The data is organized by country and year, with the number of specific incidents of each disease listed. The attributes and domain of each entry are described by the table below:\n",
    "\n",
    "| Attribute                     | Domain                          |\n",
    "|-------------------------------|---------------------------------|\n",
    "| 1. Country                    | String                          |\n",
    "| 2. Year                       | Year (2009-2014)                |\n",
    "| 3. T.b. gambiense             | Integer                         |\n",
    "| 4. T.b. rhodesiense           | Integer                         |\n",
    "| 5. Cholera                    | Integer                         |\n",
    "| 6. Meningitis (suspected)     | Integer                         |\n",
    "| 7. Congenital Rubella         | Integer                         |\n",
    "| 8. Diphtheria                 | Integer                         |\n",
    "| 9. Japanese encephalitis      | Integer                         |\n",
    "| 10. Leprosy                   | Integer                         |\n",
    "| 11. Malaria                   | Integer                         |\n",
    "| 12. Measles                   | Integer                         |\n",
    "| 13. Mumps                     | Integer                         |\n",
    "| 14. Neonatal Tetanus          | Integer                         |\n",
    "| 15. Pertussis                 | Integer                         |\n",
    "| 16. Plague                    | Integer                         |\n",
    "| 17. Poliomyelitis             | Integer                         |\n",
    "| 18. Rubella                   | Integer                         |\n",
    "| 19. Total Tetanus             | Integer                         |\n",
    "| 20. Tuberculosis              | Integer                         |\n",
    "| 21. Yellow Fever              | Integer                         |\n",
    "| 22. Cutaneous Leishmaniasis   | Integer                         |\n",
    "| 23. Visceral Leishmaniasis    | Integer                         |\n",
    "\n",
    "For more information on this data set:\n",
    "http://apps.who.int/gho/data/node.home\n",
    "\n",
    "##Obtaining the Data\n",
    "Lets begin by programmatically obtaining the data.  Here I'll define a function we can use to make HTTP requests and download the data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def download_file(url, local_filename):\n",
    "    import requests\n",
    "    \n",
    "    # stream = True allows downloading of large files; prevents loading entire file into memory\n",
    "    r = requests.get(url, stream = True)\n",
    "    with open(local_filename, 'wb') as f:\n",
    "        for chunk in r.iter_content(chunk_size=1024): \n",
    "            if chunk: # filter out keep-alive new chunks\n",
    "                f.write(chunk)\n",
    "                f.flush()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we'll specify the url of the file and the file name we will save to"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "url = 'https://raw.githubusercontent.com/dsiufl/Python-Workshops/master/GlobalHealthData.csv'\n",
    "filename = 'GlobalHealthData.csv'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And make a call to <code>download_file</code>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "download_file(url, filename)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Note:**  If you see an InsecurePlatformWarning message, ignore it. More info can be found here: https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning\n",
    "\n",
    "Now this might seem like overkill for downloading a single, small csv file, but we can use this same function to access countless APIs available on the World Wide Web by building an API request in the url.\n",
    "\n",
    "##Wrangling the Data\n",
    "Now that we have some data, lets get it into a useful form.  For this task we will use a package called pandas. pandas is an open source, BSD-licensed library providing high-performance, easy-to-use data structures and data analysis tools for Python.  The most fundamental data structure in pandas is the dataframe, which is similar to the data.frame data structure found in the R statistical programming language.\n",
    "\n",
    "For more information: http://pandas.pydata.org\n",
    "\n",
    "pandas dataframes are a 2-dimensional labeled data structures with columns of potentially different types.  Dataframes can be thought of as similar to a spreadsheet or SQL table.\n",
    "\n",
    "There are numerous ways to build a dataframe with pandas.  Since we have already attained a csv file, we can use a parser built into pandas called <code>read_csv</code> which will read the contents of a csv file directly into a data frame.\n",
    "\n",
    "For more information: http://pandas.pydata.org/pandas-docs/dev/generated/pandas.io.parsers.read_csv.html"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Country</th>\n",
       "      <th>Year</th>\n",
       "      <th>Number of new reported cases of human African trypanosomiasis (T.b. gambiense)</th>\n",
       "      <th>Number of new reported cases of human African trypanosomiasis (T.b. rhodesiense)</th>\n",
       "      <th>Number of reported cases of cholera</th>\n",
       "      <th>Number of suspected meningitis cases reported</th>\n",
       "      <th>Congenital Rubella Syndrome - number of reported cases</th>\n",
       "      <th>Diphtheria - number of reported cases</th>\n",
       "      <th>Japanese encephalitis - number of reported cases</th>\n",
       "      <th>Leprosy - number of reported cases</th>\n",
       "      <th>...</th>\n",
       "      <th>Neonatal tetanus - number of reported cases</th>\n",
       "      <th>Pertussis - number of reported cases</th>\n",
       "      <th>Plague - number of reported cases</th>\n",
       "      <th>Poliomyelitis - number of reported cases</th>\n",
       "      <th>Rubella - number of reported cases</th>\n",
       "      <th>Total tetanus - number of reported cases</th>\n",
       "      <th>Tuberculosis - new and relapse cases</th>\n",
       "      <th>Yellow fever - number of reported cases</th>\n",
       "      <th>Number of cases of cutaneous leishmaniasis reported</th>\n",
       "      <th>Number of cases of visceral leishmaniasis reported</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Afghanistan</td>\n",
       "      <td>2014</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>17</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>28.0</td>\n",
       "      <td>43</td>\n",
       "      <td>39</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Afghanistan</td>\n",
       "      <td>2013</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3957.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>39</td>\n",
       "      <td>...</td>\n",
       "      <td>13</td>\n",
       "      <td>371</td>\n",
       "      <td>NaN</td>\n",
       "      <td>17.0</td>\n",
       "      <td>367</td>\n",
       "      <td>24</td>\n",
       "      <td>30507.0</td>\n",
       "      <td>0</td>\n",
       "      <td>23621</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Afghanistan</td>\n",
       "      <td>2012</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>12.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>37</td>\n",
       "      <td>...</td>\n",
       "      <td>21</td>\n",
       "      <td>1497</td>\n",
       "      <td>NaN</td>\n",
       "      <td>37.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>37</td>\n",
       "      <td>28679.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>33894</td>\n",
       "      <td>24</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Afghanistan</td>\n",
       "      <td>2011</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3733.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>50</td>\n",
       "      <td>...</td>\n",
       "      <td>20</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>80.0</td>\n",
       "      <td>750</td>\n",
       "      <td>20</td>\n",
       "      <td>27983.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>31293</td>\n",
       "      <td>21</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Afghanistan</td>\n",
       "      <td>2010</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2369.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>51</td>\n",
       "      <td>...</td>\n",
       "      <td>23</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>25.0</td>\n",
       "      <td>46</td>\n",
       "      <td>23</td>\n",
       "      <td>28029.0</td>\n",
       "      <td>0</td>\n",
       "      <td>32145</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 23 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       Country  Year  \\\n",
       "0  Afghanistan  2014   \n",
       "1  Afghanistan  2013   \n",
       "2  Afghanistan  2012   \n",
       "3  Afghanistan  2011   \n",
       "4  Afghanistan  2010   \n",
       "\n",
       "  Number of new reported cases of human African trypanosomiasis (T.b. gambiense)  \\\n",
       "0                                                NaN                               \n",
       "1                                                NaN                               \n",
       "2                                                NaN                               \n",
       "3                                                NaN                               \n",
       "4                                                NaN                               \n",
       "\n",
       "  Number of new reported cases of human African trypanosomiasis (T.b. rhodesiense)  \\\n",
       "0                                                NaN                                 \n",
       "1                                                NaN                                 \n",
       "2                                                NaN                                 \n",
       "3                                                NaN                                 \n",
       "4                                                NaN                                 \n",
       "\n",
       "   Number of reported cases of cholera  \\\n",
       "0                                  NaN   \n",
       "1                               3957.0   \n",
       "2                                 12.0   \n",
       "3                               3733.0   \n",
       "4                               2369.0   \n",
       "\n",
       "  Number of suspected meningitis cases reported  \\\n",
       "0                                           NaN   \n",
       "1                                           NaN   \n",
       "2                                           NaN   \n",
       "3                                           NaN   \n",
       "4                                           NaN   \n",
       "\n",
       "   Congenital Rubella Syndrome - number of reported cases  \\\n",
       "0                                                0.0        \n",
       "1                                                0.0        \n",
       "2                                                NaN        \n",
       "3                                                NaN        \n",
       "4                                                0.0        \n",
       "\n",
       "   Diphtheria - number of reported cases  \\\n",
       "0                                    0.0   \n",
       "1                                    0.0   \n",
       "2                                    0.0   \n",
       "3                                    0.0   \n",
       "4                                    0.0   \n",
       "\n",
       "   Japanese encephalitis - number of reported cases  \\\n",
       "0                                               NaN   \n",
       "1                                               0.0   \n",
       "2                                               NaN   \n",
       "3                                               NaN   \n",
       "4                                               0.0   \n",
       "\n",
       "  Leprosy - number of reported cases  \\\n",
       "0                                NaN   \n",
       "1                                 39   \n",
       "2                                 37   \n",
       "3                                 50   \n",
       "4                                 51   \n",
       "\n",
       "                          ...                          \\\n",
       "0                         ...                           \n",
       "1                         ...                           \n",
       "2                         ...                           \n",
       "3                         ...                           \n",
       "4                         ...                           \n",
       "\n",
       "   Neonatal tetanus - number of reported cases  \\\n",
       "0                                           17   \n",
       "1                                           13   \n",
       "2                                           21   \n",
       "3                                           20   \n",
       "4                                           23   \n",
       "\n",
       "   Pertussis - number of reported cases Plague - number of reported cases  \\\n",
       "0                                     0                               NaN   \n",
       "1                                   371                               NaN   \n",
       "2                                  1497                               NaN   \n",
       "3                                     0                               NaN   \n",
       "4                                     0                               NaN   \n",
       "\n",
       "  Poliomyelitis - number of reported cases Rubella - number of reported cases  \\\n",
       "0                                     28.0                                 43   \n",
       "1                                     17.0                                367   \n",
       "2                                     37.0                                NaN   \n",
       "3                                     80.0                                750   \n",
       "4                                     25.0                                 46   \n",
       "\n",
       "   Total tetanus - number of reported cases  \\\n",
       "0                                        39   \n",
       "1                                        24   \n",
       "2                                        37   \n",
       "3                                        20   \n",
       "4                                        23   \n",
       "\n",
       "   Tuberculosis - new and relapse cases  \\\n",
       "0                                   NaN   \n",
       "1                               30507.0   \n",
       "2                               28679.0   \n",
       "3                               27983.0   \n",
       "4                               28029.0   \n",
       "\n",
       "  Yellow fever - number of reported cases  \\\n",
       "0                                     NaN   \n",
       "1                                       0   \n",
       "2                                     NaN   \n",
       "3                                     NaN   \n",
       "4                                       0   \n",
       "\n",
       "  Number of cases of cutaneous leishmaniasis reported  \\\n",
       "0                                                NaN    \n",
       "1                                              23621    \n",
       "2                                              33894    \n",
       "3                                              31293    \n",
       "4                                              32145    \n",
       "\n",
       "   Number of cases of visceral leishmaniasis reported  \n",
       "0                                                NaN   \n",
       "1                                                 16   \n",
       "2                                                 24   \n",
       "3                                                 21   \n",
       "4                                                 11   \n",
       "\n",
       "[5 rows x 23 columns]"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd # import the module and alias it as pd\n",
    "\n",
    "health_data = pd.read_csv('GlobalHealthData.csv')\n",
    "health_data.head() # show the first few rows of the data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lets take a look at some simple statistics for the **Cholera** column"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "count       245.000000\n",
       "mean       6134.971429\n",
       "std       26976.057407\n",
       "min           1.000000\n",
       "25%                NaN\n",
       "50%                NaN\n",
       "75%                NaN\n",
       "max      340311.000000\n",
       "Name: Number of reported cases of cholera, dtype: float64"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "health_data[\"Number of reported cases of cholera\"].describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Referring to the documentation, the data contains 1164 entries. However, if we take a look at the \"count\" section, it shows only 245 entries. This is because the original data is filled with empty strings, which pandas automatically converts to Numpy's <code>nan</code> datatype, or \"Not a Number\". \n",
    "\n",
    "Lets take a look at another column, this time **Pertussis**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "count     972\n",
       "unique    374\n",
       "top         0\n",
       "freq      333\n",
       "Name: Pertussis - number of reported cases, dtype: object"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "health_data[\"Pertussis - number of reported cases\"].describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Well at least the name is correct.  We were expecting a mean and standard deviation, and now the data type is an object.  \n",
    "\n",
    "Whats up with our data?\n",
    "\n",
    "We have arrived at arguably the most important part of performing data science: dealing with messy data.  One of most important tools in a data scientist's toolbox is the ability to audit, clean, and reshape data.  The real world is full of messy data and your sources may not always have data in the exact format you desire.\n",
    "\n",
    "In this case we are working with csv data, which is a relatively straightforward format, but this will not always be the case when performing real world data science.  Data comes in all varieties from csv all the way to something as unstructured as a collection of emails or documents.  A data scientist must be versed in a wide variety of technologies and methodologies in order to be successful.\n",
    "\n",
    "Now, lets do a little bit of digging into why were are not getting a numeric pandas column"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['0', '371', '1497', nan, '6', '16', '4', '10', '69', '104', '1',\n",
       "       '3', '1259', '1554', '2539', '1127', '561', '1112', '1239', '1594',\n",
       "       '804', '1743', '85', '30', '8', '11', '11842', '12319', '23855',\n",
       "       '38040', '34285', '29545', '571', '309', '414', '183', '18', '27',\n",
       "       '15', '5', '12', '13', '44', '17', '378', '188', '576', '151',\n",
       "       '112', '93', '1501', '1142', '548', '103', '133', '53', '75', '31',\n",
       "       '19', '25', '7687', '5211', '5400', '2257', '477', '1037', '52',\n",
       "       '89', '102', '46', '54', '251', '7', '68', '372', '513', '1527',\n",
       "       '1261', '4845', '676', '759', '1667', '241', '124', '100', '63',\n",
       "       '1964', '5762', '2582', '794', '692', '3408', '1712', '2183',\n",
       "       '2517', '1764', '1612', '734', '13682', '3289', '1010', '344',\n",
       "       '407', '21', '137', '58', '130', '79', '71', '664', '108', '61',\n",
       "       '42', '101', '9', '2', '2521', '1233', '738', '324', '662', '956',\n",
       "       '3108', '3407', '2452', '2157', '830', '855', '484', '980', '77',\n",
       "       '83', '98', '37', '35', '41', '208', '91', '43', '55', '149', '478',\n",
       "       '1295', '629', '192', '555', '336', '267', '165', '94', '115',\n",
       "       '346', '33', '49', '12260', '28', '56', '64', '34', '273', '48',\n",
       "       '113', '135', '127', '20', '36', '46706', '31089', '44154', '39091',\n",
       "       '40508', '60385', '2976', '1941', '973', '1415', '1329', '650',\n",
       "       '464', '590', '1606', '2471', '2314', '2019', '2034', '5471', '73',\n",
       "       '175', '458', '229', '107', '80', '1509', '1394', '2730', '2345',\n",
       "       '1244', '1182', '225', '302', '187', '315', '2066', '1663', '4087',\n",
       "       '4395', '5406', '5208', '23', '14', '66', '39', '90', '62', '109',\n",
       "       '224', '60', '412', '38', '81', '201', '257', '65', '197', '143',\n",
       "       '154', '233', '369', '255', '500', '222', '217', '248', '955',\n",
       "       '961', '978', '252', '559', '0 0', '72', '172', '6096', '3431',\n",
       "       '1595', '1733', '2293', '2357', '8960', '2858', '13552', '6726',\n",
       "       '2912', '6202', '1099', '3396', '5598', '1992', '462', '656', '87',\n",
       "       '24', '884', '983', '1354', '9559', '8530', '11628', '11281',\n",
       "       '3032', '2608', '4231', '3695', '3565', '5546', '88', '32', '250',\n",
       "       '156', '164', '47', '22', '3188', '6472', '4625', '4949', '2932',\n",
       "       '236', '1373', '1173', '254', '4682', '1266', '2390', '76', '230',\n",
       "       '97', '92', '57', '82', '86', '29', '4705', '4510', '7220', '4733',\n",
       "       '4795', '4056', '281', '51', '5 5', '2 2', '1123', '907', '950',\n",
       "       '936', '1379', '288', '399', '169', '178', '284', '610', '442',\n",
       "       '1001', '3784', '3028', '665', '823', '116', '181', '2342', '3439',\n",
       "       '3088', '739', '393', '67', '141', '119', '132', '209', '703',\n",
       "       '237', '289', '177', '266', '4700', '5900', '3700', '70', '145',\n",
       "       '242', '2286', '2937', '1067', '4046', '6090', '11980', '1243',\n",
       "       '518', '846', '28532', '47693', '18610', '27410', '320', '598',\n",
       "       '639', '105', '122', '2529', '4699', '2280', '2534', '2500', '681'], dtype=object)"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "health_data[\"Pertussis - number of reported cases\"].unique()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using <code>unique</code> we can see that '0 0', '5 5', and '2 2' all appear as distinct values in this series. Because of the space between the numbers, Python has classified these as *strings* rather than *integers*. Indeed, it's not immediately obvious that these were meant to be legitimate entries in the first place.\n",
    "\n",
    "Lets see what we can do with these unrecognized values. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "health_data[\"Pertussis - number of reported cases\"] = pd.to_numeric(health_data[\"Pertussis - number of reported cases\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we have attempted to convert the **Pertussis** series to a numeric type.  Lets see what the unique values are now."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([  0.00000000e+00,   3.71000000e+02,   1.49700000e+03,\n",
       "                    nan,   6.00000000e+00,   1.60000000e+01,\n",
       "         4.00000000e+00,   1.00000000e+01,   6.90000000e+01,\n",
       "         1.04000000e+02,   1.00000000e+00,   3.00000000e+00,\n",
       "         1.25900000e+03,   1.55400000e+03,   2.53900000e+03,\n",
       "         1.12700000e+03,   5.61000000e+02,   1.11200000e+03,\n",
       "         1.23900000e+03,   1.59400000e+03,   8.04000000e+02,\n",
       "         1.74300000e+03,   8.50000000e+01,   3.00000000e+01,\n",
       "         8.00000000e+00,   1.10000000e+01,   1.18420000e+04,\n",
       "         1.23190000e+04,   2.38550000e+04,   3.80400000e+04,\n",
       "         3.42850000e+04,   2.95450000e+04,   5.71000000e+02,\n",
       "         3.09000000e+02,   4.14000000e+02,   1.83000000e+02,\n",
       "         1.80000000e+01,   2.70000000e+01,   1.50000000e+01,\n",
       "         5.00000000e+00,   1.20000000e+01,   1.30000000e+01,\n",
       "         4.40000000e+01,   1.70000000e+01,   3.78000000e+02,\n",
       "         1.88000000e+02,   5.76000000e+02,   1.51000000e+02,\n",
       "         1.12000000e+02,   9.30000000e+01,   1.50100000e+03,\n",
       "         1.14200000e+03,   5.48000000e+02,   1.03000000e+02,\n",
       "         1.33000000e+02,   5.30000000e+01,   7.50000000e+01,\n",
       "         3.10000000e+01,   1.90000000e+01,   2.50000000e+01,\n",
       "         7.68700000e+03,   5.21100000e+03,   5.40000000e+03,\n",
       "         2.25700000e+03,   4.77000000e+02,   1.03700000e+03,\n",
       "         5.20000000e+01,   8.90000000e+01,   1.02000000e+02,\n",
       "         4.60000000e+01,   5.40000000e+01,   2.51000000e+02,\n",
       "         7.00000000e+00,   6.80000000e+01,   3.72000000e+02,\n",
       "         5.13000000e+02,   1.52700000e+03,   1.26100000e+03,\n",
       "         4.84500000e+03,   6.76000000e+02,   7.59000000e+02,\n",
       "         1.66700000e+03,   2.41000000e+02,   1.24000000e+02,\n",
       "         1.00000000e+02,   6.30000000e+01,   1.96400000e+03,\n",
       "         5.76200000e+03,   2.58200000e+03,   7.94000000e+02,\n",
       "         6.92000000e+02,   3.40800000e+03,   1.71200000e+03,\n",
       "         2.18300000e+03,   2.51700000e+03,   1.76400000e+03,\n",
       "         1.61200000e+03,   7.34000000e+02,   1.36820000e+04,\n",
       "         3.28900000e+03,   1.01000000e+03,   3.44000000e+02,\n",
       "         4.07000000e+02,   2.10000000e+01,   1.37000000e+02,\n",
       "         5.80000000e+01,   1.30000000e+02,   7.90000000e+01,\n",
       "         7.10000000e+01,   6.64000000e+02,   1.08000000e+02,\n",
       "         6.10000000e+01,   4.20000000e+01,   1.01000000e+02,\n",
       "         9.00000000e+00,   2.00000000e+00,   2.52100000e+03,\n",
       "         1.23300000e+03,   7.38000000e+02,   3.24000000e+02,\n",
       "         6.62000000e+02,   9.56000000e+02,   3.10800000e+03,\n",
       "         3.40700000e+03,   2.45200000e+03,   2.15700000e+03,\n",
       "         8.30000000e+02,   8.55000000e+02,   4.84000000e+02,\n",
       "         9.80000000e+02,   7.70000000e+01,   8.30000000e+01,\n",
       "         9.80000000e+01,   3.70000000e+01,   3.50000000e+01,\n",
       "         4.10000000e+01,   2.08000000e+02,   9.10000000e+01,\n",
       "         4.30000000e+01,   5.50000000e+01,   1.49000000e+02,\n",
       "         4.78000000e+02,   1.29500000e+03,   6.29000000e+02,\n",
       "         1.92000000e+02,   5.55000000e+02,   3.36000000e+02,\n",
       "         2.67000000e+02,   1.65000000e+02,   9.40000000e+01,\n",
       "         1.15000000e+02,   3.46000000e+02,   3.30000000e+01,\n",
       "         4.90000000e+01,   1.22600000e+04,   2.80000000e+01,\n",
       "         5.60000000e+01,   6.40000000e+01,   3.40000000e+01,\n",
       "         2.73000000e+02,   4.80000000e+01,   1.13000000e+02,\n",
       "         1.35000000e+02,   1.27000000e+02,   2.00000000e+01,\n",
       "         3.60000000e+01,   4.67060000e+04,   3.10890000e+04,\n",
       "         4.41540000e+04,   3.90910000e+04,   4.05080000e+04,\n",
       "         6.03850000e+04,   2.97600000e+03,   1.94100000e+03,\n",
       "         9.73000000e+02,   1.41500000e+03,   1.32900000e+03,\n",
       "         6.50000000e+02,   4.64000000e+02,   5.90000000e+02,\n",
       "         1.60600000e+03,   2.47100000e+03,   2.31400000e+03,\n",
       "         2.01900000e+03,   2.03400000e+03,   5.47100000e+03,\n",
       "         7.30000000e+01,   1.75000000e+02,   4.58000000e+02,\n",
       "         2.29000000e+02,   1.07000000e+02,   8.00000000e+01,\n",
       "         1.50900000e+03,   1.39400000e+03,   2.73000000e+03,\n",
       "         2.34500000e+03,   1.24400000e+03,   1.18200000e+03,\n",
       "         2.25000000e+02,   3.02000000e+02,   1.87000000e+02,\n",
       "         3.15000000e+02,   2.06600000e+03,   1.66300000e+03,\n",
       "         4.08700000e+03,   4.39500000e+03,   5.40600000e+03,\n",
       "         5.20800000e+03,   2.30000000e+01,   1.40000000e+01,\n",
       "         6.60000000e+01,   3.90000000e+01,   9.00000000e+01,\n",
       "         6.20000000e+01,   1.09000000e+02,   2.24000000e+02,\n",
       "         6.00000000e+01,   4.12000000e+02,   3.80000000e+01,\n",
       "         8.10000000e+01,   2.01000000e+02,   2.57000000e+02,\n",
       "         6.50000000e+01,   1.97000000e+02,   1.43000000e+02,\n",
       "         1.54000000e+02,   2.33000000e+02,   3.69000000e+02,\n",
       "         2.55000000e+02,   5.00000000e+02,   2.22000000e+02,\n",
       "         2.17000000e+02,   2.48000000e+02,   9.55000000e+02,\n",
       "         9.61000000e+02,   9.78000000e+02,   2.52000000e+02,\n",
       "         5.59000000e+02,   7.20000000e+01,   1.72000000e+02,\n",
       "         6.09600000e+03,   3.43100000e+03,   1.59500000e+03,\n",
       "         1.73300000e+03,   2.29300000e+03,   2.35700000e+03,\n",
       "         8.96000000e+03,   2.85800000e+03,   1.35520000e+04,\n",
       "         6.72600000e+03,   2.91200000e+03,   6.20200000e+03,\n",
       "         1.09900000e+03,   3.39600000e+03,   5.59800000e+03,\n",
       "         1.99200000e+03,   4.62000000e+02,   6.56000000e+02,\n",
       "         8.70000000e+01,   2.40000000e+01,   8.84000000e+02,\n",
       "         9.83000000e+02,   1.35400000e+03,   9.55900000e+03,\n",
       "         8.53000000e+03,   1.16280000e+04,   1.12810000e+04,\n",
       "         3.03200000e+03,   2.60800000e+03,   4.23100000e+03,\n",
       "         3.69500000e+03,   3.56500000e+03,   5.54600000e+03,\n",
       "         8.80000000e+01,   3.20000000e+01,   2.50000000e+02,\n",
       "         1.56000000e+02,   1.64000000e+02,   4.70000000e+01,\n",
       "         2.20000000e+01,   3.18800000e+03,   6.47200000e+03,\n",
       "         4.62500000e+03,   4.94900000e+03,   2.93200000e+03,\n",
       "         2.36000000e+02,   1.37300000e+03,   1.17300000e+03,\n",
       "         2.54000000e+02,   4.68200000e+03,   1.26600000e+03,\n",
       "         2.39000000e+03,   7.60000000e+01,   2.30000000e+02,\n",
       "         9.70000000e+01,   9.20000000e+01,   5.70000000e+01,\n",
       "         8.20000000e+01,   8.60000000e+01,   2.90000000e+01,\n",
       "         4.70500000e+03,   4.51000000e+03,   7.22000000e+03,\n",
       "         4.73300000e+03,   4.79500000e+03,   4.05600000e+03,\n",
       "         2.81000000e+02,   5.10000000e+01,   1.12300000e+03,\n",
       "         9.07000000e+02,   9.50000000e+02,   9.36000000e+02,\n",
       "         1.37900000e+03,   2.88000000e+02,   3.99000000e+02,\n",
       "         1.69000000e+02,   1.78000000e+02,   2.84000000e+02,\n",
       "         6.10000000e+02,   4.42000000e+02,   1.00100000e+03,\n",
       "         3.78400000e+03,   3.02800000e+03,   6.65000000e+02,\n",
       "         8.23000000e+02,   1.16000000e+02,   1.81000000e+02,\n",
       "         2.34200000e+03,   3.43900000e+03,   3.08800000e+03,\n",
       "         7.39000000e+02,   3.93000000e+02,   6.70000000e+01,\n",
       "         1.41000000e+02,   1.19000000e+02,   1.32000000e+02,\n",
       "         2.09000000e+02,   7.03000000e+02,   2.37000000e+02,\n",
       "         2.89000000e+02,   1.77000000e+02,   2.66000000e+02,\n",
       "         4.70000000e+03,   5.90000000e+03,   3.70000000e+03,\n",
       "         7.00000000e+01,   1.45000000e+02,   2.42000000e+02,\n",
       "         2.28600000e+03,   2.93700000e+03,   1.06700000e+03,\n",
       "         4.04600000e+03,   6.09000000e+03,   1.19800000e+04,\n",
       "         1.24300000e+03,   5.18000000e+02,   8.46000000e+02,\n",
       "         2.85320000e+04,   4.76930000e+04,   1.86100000e+04,\n",
       "         2.74100000e+04,   3.20000000e+02,   5.98000000e+02,\n",
       "         6.39000000e+02,   1.05000000e+02,   1.22000000e+02,\n",
       "         2.52900000e+03,   4.69900000e+03,   2.28000000e+03,\n",
       "         2.53400000e+03,   2.50000000e+03,   6.81000000e+02])"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "health_data[\"Pertussis - number of reported cases\"].unique()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The decimal point after each number means that it is an integer value being represented by a floating point number.  Now instead of our pesky *strings* we have <code>nan</code> (not a number).  <code>nan</code> is a construct used by pandas to represent the absence of value.  It is a data type that comes from the package numpy, used internally by pandas, and is not part of the standard Python library.\n",
    "\n",
    "Now that we have <code>nan</code> values in place of strings, we can use some nice features in pandas to deal with these missing values.\n",
    "\n",
    "What we are about to do is what is called \"imputing\" or providing a replacement for missing values so the data set becomes easier to work with.  There are a number of strategies for imputing missing values, all with their own pitfalls.  In general, imputation introduces some degree of bias to the data, so the imputation strategy taken should be in an attempt to minimize that bias.\n",
    "\n",
    "Here, we will simply ignore all of the <code>nan</code> values, however other strategies such as replacing the <code>nan</code>'s with the mean of the data are also commonly used."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Applications/anaconda/lib/python2.7/site-packages/ipykernel/__main__.py:1: FutureWarning: convert_objects is deprecated.  Use the data-type specific converters pd.to_datetime, pd.to_timedelta and pd.to_numeric.\n",
      "  if __name__ == '__main__':\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "array([  0.00000000e+00,   3.71000000e+02,   1.49700000e+03,\n",
       "                    nan,   6.00000000e+00,   1.60000000e+01,\n",
       "         4.00000000e+00,   1.00000000e+01,   6.90000000e+01,\n",
       "         1.04000000e+02,   1.00000000e+00,   3.00000000e+00,\n",
       "         1.25900000e+03,   1.55400000e+03,   2.53900000e+03,\n",
       "         1.12700000e+03,   5.61000000e+02,   1.11200000e+03,\n",
       "         1.23900000e+03,   1.59400000e+03,   8.04000000e+02,\n",
       "         1.74300000e+03,   8.50000000e+01,   3.00000000e+01,\n",
       "         8.00000000e+00,   1.10000000e+01,   1.18420000e+04,\n",
       "         1.23190000e+04,   2.38550000e+04,   3.80400000e+04,\n",
       "         3.42850000e+04,   2.95450000e+04,   5.71000000e+02,\n",
       "         3.09000000e+02,   4.14000000e+02,   1.83000000e+02,\n",
       "         1.80000000e+01,   2.70000000e+01,   1.50000000e+01,\n",
       "         5.00000000e+00,   1.20000000e+01,   1.30000000e+01,\n",
       "         4.40000000e+01,   1.70000000e+01,   3.78000000e+02,\n",
       "         1.88000000e+02,   5.76000000e+02,   1.51000000e+02,\n",
       "         1.12000000e+02,   9.30000000e+01,   1.50100000e+03,\n",
       "         1.14200000e+03,   5.48000000e+02,   1.03000000e+02,\n",
       "         1.33000000e+02,   5.30000000e+01,   7.50000000e+01,\n",
       "         3.10000000e+01,   1.90000000e+01,   2.50000000e+01,\n",
       "         7.68700000e+03,   5.21100000e+03,   5.40000000e+03,\n",
       "         2.25700000e+03,   4.77000000e+02,   1.03700000e+03,\n",
       "         5.20000000e+01,   8.90000000e+01,   1.02000000e+02,\n",
       "         4.60000000e+01,   5.40000000e+01,   2.51000000e+02,\n",
       "         7.00000000e+00,   6.80000000e+01,   3.72000000e+02,\n",
       "         5.13000000e+02,   1.52700000e+03,   1.26100000e+03,\n",
       "         4.84500000e+03,   6.76000000e+02,   7.59000000e+02,\n",
       "         1.66700000e+03,   2.41000000e+02,   1.24000000e+02,\n",
       "         1.00000000e+02,   6.30000000e+01,   1.96400000e+03,\n",
       "         5.76200000e+03,   2.58200000e+03,   7.94000000e+02,\n",
       "         6.92000000e+02,   3.40800000e+03,   1.71200000e+03,\n",
       "         2.18300000e+03,   2.51700000e+03,   1.76400000e+03,\n",
       "         1.61200000e+03,   7.34000000e+02,   1.36820000e+04,\n",
       "         3.28900000e+03,   1.01000000e+03,   3.44000000e+02,\n",
       "         4.07000000e+02,   2.10000000e+01,   1.37000000e+02,\n",
       "         5.80000000e+01,   1.30000000e+02,   7.90000000e+01,\n",
       "         7.10000000e+01,   6.64000000e+02,   1.08000000e+02,\n",
       "         6.10000000e+01,   4.20000000e+01,   1.01000000e+02,\n",
       "         9.00000000e+00,   2.00000000e+00,   2.52100000e+03,\n",
       "         1.23300000e+03,   7.38000000e+02,   3.24000000e+02,\n",
       "         6.62000000e+02,   9.56000000e+02,   3.10800000e+03,\n",
       "         3.40700000e+03,   2.45200000e+03,   2.15700000e+03,\n",
       "         8.30000000e+02,   8.55000000e+02,   4.84000000e+02,\n",
       "         9.80000000e+02,   7.70000000e+01,   8.30000000e+01,\n",
       "         9.80000000e+01,   3.70000000e+01,   3.50000000e+01,\n",
       "         4.10000000e+01,   2.08000000e+02,   9.10000000e+01,\n",
       "         4.30000000e+01,   5.50000000e+01,   1.49000000e+02,\n",
       "         4.78000000e+02,   1.29500000e+03,   6.29000000e+02,\n",
       "         1.92000000e+02,   5.55000000e+02,   3.36000000e+02,\n",
       "         2.67000000e+02,   1.65000000e+02,   9.40000000e+01,\n",
       "         1.15000000e+02,   3.46000000e+02,   3.30000000e+01,\n",
       "         4.90000000e+01,   1.22600000e+04,   2.80000000e+01,\n",
       "         5.60000000e+01,   6.40000000e+01,   3.40000000e+01,\n",
       "         2.73000000e+02,   4.80000000e+01,   1.13000000e+02,\n",
       "         1.35000000e+02,   1.27000000e+02,   2.00000000e+01,\n",
       "         3.60000000e+01,   4.67060000e+04,   3.10890000e+04,\n",
       "         4.41540000e+04,   3.90910000e+04,   4.05080000e+04,\n",
       "         6.03850000e+04,   2.97600000e+03,   1.94100000e+03,\n",
       "         9.73000000e+02,   1.41500000e+03,   1.32900000e+03,\n",
       "         6.50000000e+02,   4.64000000e+02,   5.90000000e+02,\n",
       "         1.60600000e+03,   2.47100000e+03,   2.31400000e+03,\n",
       "         2.01900000e+03,   2.03400000e+03,   5.47100000e+03,\n",
       "         7.30000000e+01,   1.75000000e+02,   4.58000000e+02,\n",
       "         2.29000000e+02,   1.07000000e+02,   8.00000000e+01,\n",
       "         1.50900000e+03,   1.39400000e+03,   2.73000000e+03,\n",
       "         2.34500000e+03,   1.24400000e+03,   1.18200000e+03,\n",
       "         2.25000000e+02,   3.02000000e+02,   1.87000000e+02,\n",
       "         3.15000000e+02,   2.06600000e+03,   1.66300000e+03,\n",
       "         4.08700000e+03,   4.39500000e+03,   5.40600000e+03,\n",
       "         5.20800000e+03,   2.30000000e+01,   1.40000000e+01,\n",
       "         6.60000000e+01,   3.90000000e+01,   9.00000000e+01,\n",
       "         6.20000000e+01,   1.09000000e+02,   2.24000000e+02,\n",
       "         6.00000000e+01,   4.12000000e+02,   3.80000000e+01,\n",
       "         8.10000000e+01,   2.01000000e+02,   2.57000000e+02,\n",
       "         6.50000000e+01,   1.97000000e+02,   1.43000000e+02,\n",
       "         1.54000000e+02,   2.33000000e+02,   3.69000000e+02,\n",
       "         2.55000000e+02,   5.00000000e+02,   2.22000000e+02,\n",
       "         2.17000000e+02,   2.48000000e+02,   9.55000000e+02,\n",
       "         9.61000000e+02,   9.78000000e+02,   2.52000000e+02,\n",
       "         5.59000000e+02,   7.20000000e+01,   1.72000000e+02,\n",
       "         6.09600000e+03,   3.43100000e+03,   1.59500000e+03,\n",
       "         1.73300000e+03,   2.29300000e+03,   2.35700000e+03,\n",
       "         8.96000000e+03,   2.85800000e+03,   1.35520000e+04,\n",
       "         6.72600000e+03,   2.91200000e+03,   6.20200000e+03,\n",
       "         1.09900000e+03,   3.39600000e+03,   5.59800000e+03,\n",
       "         1.99200000e+03,   4.62000000e+02,   6.56000000e+02,\n",
       "         8.70000000e+01,   2.40000000e+01,   8.84000000e+02,\n",
       "         9.83000000e+02,   1.35400000e+03,   9.55900000e+03,\n",
       "         8.53000000e+03,   1.16280000e+04,   1.12810000e+04,\n",
       "         3.03200000e+03,   2.60800000e+03,   4.23100000e+03,\n",
       "         3.69500000e+03,   3.56500000e+03,   5.54600000e+03,\n",
       "         8.80000000e+01,   3.20000000e+01,   2.50000000e+02,\n",
       "         1.56000000e+02,   1.64000000e+02,   4.70000000e+01,\n",
       "         2.20000000e+01,   3.18800000e+03,   6.47200000e+03,\n",
       "         4.62500000e+03,   4.94900000e+03,   2.93200000e+03,\n",
       "         2.36000000e+02,   1.37300000e+03,   1.17300000e+03,\n",
       "         2.54000000e+02,   4.68200000e+03,   1.26600000e+03,\n",
       "         2.39000000e+03,   7.60000000e+01,   2.30000000e+02,\n",
       "         9.70000000e+01,   9.20000000e+01,   5.70000000e+01,\n",
       "         8.20000000e+01,   8.60000000e+01,   2.90000000e+01,\n",
       "         4.70500000e+03,   4.51000000e+03,   7.22000000e+03,\n",
       "         4.73300000e+03,   4.79500000e+03,   4.05600000e+03,\n",
       "         2.81000000e+02,   5.10000000e+01,   1.12300000e+03,\n",
       "         9.07000000e+02,   9.50000000e+02,   9.36000000e+02,\n",
       "         1.37900000e+03,   2.88000000e+02,   3.99000000e+02,\n",
       "         1.69000000e+02,   1.78000000e+02,   2.84000000e+02,\n",
       "         6.10000000e+02,   4.42000000e+02,   1.00100000e+03,\n",
       "         3.78400000e+03,   3.02800000e+03,   6.65000000e+02,\n",
       "         8.23000000e+02,   1.16000000e+02,   1.81000000e+02,\n",
       "         2.34200000e+03,   3.43900000e+03,   3.08800000e+03,\n",
       "         7.39000000e+02,   3.93000000e+02,   6.70000000e+01,\n",
       "         1.41000000e+02,   1.19000000e+02,   1.32000000e+02,\n",
       "         2.09000000e+02,   7.03000000e+02,   2.37000000e+02,\n",
       "         2.89000000e+02,   1.77000000e+02,   2.66000000e+02,\n",
       "         4.70000000e+03,   5.90000000e+03,   3.70000000e+03,\n",
       "         7.00000000e+01,   1.45000000e+02,   2.42000000e+02,\n",
       "         2.28600000e+03,   2.93700000e+03,   1.06700000e+03,\n",
       "         4.04600000e+03,   6.09000000e+03,   1.19800000e+04,\n",
       "         1.24300000e+03,   5.18000000e+02,   8.46000000e+02,\n",
       "         2.85320000e+04,   4.76930000e+04,   1.86100000e+04,\n",
       "         2.74100000e+04,   3.20000000e+02,   5.98000000e+02,\n",
       "         6.39000000e+02,   1.05000000e+02,   1.22000000e+02,\n",
       "         2.52900000e+03,   4.69900000e+03,   2.28000000e+03,\n",
       "         2.53400000e+03,   2.50000000e+03,   6.81000000e+02])"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "health_data = health_data.convert_objects(convert_numeric=True)\n",
    "health_data[\"Pertussis - number of reported cases\"].unique()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<code>health_data.mean().round()</code> will take the mean of each column (this computation ignores the currently present nan values), then round, and return a dataframe indexed by the columns of the original dataframe.\n",
    "\n",
    "This function can be used to replace all missing values with the mean of each column. In this tutorial however, we will not use this method, because the large number of missing values would greatly skew our standard deviations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Year                                                                                  2012.0\n",
       "Number of new reported cases of human African trypanosomiasis (T.b. gambiense)         370.0\n",
       "Number of new reported cases of human African trypanosomiasis (T.b. rhodesiense)        21.0\n",
       "Number of reported cases of cholera                                                   6135.0\n",
       "Number of suspected meningitis cases reported                                         2175.0\n",
       "Congenital Rubella Syndrome - number of reported cases                                   1.0\n",
       "Diphtheria - number of reported cases                                                   33.0\n",
       "Japanese encephalitis - number of reported cases                                        48.0\n",
       "Leprosy - number of reported cases                                                    1949.0\n",
       "Malaria - number of reported confirmed cases                                        316480.0\n",
       "Measles - number of reported cases                                                    1600.0\n",
       "Mumps - number of reported cases                                                      4465.0\n",
       "Neonatal tetanus - number of reported cases                                             27.0\n",
       "Pertussis - number of reported cases                                                  1095.0\n",
       "Plague - number of reported cases                                                       28.0\n",
       "Poliomyelitis - number of reported cases                                                 3.0\n",
       "Rubella - number of reported cases                                                     535.0\n",
       "Total tetanus - number of reported cases                                                72.0\n",
       "Tuberculosis - new and relapse cases                                                 30551.0\n",
       "Yellow fever - number of reported cases                                                  5.0\n",
       "Number of cases of cutaneous leishmaniasis reported                                   4099.0\n",
       "Number of cases of visceral leishmaniasis reported                                     901.0\n",
       "dtype: float64"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "health_data.mean().round()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that we have figured out how to impute these missing values, lets start over and quickly apply this technique to the entire dataframe."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Applications/anaconda/lib/python2.7/site-packages/ipykernel/__main__.py:2: FutureWarning: convert_objects is deprecated.  Use the data-type specific converters pd.to_datetime, pd.to_timedelta and pd.to_numeric.\n",
      "  from ipykernel import kernelapp as app\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "count    9.480000e+02\n",
       "mean     3.055099e+04\n",
       "std      1.227259e+05\n",
       "min      0.000000e+00\n",
       "25%               NaN\n",
       "50%               NaN\n",
       "75%               NaN\n",
       "max      1.351913e+06\n",
       "Name: Tuberculosis - new and relapse cases, dtype: float64"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "health_data = pd.read_csv('GlobalHealthData.csv')\n",
    "health_data = health_data.convert_objects(convert_numeric=True)\n",
    "health_data[\"Tuberculosis - new and relapse cases\"].describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([             nan,   3.05070000e+04,   2.86790000e+04,\n",
       "         2.79830000e+04,   2.80290000e+04,   2.61500000e+04,\n",
       "         4.74000000e+02,   4.08000000e+02,   4.22000000e+02,\n",
       "         4.31000000e+02,   4.45000000e+02,   2.07010000e+04,\n",
       "         2.18800000e+04,   2.14290000e+04,   2.23360000e+04,\n",
       "         2.17010000e+04,   5.00000000e+00,   9.00000000e+00,\n",
       "         3.00000000e+00,   7.00000000e+00,   8.00000000e+00,\n",
       "         5.86070000e+04,   5.18190000e+04,   4.72400000e+04,\n",
       "         4.46550000e+04,   4.12210000e+04,   1.00000000e+01,\n",
       "         6.00000000e+00,   8.93300000e+03,   8.75800000e+03,\n",
       "         9.73300000e+03,   7.33600000e+03,   7.70100000e+03,\n",
       "         1.39700000e+03,   1.21300000e+03,   1.26100000e+03,\n",
       "         1.41000000e+03,   1.56000000e+03,   1.25000000e+03,\n",
       "         1.30500000e+03,   1.23900000e+03,   1.25700000e+03,\n",
       "         1.29400000e+03,   6.24000000e+02,   6.20000000e+02,\n",
       "         6.73000000e+02,   6.62000000e+02,   6.63000000e+02,\n",
       "         5.86000000e+03,   6.36300000e+03,   9.14600000e+03,\n",
       "         7.55000000e+03,   9.46200000e+03,   3.30000000e+01,\n",
       "         3.20000000e+01,   4.10000000e+01,   3.10000000e+01,\n",
       "         4.50000000e+01,   2.09000000e+02,   2.25000000e+02,\n",
       "         2.46000000e+02,   3.26000000e+02,   1.84506000e+05,\n",
       "         1.68683000e+05,   1.54358000e+05,   1.53892000e+05,\n",
       "         1.60875000e+05,   4.00000000e+00,   0.00000000e+00,\n",
       "         2.00000000e+00,   4.47000000e+03,   4.78300000e+03,\n",
       "         4.69700000e+03,   5.09800000e+03,   5.25000000e+03,\n",
       "         9.06000000e+02,   9.09000000e+02,   9.63000000e+02,\n",
       "         1.00100000e+03,   9.18000000e+02,   1.21000000e+02,\n",
       "         8.40000000e+01,   7.40000000e+01,   1.45000000e+02,\n",
       "         8.80000000e+01,   3.86600000e+03,   3.96600000e+03,\n",
       "         4.21200000e+03,   3.75600000e+03,   3.87800000e+03,\n",
       "         1.08000000e+03,   1.13000000e+03,   1.23500000e+03,\n",
       "         1.31100000e+03,   1.12500000e+03,   8.32700000e+03,\n",
       "         8.25700000e+03,   8.52100000e+03,   8.36300000e+03,\n",
       "         8.86500000e+03,   1.40900000e+03,   1.36000000e+03,\n",
       "         1.32100000e+03,   1.72500000e+03,   6.83400000e+03,\n",
       "         6.16100000e+03,   6.60300000e+03,   7.01300000e+03,\n",
       "         8.36200000e+03,   7.60200000e+04,   7.51220000e+04,\n",
       "         7.76470000e+04,   7.43950000e+04,   7.86810000e+04,\n",
       "         2.12000000e+02,   2.43000000e+02,   2.30000000e+02,\n",
       "         2.37000000e+02,   2.13000000e+02,   1.93000000e+03,\n",
       "         2.08100000e+03,   2.17100000e+03,   2.41200000e+03,\n",
       "         2.65500000e+03,   5.32600000e+03,   5.21000000e+03,\n",
       "         5.28600000e+03,   4.80000000e+03,   4.71600000e+03,\n",
       "         7.46700000e+03,   6.92100000e+03,   6.74200000e+03,\n",
       "         7.61100000e+03,   7.27700000e+03,   2.47490000e+04,\n",
       "         2.37620000e+04,   2.24760000e+04,   2.27080000e+04,\n",
       "         2.25710000e+04,   3.05000000e+02,   4.20000000e+02,\n",
       "         3.80000000e+02,   3.56000000e+02,   3.32000000e+02,\n",
       "         3.77430000e+04,   4.01850000e+04,   3.85550000e+04,\n",
       "         4.04600000e+04,   3.92020000e+04,   2.56480000e+04,\n",
       "         2.48020000e+04,   2.45330000e+04,   2.40730000e+04,\n",
       "         2.46620000e+04,   1.63800000e+03,   1.65300000e+03,\n",
       "         1.43000000e+03,   1.36100000e+03,   1.56300000e+03,\n",
       "         8.59000000e+03,   8.08400000e+03,   5.61100000e+03,\n",
       "         6.64300000e+03,   8.74300000e+03,   1.12370000e+04,\n",
       "         1.05850000e+04,   1.05050000e+04,   9.45200000e+03,\n",
       "         8.41100000e+03,   2.39400000e+03,   2.45000000e+03,\n",
       "         2.37600000e+03,   2.39800000e+03,   8.47176000e+05,\n",
       "         8.90645000e+05,   8.99669000e+05,   9.08399000e+05,\n",
       "         9.65257000e+05,   1.17370000e+04,   1.20970000e+04,\n",
       "         1.18840000e+04,   1.14200000e+04,   1.13240000e+04,\n",
       "         1.20000000e+02,   1.17000000e+02,   1.28000000e+02,\n",
       "         1.06990000e+04,   1.13030000e+04,   1.09750000e+04,\n",
       "         1.01500000e+04,   9.76500000e+03,   1.00000000e+00,\n",
       "         4.10000000e+02,   4.75000000e+02,   5.14000000e+02,\n",
       "         4.92000000e+02,   4.43000000e+02,   5.11000000e+02,\n",
       "         5.39000000e+02,   6.19000000e+02,   6.88000000e+02,\n",
       "         8.32000000e+02,   7.57000000e+02,   7.34000000e+02,\n",
       "         8.05000000e+02,   8.27000000e+02,   7.12000000e+02,\n",
       "         6.30000000e+01,   5.10000000e+01,   6.10000000e+01,\n",
       "         4.71000000e+02,   5.65000000e+02,   5.68000000e+02,\n",
       "         6.32000000e+02,   9.76650000e+04,   9.18850000e+04,\n",
       "         9.14330000e+04,   8.46480000e+04,   7.63360000e+04,\n",
       "         1.12439000e+05,   1.08984000e+05,   1.10132000e+05,\n",
       "         1.14170000e+05,   1.11709000e+05,   3.30000000e+02,\n",
       "         3.42000000e+02,   3.59000000e+02,   3.31000000e+02,\n",
       "         2.92000000e+02,   3.16200000e+03,   3.47400000e+03,\n",
       "         3.68600000e+03,   4.17200000e+03,   3.78300000e+03,\n",
       "         4.45000000e+03,   4.26200000e+03,   4.30900000e+03,\n",
       "         3.96400000e+03,   4.25600000e+03,   5.27700000e+03,\n",
       "         5.45600000e+03,   5.10600000e+03,   4.83200000e+03,\n",
       "         4.70300000e+03,   7.87600000e+03,   8.45300000e+03,\n",
       "         8.97400000e+03,   9.26000000e+03,   9.68500000e+03,\n",
       "         2.17600000e+03,   2.05300000e+03,   1.89600000e+03,\n",
       "         1.70000000e+03,   1.68600000e+03,   8.83000000e+02,\n",
       "         8.20000000e+02,   7.07000000e+02,   2.86000000e+03,\n",
       "         3.14300000e+03,   3.04900000e+03,   2.87000000e+03,\n",
       "         2.90400000e+03,   2.63000000e+02,   2.59000000e+02,\n",
       "         2.94000000e+02,   2.86000000e+02,   3.58000000e+02,\n",
       "         1.31677000e+05,   1.45323000e+05,   1.56539000e+05,\n",
       "         1.54694000e+05,   1.48936000e+05,   2.54000000e+02,\n",
       "         2.10000000e+02,   2.15000000e+02,   1.89000000e+02,\n",
       "         1.44000000e+02,   2.66000000e+02,   2.61000000e+02,\n",
       "         3.16000000e+02,   3.08000000e+02,   4.02000000e+02,\n",
       "         4.70800000e+03,   4.70500000e+03,   4.72600000e+03,\n",
       "         4.80100000e+03,   4.92500000e+03,   5.17900000e+03,\n",
       "         4.92900000e+03,   4.40400000e+03,   3.79000000e+03,\n",
       "         3.07300000e+03,   2.32500000e+03,   2.33300000e+03,\n",
       "         2.30200000e+03,   1.98900000e+03,   2.12600000e+03,\n",
       "         3.43400000e+03,   3.94000000e+03,   4.54700000e+03,\n",
       "         4.67800000e+03,   5.68700000e+03,   4.19300000e+03,\n",
       "         4.02900000e+03,   4.06200000e+03,   4.05600000e+03,\n",
       "         4.16800000e+03,   1.50430000e+04,   1.47530000e+04,\n",
       "         1.53890000e+04,   1.46070000e+04,   1.48920000e+04,\n",
       "         4.94000000e+02,   5.18000000e+02,   4.54000000e+02,\n",
       "         5.42000000e+02,   3.31100000e+03,   3.44200000e+03,\n",
       "         3.04000000e+03,   3.32200000e+03,   2.90200000e+03,\n",
       "         1.13130000e+04,   1.14070000e+04,   1.13590000e+04,\n",
       "         1.10380000e+04,   8.37300000e+03,   2.08700000e+03,\n",
       "         1.93900000e+03,   2.06300000e+03,   2.18300000e+03,\n",
       "         6.79000000e+02,   7.48000000e+02,   7.10000000e+02,\n",
       "         7.63000000e+02,   1.70400000e+04,   1.65710000e+04,\n",
       "         1.52250000e+04,   1.42220000e+04,   2.98100000e+03,\n",
       "         3.01400000e+03,   3.23300000e+03,   2.87600000e+03,\n",
       "         2.92400000e+03,   1.04000000e+03,   1.15900000e+03,\n",
       "         1.27900000e+03,   1.52800000e+03,   1.27200000e+03,\n",
       "         1.10000000e+01,   2.20000000e+01,   1.24390500e+06,\n",
       "         1.28983600e+06,   1.32394900e+06,   1.33986600e+06,\n",
       "         1.35191300e+06,   3.25582000e+05,   3.28824000e+05,\n",
       "         3.18949000e+05,   3.00659000e+05,   2.92754000e+05,\n",
       "         1.10520000e+04,   1.10420000e+04,   1.09800000e+04,\n",
       "         1.03620000e+04,   1.00970000e+04,   8.55400000e+03,\n",
       "         8.66400000e+03,   8.83700000e+03,   9.70700000e+03,\n",
       "         9.38500000e+03,   3.41000000e+02,   3.88000000e+02,\n",
       "         3.87000000e+02,   4.27000000e+02,   5.06000000e+02,\n",
       "         4.12000000e+02,   3.40000000e+02,   3.47000000e+02,\n",
       "         3.01900000e+03,   3.09600000e+03,   3.42100000e+03,\n",
       "         4.53900000e+03,   4.08700000e+03,   9.60000000e+01,\n",
       "         9.10000000e+01,   1.05000000e+02,   1.30000000e+02,\n",
       "         1.39000000e+02,   2.04950000e+04,   2.08570000e+04,\n",
       "         2.21190000e+04,   2.26930000e+04,   2.36310000e+04,\n",
       "         3.48000000e+02,   3.28000000e+02,   3.38000000e+02,\n",
       "         3.71000000e+02,   1.89580000e+04,   1.80060000e+04,\n",
       "         2.50740000e+04,   2.33990000e+04,   2.49050000e+04,\n",
       "         8.97960000e+04,   9.29870000e+04,   9.73200000e+04,\n",
       "         9.92720000e+04,   1.02997000e+05,   4.07000000e+02,\n",
       "         3.46000000e+02,   3.43000000e+02,   2.78000000e+02,\n",
       "         7.04000000e+02,   7.37000000e+02,   6.72000000e+02,\n",
       "         9.57000000e+02,   9.33000000e+02,   7.07500000e+03,\n",
       "         6.19500000e+03,   5.98000000e+03,   5.65200000e+03,\n",
       "         5.93100000e+03,   4.13000000e+03,   4.11800000e+03,\n",
       "         4.36000000e+03,   4.06100000e+03,   3.90000000e+03,\n",
       "         8.72000000e+02,   9.59000000e+02,   8.64000000e+02,\n",
       "         9.13000000e+02,   9.48000000e+02,   6.89000000e+02,\n",
       "         6.30000000e+02,   4.96000000e+02,   5.13000000e+02,\n",
       "         4.99000000e+02,   9.55500000e+03,   1.07760000e+04,\n",
       "         1.15610000e+04,   1.16740000e+04,   1.22130000e+04,\n",
       "         7.51100000e+03,   8.09300000e+03,   7.90600000e+03,\n",
       "         6.59700000e+03,   5.91800000e+03,   1.34400000e+03,\n",
       "         1.54900000e+03,   1.51800000e+03,   2.09600000e+03,\n",
       "         1.57300000e+03,   1.63500000e+03,   1.74800000e+03,\n",
       "         1.75100000e+03,   1.89500000e+03,   3.80000000e+01,\n",
       "         2.50000000e+01,   2.90000000e+01,   2.70000000e+01,\n",
       "         2.65610000e+04,   2.57820000e+04,   2.60190000e+04,\n",
       "         2.44320000e+04,   2.27580000e+04,   1.77790000e+04,\n",
       "         2.03350000e+04,   1.93610000e+04,   2.10920000e+04,\n",
       "         2.26740000e+04,   2.34170000e+04,   2.18510000e+04,\n",
       "         1.98080000e+04,   1.85170000e+04,   1.73410000e+04,\n",
       "         1.14000000e+02,   1.10000000e+02,   8.70000000e+01,\n",
       "         9.50000000e+01,   1.00000000e+02,   5.81000000e+03,\n",
       "         5.44600000e+03,   5.42800000e+03,   5.29100000e+03,\n",
       "         6.61100000e+03,   5.00000000e+01,   4.20000000e+01,\n",
       "         3.00000000e+01,   1.49000000e+02,   1.93000000e+02,\n",
       "         1.41000000e+02,   2.22300000e+03,   2.61600000e+03,\n",
       "         1.80400000e+03,   2.46100000e+03,   2.64000000e+03,\n",
       "         1.22000000e+02,   1.15000000e+02,   2.07080000e+04,\n",
       "         2.04700000e+04,   1.98570000e+04,   2.01550000e+04,\n",
       "         1.89570000e+04,   1.48000000e+02,   1.64000000e+02,\n",
       "         4.33100000e+03,   4.12800000e+03,   4.21700000e+03,\n",
       "         4.45800000e+03,   4.48100000e+03,   1.19000000e+02,\n",
       "         9.80000000e+01,   2.91260000e+04,   2.86350000e+04,\n",
       "         2.86400000e+04,   2.83590000e+04,   2.73480000e+04,\n",
       "         5.32720000e+04,   4.77410000e+04,   4.46270000e+04,\n",
       "         4.35580000e+04,   4.32210000e+04,   1.34855000e+05,\n",
       "         1.41170000e+05,   1.36737000e+05,   1.31590000e+05,\n",
       "         1.28343000e+05,   9.59700000e+03,   1.00030000e+04,\n",
       "         1.08060000e+04,   1.12810000e+04,   1.19800000e+04,\n",
       "         3.38340000e+04,   3.51950000e+04,   3.54340000e+04,\n",
       "         3.51140000e+04,   3.48880000e+04,   9.20000000e+02,\n",
       "         9.70000000e+02,   1.02700000e+03,   2.72000000e+02,\n",
       "         2.93000000e+02,   3.01000000e+02,   2.98000000e+02,\n",
       "         2.95200000e+03,   2.79000000e+03,   2.69300000e+03,\n",
       "         2.44800000e+03,   2.28300000e+03,   1.12510000e+04,\n",
       "         1.09890000e+04,   1.05100000e+04,   1.01300000e+04,\n",
       "         1.00210000e+04,   9.48250000e+04,   9.28180000e+04,\n",
       "         8.67780000e+04,   8.41210000e+04,   8.85890000e+04,\n",
       "         3.62000000e+02,   3.20000000e+02,   2.95000000e+02,\n",
       "         3.82000000e+02,   3.37000000e+02,   2.88910000e+05,\n",
       "         2.67475000e+05,   2.64934000e+05,   2.64235000e+05,\n",
       "         2.64248000e+05,   1.20000000e+01,   1.90000000e+01,\n",
       "         1.42400000e+03,   1.52000000e+03,   1.63200000e+03,\n",
       "         1.54100000e+03,   1.53500000e+03,   2.26570000e+04,\n",
       "         2.05570000e+04,   1.48930000e+04,   1.45310000e+04,\n",
       "         1.23060000e+04,   2.25500000e+03,   2.41600000e+03,\n",
       "         2.37200000e+03,   2.35200000e+03,   2.34600000e+03,\n",
       "         2.97310000e+04,   2.97600000e+04,   3.12410000e+04,\n",
       "         3.10730000e+04,   3.18440000e+04,   2.29918000e+05,\n",
       "         2.16627000e+05,   1.95560000e+05,   1.66323000e+05,\n",
       "         1.46565000e+05,   7.04300000e+03,   7.05400000e+03,\n",
       "         7.94600000e+03,   7.00200000e+03,   7.66800000e+03,\n",
       "         2.33800000e+03,   2.49000000e+03,   2.49600000e+03,\n",
       "         2.60200000e+03,   2.71400000e+03,   7.28000000e+02,\n",
       "         5.53000000e+02,   5.80000000e+02,   4.15790000e+04,\n",
       "         4.37020000e+04,   4.62530000e+04,   4.40630000e+04,\n",
       "         4.33180000e+04,   4.48500000e+03,   4.40900000e+03,\n",
       "         4.23300000e+03,   4.13500000e+03,   4.47100000e+03,\n",
       "         1.55230000e+04,   1.61070000e+04,   1.70450000e+04,\n",
       "         1.85900000e+04,   2.06430000e+04,   1.06053000e+05,\n",
       "         1.05753000e+05,   1.12910000e+05,   1.25310000e+05,\n",
       "         1.32653000e+05,   5.70200000e+03,   6.09100000e+03,\n",
       "         6.62300000e+03,   6.70300000e+03,   7.41500000e+03,\n",
       "         1.70000000e+01,   1.50000000e+01,   2.00000000e+01,\n",
       "         1.40000000e+01,   1.60000000e+01,   1.47000000e+02,\n",
       "         1.36000000e+02,   7.80000000e+01,   3.43500000e+03,\n",
       "         3.69000000e+03,   3.93200000e+03,   4.46500000e+03,\n",
       "         4.04300000e+03,   1.31860000e+04,   1.22650000e+04,\n",
       "         1.10220000e+04,   1.10610000e+04,   1.11390000e+04,\n",
       "         2.08300000e+03,   1.87200000e+03,   2.17400000e+03,\n",
       "         2.53100000e+03,   2.40000000e+01,   2.10000000e+01,\n",
       "         1.20720000e+04,   1.30740000e+04,   1.27340000e+04,\n",
       "         1.28590000e+04,   1.15240000e+04,   2.16400000e+03,\n",
       "         2.30100000e+03,   1.64100000e+03,   1.52500000e+03,\n",
       "         3.98000000e+02,   3.21000000e+02,   3.78000000e+02,\n",
       "         4.09000000e+02,   4.53000000e+02,   1.34000000e+02,\n",
       "         1.81000000e+02,   1.69000000e+02,   1.86000000e+02,\n",
       "         3.60000000e+02,   3.61000000e+02,   3.66000000e+02,\n",
       "         1.29940000e+04,   1.19750000e+04,   1.16530000e+04,\n",
       "         1.01390000e+04,   1.10750000e+04,   3.12380000e+05,\n",
       "         3.23664000e+05,   3.62453000e+05,   3.54786000e+05,\n",
       "         3.60183000e+05,   6.42200000e+03,   8.40300000e+03,\n",
       "         7.21700000e+03,   5.24400000e+03,   5.67700000e+03,\n",
       "         6.41000000e+03,   6.88800000e+03,   7.19100000e+03,\n",
       "         9.33100000e+03,   9.15500000e+03,   1.01810000e+04,\n",
       "         9.93400000e+03,   9.57500000e+03,   1.90560000e+04,\n",
       "         1.87750000e+04,   1.93480000e+04,   2.61310000e+04,\n",
       "         2.60010000e+04,   1.25000000e+02,   1.94000000e+02,\n",
       "         1.78000000e+02,   6.64100000e+03,   7.16500000e+03,\n",
       "         8.33700000e+03,   1.01010000e+04,   1.00380000e+04,\n",
       "         6.10000000e+02,   5.93000000e+02,   5.38000000e+02,\n",
       "         6.15000000e+02,   5.72000000e+02,   4.66000000e+02,\n",
       "         4.16000000e+02,   5.24000000e+02,   5.08000000e+02,\n",
       "         5.03000000e+02,   2.73500000e+03,   3.00300000e+03,\n",
       "         3.62000000e+03,   3.66600000e+03,   4.05000000e+03,\n",
       "         5.57600000e+03,   6.50800000e+03,   7.03500000e+03,\n",
       "         6.99400000e+03,   7.21000000e+03,   6.35410000e+04,\n",
       "         6.03040000e+04,   6.58240000e+04,   6.71280000e+04,\n",
       "         6.39750000e+04,   3.18000000e+02,   3.35000000e+02,\n",
       "         3.84000000e+02,   4.50000000e+02,   3.75700000e+03,\n",
       "         3.82800000e+03,   4.38600000e+03,   4.74800000e+03,\n",
       "         2.60000000e+03,   2.84300000e+03,   2.88800000e+03,\n",
       "         2.79100000e+03,   2.98600000e+03,   2.50000000e+02,\n",
       "         2.74000000e+02,   2.24000000e+02,   2.19000000e+02,\n",
       "         3.03500000e+03,   3.23900000e+03,   3.01500000e+03,\n",
       "         2.36800000e+03,   2.15500000e+03,   1.31700000e+04,\n",
       "         1.41390000e+04,   1.50540000e+04,   1.58790000e+04,\n",
       "         1.67710000e+04,   3.04600000e+03,   2.93700000e+03,\n",
       "         2.73100000e+03,   3.23000000e+03,   3.15700000e+03,\n",
       "         1.80000000e+01,   4.55490000e+04,   4.46630000e+04,\n",
       "         4.63060000e+04,   4.28850000e+04,   4.17030000e+04,\n",
       "         3.65100000e+04,   4.09900000e+04,   3.42370000e+04,\n",
       "         3.38570000e+04,   3.60750000e+04,   8.20000000e+01,\n",
       "         7.90000000e+01,   1.03000000e+02,   1.31000000e+02,\n",
       "         1.16000000e+02,   7.38400000e+03,   8.26900000e+03,\n",
       "         8.40700000e+03,   7.82200000e+03,   8.10100000e+03,\n",
       "         6.40530000e+04,   6.24790000e+04,   5.93570000e+04,\n",
       "         6.10980000e+04,   6.15370000e+04,   9.11300000e+03,\n",
       "         9.94500000e+03,   1.05210000e+04,   1.11810000e+04,\n",
       "         1.15450000e+04,   8.81000000e+02,   8.08000000e+02,\n",
       "         8.17000000e+02,   6.99000000e+02,   2.08120000e+04,\n",
       "         1.48320000e+04,   1.53450000e+04,   1.68830000e+04,\n",
       "         1.99730000e+04,   1.23000000e+02,   6.55500000e+03,\n",
       "         6.49500000e+03,   6.28200000e+03,   6.45100000e+03,\n",
       "         6.47400000e+03,   1.00395000e+05,   1.02112000e+05,\n",
       "         9.88040000e+04,   9.74480000e+04,   9.68610000e+04,\n",
       "         1.03250000e+04,   9.86700000e+03,   8.63600000e+03,\n",
       "         8.91600000e+03,   8.56200000e+03,   4.06380000e+04,\n",
       "         4.07260000e+04,   4.35830000e+04,   4.41540000e+04,\n",
       "         4.48790000e+04,   3.28990000e+04,   3.57600000e+04,\n",
       "         3.84040000e+04,   4.42090000e+04,   4.29710000e+04])"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "health_data[\"Tuberculosis - new and relapse cases\"].unique()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Structurally, Pandas dataframes are a collection of Series objects sharing a common index.  In general, the Series object and Dataframe object share a large number of functions with some behavioral differences.  In other words, whatever computation you can do on a single column can generally be applied to the entire dataframe.\n",
    "\n",
    "Now we can use the dataframe version of <code>describe</code> to get an overview of all of our data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Year</th>\n",
       "      <th>Number of new reported cases of human African trypanosomiasis (T.b. gambiense)</th>\n",
       "      <th>Number of new reported cases of human African trypanosomiasis (T.b. rhodesiense)</th>\n",
       "      <th>Number of reported cases of cholera</th>\n",
       "      <th>Number of suspected meningitis cases reported</th>\n",
       "      <th>Congenital Rubella Syndrome - number of reported cases</th>\n",
       "      <th>Diphtheria - number of reported cases</th>\n",
       "      <th>Japanese encephalitis - number of reported cases</th>\n",
       "      <th>Leprosy - number of reported cases</th>\n",
       "      <th>Malaria - number of reported confirmed cases</th>\n",
       "      <th>...</th>\n",
       "      <th>Neonatal tetanus - number of reported cases</th>\n",
       "      <th>Pertussis - number of reported cases</th>\n",
       "      <th>Plague - number of reported cases</th>\n",
       "      <th>Poliomyelitis - number of reported cases</th>\n",
       "      <th>Rubella - number of reported cases</th>\n",
       "      <th>Total tetanus - number of reported cases</th>\n",
       "      <th>Tuberculosis - new and relapse cases</th>\n",
       "      <th>Yellow fever - number of reported cases</th>\n",
       "      <th>Number of cases of cutaneous leishmaniasis reported</th>\n",
       "      <th>Number of cases of visceral leishmaniasis reported</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>1164.000000</td>\n",
       "      <td>109.000000</td>\n",
       "      <td>36.000000</td>\n",
       "      <td>245.000000</td>\n",
       "      <td>96.000000</td>\n",
       "      <td>749.000000</td>\n",
       "      <td>953.000000</td>\n",
       "      <td>487.000000</td>\n",
       "      <td>588.000000</td>\n",
       "      <td>5.030000e+02</td>\n",
       "      <td>...</td>\n",
       "      <td>963.000000</td>\n",
       "      <td>962.000000</td>\n",
       "      <td>33.000000</td>\n",
       "      <td>970.000000</td>\n",
       "      <td>1010.000000</td>\n",
       "      <td>1008.000000</td>\n",
       "      <td>9.480000e+02</td>\n",
       "      <td>789.000000</td>\n",
       "      <td>244.000000</td>\n",
       "      <td>246.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>2011.500000</td>\n",
       "      <td>369.844037</td>\n",
       "      <td>21.444444</td>\n",
       "      <td>6134.971429</td>\n",
       "      <td>2175.250000</td>\n",
       "      <td>1.311081</td>\n",
       "      <td>32.569780</td>\n",
       "      <td>47.926078</td>\n",
       "      <td>1949.170068</td>\n",
       "      <td>3.164803e+05</td>\n",
       "      <td>...</td>\n",
       "      <td>27.439252</td>\n",
       "      <td>1095.180873</td>\n",
       "      <td>27.545455</td>\n",
       "      <td>3.189691</td>\n",
       "      <td>535.100990</td>\n",
       "      <td>71.886905</td>\n",
       "      <td>3.055099e+04</td>\n",
       "      <td>5.012674</td>\n",
       "      <td>4099.094262</td>\n",
       "      <td>901.495935</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>1.708559</td>\n",
       "      <td>1291.601046</td>\n",
       "      <td>32.574700</td>\n",
       "      <td>26976.057407</td>\n",
       "      <td>6133.867934</td>\n",
       "      <td>9.397670</td>\n",
       "      <td>322.362516</td>\n",
       "      <td>287.843712</td>\n",
       "      <td>12400.256473</td>\n",
       "      <td>7.290974e+05</td>\n",
       "      <td>...</td>\n",
       "      <td>124.042240</td>\n",
       "      <td>4768.084078</td>\n",
       "      <td>74.925835</td>\n",
       "      <td>26.790189</td>\n",
       "      <td>3970.447112</td>\n",
       "      <td>303.368764</td>\n",
       "      <td>1.227259e+05</td>\n",
       "      <td>45.072388</td>\n",
       "      <td>9779.236568</td>\n",
       "      <td>3670.883278</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>2009.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>2010.000000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>2011.500000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>2013.000000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>2014.000000</td>\n",
       "      <td>7183.000000</td>\n",
       "      <td>129.000000</td>\n",
       "      <td>340311.000000</td>\n",
       "      <td>56128.000000</td>\n",
       "      <td>189.000000</td>\n",
       "      <td>6094.000000</td>\n",
       "      <td>3913.000000</td>\n",
       "      <td>134752.000000</td>\n",
       "      <td>6.715223e+06</td>\n",
       "      <td>...</td>\n",
       "      <td>1412.000000</td>\n",
       "      <td>60385.000000</td>\n",
       "      <td>313.000000</td>\n",
       "      <td>460.000000</td>\n",
       "      <td>69860.000000</td>\n",
       "      <td>5017.000000</td>\n",
       "      <td>1.351913e+06</td>\n",
       "      <td>1024.000000</td>\n",
       "      <td>71996.000000</td>\n",
       "      <td>33155.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>8 rows × 22 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "              Year  \\\n",
       "count  1164.000000   \n",
       "mean   2011.500000   \n",
       "std       1.708559   \n",
       "min    2009.000000   \n",
       "25%    2010.000000   \n",
       "50%    2011.500000   \n",
       "75%    2013.000000   \n",
       "max    2014.000000   \n",
       "\n",
       "       Number of new reported cases of human African trypanosomiasis (T.b. gambiense)  \\\n",
       "count                                         109.000000                                \n",
       "mean                                          369.844037                                \n",
       "std                                          1291.601046                                \n",
       "min                                             0.000000                                \n",
       "25%                                                  NaN                                \n",
       "50%                                                  NaN                                \n",
       "75%                                                  NaN                                \n",
       "max                                          7183.000000                                \n",
       "\n",
       "       Number of new reported cases of human African trypanosomiasis (T.b. rhodesiense)  \\\n",
       "count                                          36.000000                                  \n",
       "mean                                           21.444444                                  \n",
       "std                                            32.574700                                  \n",
       "min                                             0.000000                                  \n",
       "25%                                                  NaN                                  \n",
       "50%                                                  NaN                                  \n",
       "75%                                                  NaN                                  \n",
       "max                                           129.000000                                  \n",
       "\n",
       "       Number of reported cases of cholera  \\\n",
       "count                           245.000000   \n",
       "mean                           6134.971429   \n",
       "std                           26976.057407   \n",
       "min                               1.000000   \n",
       "25%                                    NaN   \n",
       "50%                                    NaN   \n",
       "75%                                    NaN   \n",
       "max                          340311.000000   \n",
       "\n",
       "       Number of suspected meningitis cases reported  \\\n",
       "count                                      96.000000   \n",
       "mean                                     2175.250000   \n",
       "std                                      6133.867934   \n",
       "min                                         1.000000   \n",
       "25%                                              NaN   \n",
       "50%                                              NaN   \n",
       "75%                                              NaN   \n",
       "max                                     56128.000000   \n",
       "\n",
       "       Congenital Rubella Syndrome - number of reported cases  \\\n",
       "count                                         749.000000        \n",
       "mean                                            1.311081        \n",
       "std                                             9.397670        \n",
       "min                                             0.000000        \n",
       "25%                                                  NaN        \n",
       "50%                                                  NaN        \n",
       "75%                                                  NaN        \n",
       "max                                           189.000000        \n",
       "\n",
       "       Diphtheria - number of reported cases  \\\n",
       "count                             953.000000   \n",
       "mean                               32.569780   \n",
       "std                               322.362516   \n",
       "min                                 0.000000   \n",
       "25%                                      NaN   \n",
       "50%                                      NaN   \n",
       "75%                                      NaN   \n",
       "max                              6094.000000   \n",
       "\n",
       "       Japanese encephalitis - number of reported cases  \\\n",
       "count                                        487.000000   \n",
       "mean                                          47.926078   \n",
       "std                                          287.843712   \n",
       "min                                            0.000000   \n",
       "25%                                                 NaN   \n",
       "50%                                                 NaN   \n",
       "75%                                                 NaN   \n",
       "max                                         3913.000000   \n",
       "\n",
       "       Leprosy - number of reported cases  \\\n",
       "count                          588.000000   \n",
       "mean                          1949.170068   \n",
       "std                          12400.256473   \n",
       "min                              0.000000   \n",
       "25%                                   NaN   \n",
       "50%                                   NaN   \n",
       "75%                                   NaN   \n",
       "max                         134752.000000   \n",
       "\n",
       "       Malaria - number of reported confirmed cases  \\\n",
       "count                                  5.030000e+02   \n",
       "mean                                   3.164803e+05   \n",
       "std                                    7.290974e+05   \n",
       "min                                    0.000000e+00   \n",
       "25%                                             NaN   \n",
       "50%                                             NaN   \n",
       "75%                                             NaN   \n",
       "max                                    6.715223e+06   \n",
       "\n",
       "                              ...                          \\\n",
       "count                         ...                           \n",
       "mean                          ...                           \n",
       "std                           ...                           \n",
       "min                           ...                           \n",
       "25%                           ...                           \n",
       "50%                           ...                           \n",
       "75%                           ...                           \n",
       "max                           ...                           \n",
       "\n",
       "       Neonatal tetanus - number of reported cases  \\\n",
       "count                                   963.000000   \n",
       "mean                                     27.439252   \n",
       "std                                     124.042240   \n",
       "min                                       0.000000   \n",
       "25%                                            NaN   \n",
       "50%                                            NaN   \n",
       "75%                                            NaN   \n",
       "max                                    1412.000000   \n",
       "\n",
       "       Pertussis - number of reported cases  \\\n",
       "count                            962.000000   \n",
       "mean                            1095.180873   \n",
       "std                             4768.084078   \n",
       "min                                0.000000   \n",
       "25%                                     NaN   \n",
       "50%                                     NaN   \n",
       "75%                                     NaN   \n",
       "max                            60385.000000   \n",
       "\n",
       "       Plague - number of reported cases  \\\n",
       "count                          33.000000   \n",
       "mean                           27.545455   \n",
       "std                            74.925835   \n",
       "min                             0.000000   \n",
       "25%                                  NaN   \n",
       "50%                                  NaN   \n",
       "75%                                  NaN   \n",
       "max                           313.000000   \n",
       "\n",
       "       Poliomyelitis - number of reported cases  \\\n",
       "count                                970.000000   \n",
       "mean                                   3.189691   \n",
       "std                                   26.790189   \n",
       "min                                    0.000000   \n",
       "25%                                         NaN   \n",
       "50%                                         NaN   \n",
       "75%                                         NaN   \n",
       "max                                  460.000000   \n",
       "\n",
       "       Rubella - number of reported cases  \\\n",
       "count                         1010.000000   \n",
       "mean                           535.100990   \n",
       "std                           3970.447112   \n",
       "min                              0.000000   \n",
       "25%                                   NaN   \n",
       "50%                                   NaN   \n",
       "75%                                   NaN   \n",
       "max                          69860.000000   \n",
       "\n",
       "       Total tetanus - number of reported cases  \\\n",
       "count                               1008.000000   \n",
       "mean                                  71.886905   \n",
       "std                                  303.368764   \n",
       "min                                    0.000000   \n",
       "25%                                         NaN   \n",
       "50%                                         NaN   \n",
       "75%                                         NaN   \n",
       "max                                 5017.000000   \n",
       "\n",
       "       Tuberculosis - new and relapse cases  \\\n",
       "count                          9.480000e+02   \n",
       "mean                           3.055099e+04   \n",
       "std                            1.227259e+05   \n",
       "min                            0.000000e+00   \n",
       "25%                                     NaN   \n",
       "50%                                     NaN   \n",
       "75%                                     NaN   \n",
       "max                            1.351913e+06   \n",
       "\n",
       "       Yellow fever - number of reported cases  \\\n",
       "count                               789.000000   \n",
       "mean                                  5.012674   \n",
       "std                                  45.072388   \n",
       "min                                   0.000000   \n",
       "25%                                        NaN   \n",
       "50%                                        NaN   \n",
       "75%                                        NaN   \n",
       "max                                1024.000000   \n",
       "\n",
       "       Number of cases of cutaneous leishmaniasis reported  \\\n",
       "count                                         244.000000     \n",
       "mean                                         4099.094262     \n",
       "std                                          9779.236568     \n",
       "min                                             0.000000     \n",
       "25%                                                  NaN     \n",
       "50%                                                  NaN     \n",
       "75%                                                  NaN     \n",
       "max                                         71996.000000     \n",
       "\n",
       "       Number of cases of visceral leishmaniasis reported  \n",
       "count                                         246.000000   \n",
       "mean                                          901.495935   \n",
       "std                                          3670.883278   \n",
       "min                                             0.000000   \n",
       "25%                                                  NaN   \n",
       "50%                                                  NaN   \n",
       "75%                                                  NaN   \n",
       "max                                         33155.000000   \n",
       "\n",
       "[8 rows x 22 columns]"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "health_data.describe()"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python [default]",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}