Web_Scraping_Instructor.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Web Scraping in Python3"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Please make sure you signed in. This workshop will be around 90 minutes in length."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Introduction\n",
    "\n",
    "Web Scraping is a very useful method used by data scientists to gather data from websites. This workshop will introduce the basics of web scraping and review common web scraping methodologies. Although there are many different ways to scrape data from websites we will cover some of the most popularly used libraries that python has to offer."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## How to stay out of jail\n",
    "\n",
    "Web Scraping is one of those legal gray areas (sadly). Make sure the site you are scraping is public domain, other wise make sure you have approval from the site's owner. Even with public domain be careful as sometimes your web scrape can overlook a site's terms of agreement. \n",
    "\n",
    "Ex.) LinkedIn vs. Doe Defendants. LinkedIn is suing 100 anonymous users for the following violations:\n",
    "\n",
    "-Computer Fraud and Abuse Act (CFAA)\n",
    "\n",
    "-California Penal Code\n",
    "\n",
    "-Digital Millennium Copyright Act (DMCA)\n",
    "\n",
    "\n",
    "A lot of web scraping/crawling rules will be listed in a sites robots.txt file which I will show you how to access. One example of restricion would be the crawl rate which tell you how many request you can per minute. This is to keep your crawl from overloading the server (simply add a delay of 10 sec between requests in code)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## HTML Basics Review\n",
    "\n",
    "HTML stands for Hypertext Markup Language. \n",
    "\n",
    "It is used to describe the structure of web pages. It uses HTML elements that are represented by tags. This are a few of the HTML elements that are available but there are many more that can be used. Familiarity with different tags and thier functions are good know, but it is not as important as knowing the structure of HTML.\n",
    "\n",
    "The following is an example of the HTML code structure:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "#   <!DOCTYPE html>\n",
    "#   <html>\n",
    "#      <head>\n",
    "#          <title>Page Title</title>\n",
    "#      </head>\n",
    "#      <body>\n",
    "#         <h1>My First Heading</h1>\n",
    "#         <p>My first paragraph.</p>\n",
    "#         <input type=\"button\" onclick=\"location.href='http://google.com'\" value=\"Go to Google\">\n",
    "#      </body>\n",
    "#   </html>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### HTML Attributes\n",
    "\n",
    "All HTML attributes can have attributes that provide additional information about an element. In the example above the input element has the attributes \"type\", \"onclick\", and \"value\". \"Type\" here is used to declare that this will be a button input. The \"onclick\" attribute tells what to do when the button is clicked. \"Value\" is used to display text on the button. In this case a button that says \"go to google\" will redirect you to google.com when clicked."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Import Modules\n",
    "\n",
    "Here there are two important libraries we will be using. BeautifulSoup allows us to work with HTML easily. The second is requests which we will use to gather the HTML from the website of our choice."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from bs4 import BeautifulSoup as bs # importing the BeautifulSoup: Helps parse and \"beautify\" HTML \n",
    "import requests as req              # importing the urlopen: Helps make web client to sites."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you get an error when running this block, you may not have the bs4 library installed. To do this open up Anaconda Prompt (or Terminal if on Mac) and type <code>pip install beautifulsoup4</code>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Making a Request\n",
    "\n",
    "This is how to make a request and download the HTML from the website. We will be using Wikipedia for our example as it is a fairly easy site to scrape from. Lets use the machine learning wiki at: https://en.wikipedia.org/wiki/Machine_learning"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'requests.models.Response'>\n"
     ]
    }
   ],
   "source": [
    "url = 'https://en.wikipedia.org/wiki/Machine_learning'\n",
    "results = req.get(url)\n",
    "print(type(results))\n",
    "#results.text   # the .text attribute shows the text stores in this req object."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Basics of BeatifulSoup4 \n",
    "\n",
    "So we now have a variable 'results' that basically spits out the HTML from this website. This is kind of daunting and unhelpful but luckily we can use BeautifulSoup for this. This library is essentially an HTML \"beautifier\" that makes working with HTML bearable."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now lets convert our object to type requests to one of type bs4."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'bs4.BeautifulSoup'>\n"
     ]
    }
   ],
   "source": [
    "soup = bs(results.text, \"lxml\")\n",
    "print(type(soup))  # confirms conversion between requests to bs4\n",
    "#soup"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Select\n",
    "\n",
    "We can select certain tags from HTML. Helpful to use the HTML inspector in a web browser like Chrome or Safari. In this case let's select all the headers on the page so we can see the different topics that the wiki page contains. Select will pull all the items given the tag and put them into an array."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<span class=\"mw-headline\" id=\"Overview\">Overview</span>,\n",
       " <span class=\"mw-headline\" id=\"Machine_learning_tasks\">Machine learning tasks</span>,\n",
       " <span class=\"mw-headline\" id=\"Machine_learning_applications\">Machine learning applications</span>,\n",
       " <span class=\"mw-headline\" id=\"History_and_relationships_to_other_fields\">History and relationships to other fields</span>,\n",
       " <span class=\"mw-headline\" id=\"Relation_to_statistics\">Relation to statistics</span>,\n",
       " <span class=\"mw-headline\" id=\"Theory\"><span id=\"Generalization\"></span> Theory</span>,\n",
       " <span class=\"mw-headline\" id=\"Approaches\">Approaches</span>,\n",
       " <span class=\"mw-headline\" id=\"Decision_tree_learning\">Decision tree learning</span>,\n",
       " <span class=\"mw-headline\" id=\"Association_rule_learning\">Association rule learning</span>,\n",
       " <span class=\"mw-headline\" id=\"Artificial_neural_networks\">Artificial neural networks</span>,\n",
       " <span class=\"mw-headline\" id=\"Deep_learning\">Deep learning</span>,\n",
       " <span class=\"mw-headline\" id=\"Inductive_logic_programming\">Inductive logic programming</span>,\n",
       " <span class=\"mw-headline\" id=\"Support_vector_machines\">Support vector machines</span>,\n",
       " <span class=\"mw-headline\" id=\"Clustering\">Clustering</span>,\n",
       " <span class=\"mw-headline\" id=\"Bayesian_networks\">Bayesian networks</span>,\n",
       " <span class=\"mw-headline\" id=\"Representation_learning\">Representation learning</span>,\n",
       " <span class=\"mw-headline\" id=\"Similarity_and_metric_learning\">Similarity and metric learning</span>,\n",
       " <span class=\"mw-headline\" id=\"Sparse_dictionary_learning\">Sparse dictionary learning</span>,\n",
       " <span class=\"mw-headline\" id=\"Genetic_algorithms\">Genetic algorithms</span>,\n",
       " <span class=\"mw-headline\" id=\"Rule-based_machine_learning\">Rule-based machine learning</span>,\n",
       " <span class=\"mw-headline\" id=\"Learning_classifier_systems\">Learning classifier systems</span>,\n",
       " <span class=\"mw-headline\" id=\"Applications\">Applications</span>,\n",
       " <span class=\"mw-headline\" id=\"Limitations\">Limitations</span>,\n",
       " <span class=\"mw-headline\" id=\"Bias\">Bias</span>,\n",
       " <span class=\"mw-headline\" id=\"Model_assessments\">Model assessments</span>,\n",
       " <span class=\"mw-headline\" id=\"Ethics\">Ethics</span>,\n",
       " <span class=\"mw-headline\" id=\"Software\">Software</span>,\n",
       " <span class=\"mw-headline\" id=\"Free_and_open-source_software\">Free and open-source software</span>,\n",
       " <span class=\"mw-headline\" id=\"Proprietary_software_with_free_and_open-source_editions\">Proprietary software with free and open-source editions</span>,\n",
       " <span class=\"mw-headline\" id=\"Proprietary_software\">Proprietary software</span>,\n",
       " <span class=\"mw-headline\" id=\"Journals\">Journals</span>,\n",
       " <span class=\"mw-headline\" id=\"Conferences\">Conferences</span>,\n",
       " <span class=\"mw-headline\" id=\"See_also\">See also</span>,\n",
       " <span class=\"mw-headline\" id=\"References\">References</span>,\n",
       " <span class=\"mw-headline\" id=\"Further_reading\">Further reading</span>,\n",
       " <span class=\"mw-headline\" id=\"External_links\">External links</span>]"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Selecting all the headlines\n",
    "soup.select(\".mw-headline\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice that <code>select()</code> returns a list of bs4.Tag"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'list'>\n",
      "<class 'bs4.element.Tag'>\n"
     ]
    }
   ],
   "source": [
    "print(type(soup.select(\".mw-headline\")))\n",
    "print(type(soup.select(\".mw-headline\")[0]))  # Using index 0 to just see the object type"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Overview\n",
      "Machine learning tasks\n",
      "Machine learning applications\n",
      "History and relationships to other fields\n",
      "Relation to statistics\n",
      " Theory\n",
      "Approaches\n",
      "Decision tree learning\n",
      "Association rule learning\n",
      "Artificial neural networks\n",
      "Deep learning\n",
      "Inductive logic programming\n",
      "Support vector machines\n",
      "Clustering\n",
      "Bayesian networks\n",
      "Representation learning\n",
      "Similarity and metric learning\n",
      "Sparse dictionary learning\n",
      "Genetic algorithms\n",
      "Rule-based machine learning\n",
      "Learning classifier systems\n",
      "Applications\n",
      "Limitations\n",
      "Bias\n",
      "Model assessments\n",
      "Ethics\n",
      "Software\n",
      "Free and open-source software\n",
      "Proprietary software with free and open-source editions\n",
      "Proprietary software\n",
      "Journals\n",
      "Conferences\n",
      "See also\n",
      "References\n",
      "Further reading\n",
      "External links\n"
     ]
    }
   ],
   "source": [
    "# List all the titles in readable format\n",
    "for x in soup.select(\".mw-headline\"):\n",
    "    print(x.text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Web Scraping for Real\n",
    "\n",
    "Now that we have the basics, lets try web scraping a real site! With permission of course. Lets use this website we are allowed to scrape. Here is the link: http://books.toscrape.com/catalogue/category/books_1/index.html"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "url = \"http://books.toscrape.com/catalogue/category/books_1/index.html\"\n",
    "results = req.get(url)\n",
    "soup = bs(results.text, \"lxml\")\n",
    "#soup"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Lets get all the product information we can. Seems like each item is separated into a \"product_pod\"**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "products = soup.select(\".product_pod\")\n",
    "#products"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Scraping titles**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['A Light in the ...',\n",
       " 'Tipping the Velvet',\n",
       " 'Soumission',\n",
       " 'Sharp Objects',\n",
       " 'Sapiens: A Brief History ...',\n",
       " 'The Requiem Red',\n",
       " 'The Dirty Little Secrets ...',\n",
       " 'The Coming Woman: A ...',\n",
       " 'The Boys in the ...',\n",
       " 'The Black Maria',\n",
       " 'Starving Hearts (Triangular Trade ...',\n",
       " \"Shakespeare's Sonnets\",\n",
       " 'Set Me Free',\n",
       " \"Scott Pilgrim's Precious Little ...\",\n",
       " 'Rip it Up and ...',\n",
       " 'Our Band Could Be ...',\n",
       " 'Olio',\n",
       " 'Mesaerion: The Best Science ...',\n",
       " 'Libertarianism for Beginners',\n",
       " \"It's Only the Himalayas\"]"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "titles = [x.text for x in soup.find_all(\"h3\")]\n",
    "titles"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Scraping prices**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['51.77',\n",
       " '53.74',\n",
       " '50.10',\n",
       " '47.82',\n",
       " '54.23',\n",
       " '22.65',\n",
       " '33.34',\n",
       " '17.93',\n",
       " '22.60',\n",
       " '52.15',\n",
       " '13.99',\n",
       " '20.66',\n",
       " '17.46',\n",
       " '52.29',\n",
       " '35.02',\n",
       " '57.25',\n",
       " '23.88',\n",
       " '37.59',\n",
       " '51.33',\n",
       " '45.17']"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "price = [x.text[2:] for x in soup.find_all('p', attrs={'class': 'price_color'})]\n",
    "price"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Scraping availability**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['In stock',\n",
       " 'In stock',\n",
       " 'In stock',\n",
       " 'In stock',\n",
       " 'In stock',\n",
       " 'In stock',\n",
       " 'In stock',\n",
       " 'In stock',\n",
       " 'In stock',\n",
       " 'In stock',\n",
       " 'In stock',\n",
       " 'In stock',\n",
       " 'In stock',\n",
       " 'In stock',\n",
       " 'In stock',\n",
       " 'In stock',\n",
       " 'In stock',\n",
       " 'In stock',\n",
       " 'In stock',\n",
       " 'In stock']"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "instock = [x.text[15:23] for x in soup.find_all('p', attrs={'class': 'instock availability'})]\n",
    "instock\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Scraping multiple pages\n",
    "\n",
    "Sadly there isn't really an easy function to do this. Atleast none that I have seen"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "# We can't easily include the first page in the link becasue it has \"index.html\" rather than the regular file\n",
    "\n",
    "# Loops through a formatted url string. Will take a while to run\n",
    "for page in range(2,51):\n",
    "    url = \"http://books.toscrape.com/catalogue/category/books_1/page-\" + str(page) +\".html\"\n",
    "    results = req.get(url)\n",
    "    soup = bs(results.text, \"lxml\")\n",
    "    titles.extend([x.text for x in soup.find_all(\"h3\")])\n",
    "    price.extend([x.text[2:] for x in soup.find_all('p', attrs={'class': 'price_color'})])\n",
    "    instock.extend([x.text[15:23] for x in soup.find_all('p', attrs={'class': 'instock availability'})])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Creating your data\n",
    "\n",
    "Now that we have retrieved our data lets create our dataframe and do some cleaning"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>title</th>\n",
       "      <th>price</th>\n",
       "      <th>availability</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>A Light in the ...</td>\n",
       "      <td>51.77</td>\n",
       "      <td>In stock</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Tipping the Velvet</td>\n",
       "      <td>53.74</td>\n",
       "      <td>In stock</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Soumission</td>\n",
       "      <td>50.10</td>\n",
       "      <td>In stock</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Sharp Objects</td>\n",
       "      <td>47.82</td>\n",
       "      <td>In stock</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Sapiens: A Brief History ...</td>\n",
       "      <td>54.23</td>\n",
       "      <td>In stock</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                          title  price availability\n",
       "0            A Light in the ...  51.77     In stock\n",
       "1            Tipping the Velvet  53.74     In stock\n",
       "2                    Soumission  50.10     In stock\n",
       "3                 Sharp Objects  47.82     In stock\n",
       "4  Sapiens: A Brief History ...  54.23     In stock"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "\n",
    "d = {\"title\": titles, \"price\": price, \"availability\": instock}\n",
    "books = pd.DataFrame(data=d)\n",
    "books.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "str"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Remember that the prices were stored as String\n",
    "type(books['price'][0])           "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "count    1000.0\n",
       "mean       35.0\n",
       "std        14.0\n",
       "min        10.0\n",
       "25%        22.0\n",
       "50%        36.0\n",
       "75%        47.0\n",
       "max        60.0\n",
       "Name: price, dtype: float64"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Lets clean this real quick\n",
    "books[\"price\"] = [float(x) for x in books['price']]\n",
    "books['price'].describe().round()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>title</th>\n",
       "      <th>price</th>\n",
       "      <th>availability</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>A Light in the ...</td>\n",
       "      <td>51.77</td>\n",
       "      <td>In stock</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Tipping the Velvet</td>\n",
       "      <td>53.74</td>\n",
       "      <td>In stock</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Soumission</td>\n",
       "      <td>50.10</td>\n",
       "      <td>In stock</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Sharp Objects</td>\n",
       "      <td>47.82</td>\n",
       "      <td>In stock</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Sapiens: A Brief History ...</td>\n",
       "      <td>54.23</td>\n",
       "      <td>In stock</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                          title  price availability\n",
       "0            A Light in the ...  51.77     In stock\n",
       "1            Tipping the Velvet  53.74     In stock\n",
       "2                    Soumission  50.10     In stock\n",
       "3                 Sharp Objects  47.82     In stock\n",
       "4  Sapiens: A Brief History ...  54.23     In stock"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "books.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we have data we can actually work with. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Visualization\n",
    "\n",
    "Lets make a boxplot of all the prices to check for outliers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAu8AAADYCAYAAABbYw1tAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAADh9JREFUeJzt3X9s1fW5wPGntAeBpMgqJU7DJbphNGyK0TtHJAK7BpGCGEbCj8yasP1hYoZxyQQZumSAQiVh/oj/uM3EzC1uMUrAzbho3FUgLDMKQfGKOtpq6lac5YdSeijf+4dZg9NcJ+P224fzev3FOTnkPIc8OZ83X74pdUVRFAEAAAx5w8oeAAAA+NeIdwAASEK8AwBAEuIdAACSEO8AAJCEeAcAgCTEOwAAJCHeAQAgCfEOAABJiHcAAEhCvAMAQBINJ/ObursPneo5vrAvfWlUfPDBR2WPwRBhHziRfeBE9oET2QdONBT2obm58Qu9Pu2V94aG+rJHYAixD5zIPnAi+8CJ7AMnyrgPaeMdAABqjXgHAIAkxDsAACQh3gEAIAnxDgAASYh3AABIQrwDAEAS4h0AAJIQ7wAAkIR4BwCAJMQ7AAAkId4BACAJ8Q4AAEmIdwAASKKh7AEgm1/96pHo7Gwvewz+D5VKfVSr/WWPkc6BAz0REXHmmWNKnuTUsg9Dw/jxE2LJktayx4D0xDt8QZ2d7fE/e9+M+hGnV+BAf+/H8d598FjJk3C6+cduAf8+8Q4noX7EmBg14b/KHgNOqY/an42IsNuccv/YLeDf5553AABIQrwDAEAS4h0AAJIQ7wAAkIR4BwCAJMQ7AAAkId4BACAJ8Q4AAEmIdwAASEK8AwBAEuIdAACSEO8AAJCEeAcAgCTEOwAAJCHeAQAgCfEOAABJiHcAAEhCvAMAQBLiHQAAkhDvAACQhHgHAIAkxDsAACQh3gEAIAnxDgAASYh3AABIQrwDAEAS4h0AAJIQ7wAAkIR4BwCAJBrKHuBkbN363zF69Mj4+tf/s+xRAABIKGtPpoz3F1/8Y1Qq9en+sAEAGBqy9qTbZgAAIAnxDgAASYh3AABIQrwDAEAS4h0AAJIQ7wAAkIR4BwCAJMQ7AAAkId4BACAJ8Q4AAEmIdwAASEK8AwBAEuIdAACSEO8AAJCEeAcAgCTEOwAAJCHeAQAgCfEOAABJiHcAAEhCvAMAQBLiHQAAkhDvAACQhHgHAIAkxDsAACQh3gEAIAnxDgAASYh3AABIQrwDAEAS4h0AAJIQ7wAAkIR4BwCAJBrKHuBkHDjQEwcPHoj161eXPQpDRKVSH9Vq/6C8V0dHexzvrx+U9wI4HRw/1hsdHe1D4twezPOCoa2joz3OOqup7DG+MFfeAQAgiZRX3s88c0yMHXtW/OAHK8sehSGiubkxursPDcp7rV+/Ot7s3D8o7wVwOhjWMCL+Y/zYWL78jrJHGdTzgqFt/frVUank+5d0V94BACAJ8Q4AAEmIdwAASEK8AwBAEuIdAACSEO8AAJCEeAcAgCTEOwAAJCHeAQAgCfEOAABJiHcAAEhCvAMAQBLiHQAAkhDvAACQhHgHAIAkxDsAACQh3gEAIAnxDgAASYh3AABIQrwDAEAS4h0AAJIQ7wAAkIR4BwCAJMQ7AAAkId4BACAJ8Q4AAEmIdwAASEK8AwBAEuIdAACSEO8AAJBEQ9kDnIypU6fF6NEjyx4DAICksvZkyni/8sqrorm5Mbq7D5U9CgAACWXtSbfNAABAEuIdAACSEO8AAJCEeAcAgCTEOwAAJCHeAQAgCfEOAABJiHcAAEhCvAMAQBLiHQAAkhDvAACQhHgHAIAkxDsAACQh3gEAIAnxDgAASYh3AABIQrwDAEAS4h0AAJIQ7wAAkIR4BwCAJMQ7AAAkId4BACAJ8Q4AAEmIdwAASEK8AwBAEuIdAACSEO8AAJCEeAcAgCTEOwAAJCHeAQAgiYayB4CM+nt74qP2Z8seA06p/t6eiAi7zSn38W6NLXsMOC2Id/iCxo+fUPYIfI5KpT6q1f6yx0jnwIGPj4QzzxxT8iSnln0YCsb67oRTRLzDF7RkSWvZI/A5mpsbo7v7UNljMETYB+B04p53AABIQrwDAEAS4h0AAJIQ7wAAkIR4BwCAJMQ7AAAkId4BACAJ8Q4AAEmIdwAASEK8AwBAEuIdAACSEO8AAJCEeAcAgCTEOwAAJFFXFEVR9hAAAMDnc+UdAACSEO8AAJCEeAcAgCTEOwAAJCHeAQAgCfEOAABJiHcAAEhCvAMAQBLiHQAAkhDvAACQRJp4P3z4cMyZMyfeeeediIjYtm1bzJ07N2bOnBkbN24seToG0wMPPBAtLS3R0tISbW1tEWEfatm9994bs2fPjpaWlnj44Ycjwj4QsX79+lixYkVEROzZsyfmz58f11xzTfzoRz+KY8eOlTwdg+WGG26IlpaWmDdvXsybNy927twZmzdvjtmzZ8fMmTPj0UcfLXtEBtFzzz0X8+fPj2uvvTbWrFkTEUnPiyKBV155pZgzZ04xadKkorOzszhy5Egxbdq0oqOjo6hWq8XSpUuL559/vuwxGQRbt24tFi5cWBw9erTo6+srWltbi82bN9uHGrVjx45i0aJFRbVaLY4cOVLMmDGj2LNnj32ocdu2bSuuuOKKYvny5UVRFEVLS0vx8ssvF0VRFLfffnvx6KOPljkeg+T48ePF1KlTi2q1OvDce++9V8yYMaP44IMPig8//LCYO3dusXfv3hKnZLB0dHQUU6dOLbq6uoq+vr5i8eLFxfPPP5/yvEhx5f03v/lN/PjHP45x48ZFRMSuXbtiwoQJMX78+GhoaIi5c+fG008/XfKUDIbm5uZYsWJFDB8+PCqVSnzlK1+Jffv22Yca9Y1vfCMeeeSRaGhoiPfffz/6+/vj4MGD9qGG9fT0xMaNG+Omm26KiIh33303ent7Y/LkyRERMX/+fPtQI95+++2IiFi6dGlcd9118ctf/jK2bdsW3/zmN2PMmDExatSouOaaa+xDjfjDH/4Qs2fPjrPPPjsqlUps3LgxRo4cmfK8aCh7gH/F2rVrP/H4b3/7WzQ3Nw88HjduXPz1r38d7LEowcSJEwd+vW/fvvj9738f3/nOd+xDDatUKnHffffFL37xi5g1a5bvhxp35513xq233hpdXV0R8enzorm52T7UiIMHD8aUKVPijjvuiGq1Gq2trXHttdd+6vth165dJU7JYGlvb49KpRI33XRTdHV1xfTp02PixIkpz4sUV97/2fHjx6Ourm7gcVEUn3jM6W/v3r2xdOnSuO2222L8+PH2ocYtW7Ystm/fHl1dXbFv3z77UKN++9vfxpe//OWYMmXKwHPOi9p16aWXRltbWzQ2NkZTU1MsWLAg7rvvPvtQo/r7+2P79u1x1113xWOPPRa7du2Kzs7OlPuQ4sr7Pzv77LOju7t74HF3d/fALTWc/l566aVYtmxZrFy5MlpaWuJPf/qTfahRb731VvT19cVFF10UI0eOjJkzZ8bTTz8d9fX1A6+xD7Xjd7/7XXR3d8e8efPiwIED8dFHH0VdXd0nvh/2799vH2rEn//856hWqwN/mSuKIs4991znRY0aO3ZsTJkyJZqamiIi4uqrr057XqS88n7JJZfEX/7yl2hvb4/+/v7YsmVLXHXVVWWPxSDo6uqKm2++OTZs2BAtLS0RYR9q2TvvvBOrVq2Kvr6+6Ovri2effTYWLVpkH2rUww8/HFu2bIlNmzbFsmXL4lvf+lbcfffdccYZZ8RLL70UERGbNm2yDzXi0KFD0dbWFkePHo3Dhw/HE088Effcc09s3749/v73v8eRI0fimWeesQ81YsaMGfHiiy/GwYMHo7+/P1544YWYNWtWyvMi5ZX3M844I9atWxff//734+jRozFt2rSYNWtW2WMxCH7+85/H0aNHY926dQPPLVq0yD7UqGnTpsWuXbvi+uuvj/r6+pg5c2a0tLREU1OTfWDAhg0bYtWqVXH48OGYNGlStLa2lj0Sg2DGjBmxc+fOuP766+P48eOxZMmSuOyyy+LWW2+N1tbWqFarsWDBgrj44ovLHpVBcMkll8T3vve9WLJkSVSr1bjyyitj8eLFcf7556c7L+qKoijKHgIAAPh8KW+bAQCAWiTeAQAgCfEOAABJiHcAAEhCvAMAQBLiHeA0c++998aTTz5Z9hgA/D/woyIBACCJlP9JE0At2rFjR2zYsCHOOeecePvtt2PEiBGxbt26eOihh6Knpyc6Oztj+vTp8f7778fEiRPju9/9buzcuTPWrFkTR44ciUqlErfddltMmTIl3nrrrVi7dm309PREf39/3HDDDbFgwYKyPyIAn0O8AySye/fuWL58eVx++eXx61//On74wx/GBRdcEL29vfHUU09FRMSKFSsiIqJarcbNN98ca9asienTp8fu3bvj9ttvj8cffzyWLVsWbW1tMWnSpDh06FAsXLgwvvrVr8bkyZPL/HgAfA7xDpDIhRdeGJdffnlERHz729+On/zkJzFu3Li47LLLPvXaN954I4YNGxbTp0+PiIivfe1rsXnz5njzzTejo6MjVq5cOfDa3t7eeO2118Q7wBAn3gESqa+v/9Rzw4YNi1GjRn3ma+vq6j7x3BtvvBFFUURjY2Ns2rRp4Pn9+/dHY2PjqR8YgFPKT5sBSOT111+P119/PSIiHnvssbj00ktj9OjRn/na888/P+rq6mLr1q0REfHqq6/GjTfeGOedd16MGDFiIN67urpizpw5sXv37sH5EACcNFfeARIZO3Zs/PSnP4133303mpqaoq2tLR544IHPfO3w4cPj/vvvj7vuuiva2tqiUqnE/fffH8OHD48HH3ww1q5dGz/72c/i2LFjccstt3zmrTcADC1+VCRAEjt27IjVq1fHli1byh4FgJK4bQYAAJJw5R0AAJJw5R0AAJIQ7wAAkIR4BwCAJMQ7AAAkId4BACCJ/wWVHdfZV3CobQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 936x216 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "sns.set(style = 'darkgrid', color_codes = True)\n",
    "f, ax = plt.subplots(figsize=(13, 3))\n",
    "sns.despine(f, left=True, bottom=True)\n",
    "\n",
    "boxplt = sns.boxplot(books[\"price\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Thank You for Coming!\n",
    "\n",
    "Visit us at https://www.dsiufl.org"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}