Top_Ranked_Gutenberg_Ebooks_Download.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Top Gutenberg Ebooks (yesterday's ranking) download\n",
    "\n",
    "## Tirthajyoti Sarkar, Sunnyvale, CA, 2017\n",
    "\n",
    "### What is Project Gutenberg? - \n",
    "Project Gutenberg is a volunteer effort to digitize and archive cultural works, to \"encourage the creation and distribution of eBooks\". It was founded in 1971 by American writer Michael S. Hart and is the **oldest digital library.** This longest-established ebook project releases books that entered the public domain, and can be freely read or downloaded in various electronic formats.\n",
    "\n",
    "* **This starter code scrapes the url of the Project Gutenberg's Top 100 ebooks (yesterday's ranking) for identifying the ebook links. **\n",
    "* **It uses BeautifulSoup4 for parsing the HTML and regular expression code for identifying the Top 100 ebook file numbers.**\n",
    "* **It includes a function to take an usser input on how many books to download and then crawls the server to download them in a dictionary object.**\n",
    "* **Finally, it also includes a function to save the downloaded Ebooks as text files in a local directory**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import urllib.request, urllib.parse, urllib.error\n",
    "from bs4 import BeautifulSoup\n",
    "import ssl\n",
    "import re"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Ignore SSL certificate errors"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Ignore SSL certificate errors\n",
    "ctx = ssl.create_default_context()\n",
    "ctx.check_hostname = False\n",
    "ctx.verify_mode = ssl.CERT_NONE"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Read the HTML from the URL and pass on to BeautifulSoup"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Opening the file connection to https://www.gutenberg.org/browse/scores/top\n",
      "Connection established and HTML parsed...\n"
     ]
    }
   ],
   "source": [
    "# Read the HTML from the URL and pass on to BeautifulSoup\n",
    "top100url = 'https://www.gutenberg.org/browse/scores/top'\n",
    "url = top100url\n",
    "print(f\"Opening the file connection to {url}\")\n",
    "html = urllib.request.urlopen(url, context=ctx).read()\n",
    "soup = BeautifulSoup(html, 'html.parser')\n",
    "print(\"Connection established and HTML parsed...\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Find all the _'href'_ tags and store them in the list of links"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Empty list to hold all the http links in the HTML page\n",
    "lst_links=[]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Find all the href tags and store them in the list of links\n",
    "for link in soup.find_all('a'):\n",
    "    #print(link.get('href'))\n",
    "    lst_links.append(link.get('href'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Use regular expression to find the numeric digits in these links. These are the file number for the Top 100 books."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Use regular expression to find the numeric digits in these links. These are the file number for the Top 100 books. \n",
    "# Initialize empty list to hold the file numbers\n",
    "booknum=[]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "The file numbers for the top 100 ebooks on Gutenberg are shown below\n",
      "----------------------------------------------------------------------\n",
      "[1342, 84, 1080, 46, 219, 2542, 98, 345, 2701, 844, 11, 5200, 43, 16328, 76, 74, 1952, 6130, 2591, 1661, 41, 174, 23, 1260, 1497, 408, 3207, 1400, 30254, 58271, 1232, 25344, 58269, 158, 44881, 1322, 205, 2554, 1184, 2600, 120, 16, 58276, 5740, 34901, 28054, 829, 33, 2814, 4300, 100, 55, 160, 1404, 786, 58267, 3600, 19942, 8800, 514, 244, 2500, 2852, 135, 768, 58263, 1251, 3825, 779, 58262, 203, 730, 20203, 35, 1250, 45, 161, 30360, 7370, 58274, 209, 27827, 58256, 33283, 4363, 375, 996, 58270, 521, 58268, 36, 815, 1934, 3296, 58279, 105, 2148, 932, 1064, 13415]\n"
     ]
    }
   ],
   "source": [
    "# Number 19 to 119 in the original list of links have the Top 100 books' number.\n",
    "for i in range(19,119):\n",
    "\tlink=lst_links[i]\n",
    "\tlink=link.strip()\n",
    "\t# Regular expression to find the numeric digits in the link (href) string\n",
    "\tn=re.findall('[0-9]+',link)\n",
    "\tif len(n)==1:\n",
    "\t\t# Append the filenumber casted as integer\n",
    "\t\tbooknum.append(int(n[0]))\n",
    "\n",
    "print (\"\\nThe file numbers for the top 100 ebooks on Gutenberg are shown below\\n\"+\"-\"*70)\n",
    "print(booknum)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Search in the extracted text (using Regular Expression) from the soup object to find the names of top 100 Ebooks (Yesterday's rank)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "start_idx=soup.text.splitlines().index('Top 100 EBooks yesterday')\n",
    "lst_titles_temp=[] # Empty list of Ebook names\n",
    "for i in range(100):\n",
    "    lst_titles_temp.append(soup.text.splitlines()[start_idx+2+i])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Pride and Prejudice by Jane Austen \n",
      "Frankenstein\n",
      "A Modest Proposal by Jonathan Swift \n",
      "A Christmas Carol in Prose\n",
      "Heart of Darkness by Joseph Conrad \n",
      "Et dukkehjem\n",
      "A Tale of Two Cities by Charles Dickens \n",
      "Dracula by Bram Stoker \n",
      "Moby Dick\n",
      "The Importance of Being Earnest\n",
      "Alice\n",
      "Metamorphosis by Franz Kafka \n",
      "The Strange Case of Dr\n",
      "Beowulf\n",
      "Adventures of Huckleberry Finn by Mark Twain \n",
      "The Adventures of Tom Sawyer by Mark Twain \n",
      "The Yellow Wallpaper by Charlotte Perkins Gilman \n",
      "The Iliad by Homer \n",
      "Grimms\n",
      "The Adventures of Sherlock Holmes by Arthur Conan Doyle \n",
      "The Legend of Sleepy Hollow by Washington Irving \n",
      "The Picture of Dorian Gray by Oscar Wilde \n",
      "Narrative of the Life of Frederick Douglass\n",
      "Jane Eyre\n",
      "The Republic by Plato \n",
      "The Souls of Black Folk by W\n",
      "Leviathan by Thomas Hobbes \n",
      "Great Expectations by Charles Dickens \n",
      "The Romance of Lust\n",
      "The Tower of London by William Benham \n",
      "Il Principe\n",
      "The Scarlet Letter by Nathaniel Hawthorne \n",
      "\n",
      "Emma by Jane Austen \n",
      "Confessions of a Thug by Meadows Taylor \n",
      "Leaves of Grass by Walt Whitman \n",
      "Walden\n",
      "Prestuplenie i nakazanie\n",
      "The Count of Monte Cristo\n",
      "War and Peace by graf Leo Tolstoy \n",
      "Treasure Island by Robert Louis Stevenson \n",
      "Peter Pan by J\n",
      "The Florist and Horticultural Journal\n",
      "Tractatus Logico\n",
      "On Liberty by John Stuart Mill \n",
      "The Brothers Karamazov by Fyodor Dostoyevsky \n",
      "Gulliver\n",
      "The Scarlet Letter by Nathaniel Hawthorne \n",
      "Dubliners by James Joyce \n",
      "Ulysses by James Joyce \n",
      "The Complete Works of William Shakespeare by William Shakespeare \n",
      "The Wonderful Wizard of Oz by L\n",
      "The Awakening\n",
      "The Federalist Papers by Alexander Hamilton and John Jay and James Madison \n",
      "Hard Times by Charles Dickens \n",
      "The Delinquent \n",
      "Essays of Michel de Montaigne \n",
      "Candide by Voltaire \n",
      "The Divine Comedy by Dante\n",
      "Little Women by Louisa May Alcott \n",
      "A Study in Scarlet by Arthur Conan Doyle \n",
      "Siddhartha by Hermann Hesse \n",
      "The Hound of the Baskervilles by Arthur Conan Doyle \n",
      "Les Mis\n",
      "Wuthering Heights by Emily Bront\n",
      "The Candle and the Cat by Mary Finley Leonard \n",
      "Le Morte d\n",
      "Pygmalion by Bernard Shaw \n",
      "The Tragical History of Doctor Faustus by Christopher Marlowe \n",
      "Captain John\n",
      "Uncle Tom\n",
      "Oliver Twist by Charles Dickens \n",
      "Autobiography of Benjamin Franklin by Benjamin Franklin \n",
      "The Time Machine by H\n",
      "Anthem by Ayn Rand \n",
      "Anne of Green Gables by L\n",
      "Sense and Sensibility by Jane Austen \n",
      "My Secret Life\n",
      "Second Treatise of Government by John Locke \n",
      "The Tragic Story of the Empress of Ireland by Logan Marshall \n",
      "The Turn of the Screw by Henry James \n",
      "The Kama Sutra of Vatsyayana by Vatsyayana \n",
      "The Russian Army and the Japanese War\n",
      "Calculus Made Easy by Silvanus P\n",
      "Beyond Good and Evil by Friedrich Wilhelm Nietzsche \n",
      "An Occurrence at Owl Creek Bridge by Ambrose Bierce \n",
      "Don Quixote by Miguel de Cervantes Saavedra \n",
      "Blue Jackets by Edward Greey \n",
      "The Life and Adventures of Robinson Crusoe by Daniel Defoe \n",
      "The Waterloo Campaign \n",
      "The War of the Worlds by H\n",
      "Democracy in America \n",
      "Songs of Innocence\n",
      "The Confessions of St\n",
      "Modern French Masters by Marie Van Vorst \n",
      "Persuasion by Jane Austen \n",
      "The Works of Edgar Allan Poe \n",
      "The Fall of the House of Usher by Edgar Allan Poe \n",
      "The Masque of the Red Death by Edgar Allan Poe \n",
      "The Lady with the Dog and Other Stories by Anton Pavlovich Chekhov \n"
     ]
    }
   ],
   "source": [
    "# Use regular expression to extract only text from the name strings and append to an empty list\n",
    "lst_titles=[]\n",
    "for i in range(100):\n",
    "    id1,id2=re.match('^[a-zA-Z ]*',lst_titles_temp[i]).span()\n",
    "    lst_titles.append(lst_titles_temp[i][id1:id2])\n",
    "for l in lst_titles:\n",
    "    print(l)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Define a function that takes an user input of how many top books to download and crawls the server to download"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "code_folding": []
   },
   "outputs": [],
   "source": [
    "def download_top_books(num_download=10,verbosity=0):\n",
    "    \"\"\"\n",
    "    Function: Download top N books from Gutenberg.org where N is specified by user\n",
    "    Verbosity: If verbosity is turned on (set to 1) then prints the downloading status for every book\n",
    "    Returns: Returns a dictionary where keys are the names of the books and values are the raw text.\n",
    "    Exception Handling: If a book is not found on the server (due to broken link or whatever reason), inserts \"NOT FOUND\" as the text.\n",
    "    \"\"\"\n",
    "    topEBooks = {}\n",
    "    \n",
    "    if num_download<=0:\n",
    "        print(\"I guess no download is necessary\")\n",
    "        return topEBooks\n",
    "    \n",
    "    if num_download>100:\n",
    "        print(\"You asked for more than 100 downloads.\\nUnfortunately, Gutenberg ranks only top 100 books.\\nProceeding to download top 100 books.\")\n",
    "        num_download=100\n",
    "    \n",
    "    # Base URL for files repository\n",
    "    baseurl= 'http://www.gutenberg.org/files/'\n",
    "    \n",
    "    if verbosity==1:\n",
    "        count_done=0\n",
    "        for i in range(num_download):\n",
    "            print (\"Working on book:\", lst_titles[i])\n",
    "            \n",
    "            # Create the proper download link (url) from the book id\n",
    "            # You have to examine the Gutenberg.org file structure carefully to come up with the proper url\n",
    "            bookid=booknum[i]\n",
    "            bookurl= baseurl+str(bookid)+'/'+str(bookid)+'-0.txt'\n",
    "            # Create a file handler object\n",
    "            try:\n",
    "                fhand = urllib.request.urlopen(bookurl)\n",
    "                txt_dump = ''\n",
    "                # Iterate over the lines in the file handler object and dump the data into the text string\n",
    "                for line in fhand:\n",
    "                    # Use decode method to convert the UTF-8 to Unicode string\n",
    "                    txt_dump+=line.decode()\n",
    "                # Add downloaded text to the dictionary with keys matching the list of book titles.\n",
    "                # This puts the raw text as the value of the key of the dictionary bearing the name of the Ebook \n",
    "                topEBooks[lst_titles[i]]=txt_dump\n",
    "                count_done+=1\n",
    "                print (f\"Finished downloading {round(100*count_done/num_download,2)}%\")\n",
    "            except urllib.error.URLError as e:\n",
    "                topEBooks[lst_titles[i]]=\"NOT FOUND\"\n",
    "                count_done+=1\n",
    "                print(f\"**ERROR: {lst_titles[i]} {e.reason}**\")\n",
    "    else:\n",
    "        count_done=0\n",
    "        from tqdm import tqdm, tqdm_notebook\n",
    "        for i in tqdm(range(num_download),desc='Download % completed',dynamic_ncols=True):\n",
    "            # Create the proper download link (url) from the book id\n",
    "            # You have to examine the Gutenberg.org file structure carefully to come up with the proper url\n",
    "            bookid=booknum[i]\n",
    "            bookurl= baseurl+str(bookid)+'/'+str(bookid)+'-0.txt'\n",
    "            # Create a file handler object\n",
    "            try:\n",
    "                fhand = urllib.request.urlopen(bookurl)\n",
    "                txt_dump = ''\n",
    "                # Iterate over the lines in the file handler object and dump the data into the text string\n",
    "                for line in fhand:\n",
    "                    # Use decode method to convert the UTF-8 to Unicode string\n",
    "                    txt_dump+=line.decode()\n",
    "                # Add downloaded text to the dictionary with keys matching the list of book titles.\n",
    "                # This puts the raw text as the value of the key of the dictionary bearing the name of the Ebook \n",
    "                topEBooks[lst_titles[i]]=txt_dump\n",
    "                count_done+=1\n",
    "            except urllib.error.URLError as e:\n",
    "                topEBooks[lst_titles[i]]=\"NOT FOUND\"\n",
    "                count_done+=1\n",
    "                print(f\"**ERROR: {lst_titles[i]} {e.reason}**\")\n",
    "        \n",
    "    print (\"-\"*40+\"\\nFinished downloading all books!\\n\"+\"-\"*40)\n",
    "       \n",
    "    return (topEBooks)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Test the function with verbosity=0 (default)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Download % completed: 100%|██████████████████████████████████████████████████████████████| 1/1 [00:01<00:00,  1.83s/it]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "----------------------------------------\n",
      "Finished downloading all books!\n",
      "----------------------------------------\n"
     ]
    }
   ],
   "source": [
    "dict_books=download_top_books(1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Test the function with verbosity=1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 272,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "You asked for more than 100 downloads. \n",
      "Unfortunately, Gutenberg ranks only top 100 books.\n",
      "Proceeding to download 100 books.\n"
     ]
    }
   ],
   "source": [
    "dict_books=download_top_books(105,1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Show the final dictionary and an example of the downloaded text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "﻿The Project Gutenberg EBook of Pride and Prejudice, by Jane Austen\n",
      "\n",
      "This eBook is for the use of anyone anywhere at no cost and with\n",
      "almost no restrictions whatsoever.  You may copy it, give it away or\n",
      "re-use it under the terms of the Project Gutenberg License included\n",
      "with this eBook or online at www.gutenberg.org\n",
      "\n",
      "\n",
      "Title: Pride and Prejudice\n",
      "\n",
      "Author: Jane Austen\n",
      "\n",
      "Posting Date: August 26, 2008 [EBook #1342]\n",
      "Release Date: June, 1998\n",
      "Last Updated: March 10, 2018\n",
      "\n",
      "Language: English\n",
      "\n",
      "Character set encoding: UTF-8\n",
      "\n",
      "*** START OF THIS PROJECT GUTENBERG EBOOK PRIDE AND PREJUDICE ***\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "Produced by Anonymous Volunteers\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "PRIDE AND PREJUDICE\n",
      "\n",
      "By Jane Austen\n",
      "\n",
      "\n",
      "\n",
      "Chapter 1\n",
      "\n",
      "\n",
      "It is a truth universally acknowledged, that a single man in possession\n",
      "of a good fortune, must be in want of a wife.\n",
      "\n",
      "However little known the feelings or views of such a man may be on his\n",
      "first entering a neighbourhood, this truth is so well fixed in the minds\n",
      "of the surrounding families, that he is considered the rightful property\n",
      "of some one or other of their daughters.\n",
      "\n",
      "“My dear Mr. Bennet,” said his lady to him one day, “have you heard that\n",
      "Netherfield Park is let at last?”\n",
      "\n",
      "Mr. Bennet replied that he had not.\n",
      "\n",
      "“But it is,” returned she; “for Mrs. Long has just been here, and she\n",
      "told me all about it.”\n",
      "\n",
      "Mr. Bennet made no answer.\n",
      "\n",
      "“Do you not want to know who has taken it?” cried his wife impatiently.\n",
      "\n",
      "“_You_ want to tell me, and I have no objecti\n"
     ]
    }
   ],
   "source": [
    "print(dict_books[lst_titles[0]][:1500])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Write a function to download and save the downloaded texts"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "code_folding": []
   },
   "outputs": [],
   "source": [
    "def save_text_files(num_download=10,verbosity=1):\n",
    "    \"\"\"\n",
    "    Downloads top N books from Gutenberg.org where N is specified by user.\n",
    "    If verbosity is turned on (set to 1) then prints the downloading status for every book.\n",
    "    Asks user for a location on computer where to save the downloaded Ebooks and process accordingly.\n",
    "    Returns status message indicating how many ebooks could be successfully downloaded and saved\n",
    "    \"\"\"\n",
    "    \n",
    "    import os\n",
    "    \n",
    "    # Download the Ebooks and save in a dictionary object (in-memory)\n",
    "    dict_books=download_top_books(num_download=num_download,verbosity=verbosity)\n",
    "    \n",
    "    if dict_books=={}:\n",
    "        return None\n",
    "    \n",
    "    # Ask use for a save location (directory path)\n",
    "    savelocation=input(\"Please enter a folder location to save the Ebooks in:\")\n",
    "    \n",
    "    count_successful_download=0\n",
    "    \n",
    "    # Create a default folder/directory in the current working directory if the input is blank\n",
    "    if (len(savelocation)<1):\n",
    "        savelocation=os.getcwd()+'\\\\'+'Ebooks'+'\\\\'\n",
    "        # Creates new directory if the directory does not exist. Otherwise, just use the existing path.\n",
    "        if not os.path.isdir(savelocation):\n",
    "            os.mkdir(savelocation)\n",
    "    else:\n",
    "        if savelocation[-1]=='\\\\':\n",
    "            os.mkdir(savelocation)\n",
    "        else:\n",
    "            os.mkdir(savelocation+'\\\\')\n",
    "    #print(\"Saving files at:\",savelocation)\n",
    "    for k,v in dict_books.items():\n",
    "        if (v!=\"NOT FOUND\"):\n",
    "            filename=savelocation+str(k)+'.txt'\n",
    "            file=open(filename,'wb')\n",
    "            file.write(v.encode(\"UTF-8\",'ignore'))\n",
    "            file.close()\n",
    "            count_successful_download+=1\n",
    "    \n",
    "    # Status message\n",
    "    print (f\"{count_successful_download} book(s) was/were successfully downloaded and saved to the location {savelocation}\")\n",
    "    if (num_download!=count_successful_download):\n",
    "        print(f\"{num_download-count_successful_download} books were not found on the server!\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Working on book: Pride and Prejudice by Jane Austen \n",
      "Finished downloading 1.0%\n",
      "Working on book: Heart of Darkness by Joseph Conrad \n",
      "Finished downloading 2.0%\n",
      "Working on book: Dracula by Bram Stoker \n",
      "**ERROR: Dracula by Bram Stoker  Not Found**\n",
      "Working on book: A Tale of Two Cities by Charles Dickens \n",
      "Finished downloading 4.0%\n",
      "Working on book: The Art of Being Happy by Joseph Droz \n",
      "Finished downloading 5.0%\n",
      "Working on book: Moby Dick\n",
      "Finished downloading 6.0%\n",
      "Working on book: Alice\n",
      "Finished downloading 7.0%\n",
      "Working on book: Metamorphosis by Franz Kafka \n",
      "**ERROR: Metamorphosis by Franz Kafka  Not Found**\n",
      "Working on book: Frankenstein\n",
      "Finished downloading 9.0%\n",
      "Working on book: The Romance of Lust\n",
      "Finished downloading 10.0%\n",
      "Working on book: The Importance of Being Earnest\n",
      "**ERROR: The Importance of Being Earnest Not Found**\n",
      "Working on book: The Adventures of Tom Sawyer by Mark Twain \n",
      "Finished downloading 12.0%\n",
      "Working on book: The Adventures of Sherlock Holmes by Arthur Conan Doyle \n",
      "**ERROR: The Adventures of Sherlock Holmes by Arthur Conan Doyle  Not Found**\n",
      "Working on book: Adventures of Huckleberry Finn by Mark Twain \n",
      "Finished downloading 14.0%\n",
      "Working on book: Grimms\n",
      "Finished downloading 15.0%\n",
      "Working on book: The Iliad by Homer \n",
      "Finished downloading 16.0%\n",
      "Working on book: War and Peace by graf Leo Tolstoy \n",
      "Finished downloading 17.0%\n",
      "Working on book: The Strange Case of Dr\n",
      "Finished downloading 18.0%\n",
      "Working on book: Forged Egyptian Antiquities by T\n",
      "Finished downloading 19.0%\n",
      "Working on book: New Amazonia by Elizabeth Burgoyne Corbett \n",
      "Finished downloading 20.0%\n",
      "Working on book: The Picture of Dorian Gray by Oscar Wilde \n",
      "**ERROR: The Picture of Dorian Gray by Oscar Wilde  Not Found**\n",
      "Working on book: Ulysses by James Joyce \n",
      "Finished downloading 22.0%\n",
      "Working on book: Il Principe\n",
      "**ERROR: Il Principe Not Found**\n",
      "Working on book: Et dukkehjem\n",
      "**ERROR: Et dukkehjem Not Found**\n",
      "Working on book: The Yellow Wallpaper by Charlotte Perkins Gilman \n",
      "Finished downloading 25.0%\n",
      "Working on book: The Brothers Karamazov by Fyodor Dostoyevsky \n",
      "Finished downloading 26.0%\n",
      "Working on book: Great Expectations by Charles Dickens \n",
      "Finished downloading 27.0%\n",
      "Working on book: First love and other stories by Ivan Sergeevich Turgenev \n",
      "Finished downloading 28.0%\n",
      "Working on book: The Secret Doctrine \n",
      "Finished downloading 29.0%\n",
      "Working on book: Emma by Jane Austen \n",
      "Finished downloading 30.0%\n",
      "Working on book: Captain Billy\n",
      "Finished downloading 31.0%\n",
      "Working on book: Tractatus Logico\n",
      "**ERROR: Tractatus Logico Not Found**\n",
      "Working on book: Essays of Michel de Montaigne \n",
      "Finished downloading 33.0%\n",
      "Working on book: The Count of Monte Cristo\n",
      "Finished downloading 34.0%\n",
      "Working on book: Jane Eyre\n",
      "**ERROR: Jane Eyre Not Found**\n",
      "Working on book: Sense and Sensibility by Jane Austen \n",
      "**ERROR: Sense and Sensibility by Jane Austen  Not Found**\n",
      "Working on book: Tales from Gorky by Maxime Gorky \n",
      "**ERROR: Tales from Gorky by Maxime Gorky  Not Found**\n",
      "Working on book: Minerva\n",
      "Finished downloading 38.0%\n",
      "Working on book: Treasure Island by Robert Louis Stevenson \n",
      "Finished downloading 39.0%\n",
      "Working on book: The Tale of Peter Rabbit by Beatrix Potter \n",
      "**ERROR: The Tale of Peter Rabbit by Beatrix Potter  Not Found**\n",
      "Working on book: My Secret Life\n",
      "**ERROR: My Secret Life Not Found**\n",
      "Working on book: Calculus Made Easy by Silvanus P\n",
      "**ERROR: Calculus Made Easy by Silvanus P Not Found**\n",
      "Working on book: Beyond Good and Evil by Friedrich Wilhelm Nietzsche \n",
      "**ERROR: Beyond Good and Evil by Friedrich Wilhelm Nietzsche  Not Found**\n",
      "Working on book: The Mysterious Affair at Styles by Agatha Christie \n",
      "Finished downloading 44.0%\n",
      "Working on book: The Kama Sutra of Vatsyayana by Vatsyayana \n",
      "**ERROR: The Kama Sutra of Vatsyayana by Vatsyayana  Not Found**\n",
      "Working on book: Psychology and parenthood by H\n",
      "Finished downloading 46.0%\n",
      "Working on book: Peter Pan by J\n",
      "Finished downloading 47.0%\n",
      "Working on book: The King James Version of the Bible \n",
      "**ERROR: The King James Version of the Bible  Not Found**\n",
      "Working on book: The Republic by Plato \n",
      "**ERROR: The Republic by Plato  Not Found**\n",
      "Working on book: The Federalist Papers by Alexander Hamilton and John Jay and James Madison \n",
      "**ERROR: The Federalist Papers by Alexander Hamilton and John Jay and James Madison  Not Found**\n",
      "Working on book: Siddhartha by Hermann Hesse \n",
      "**ERROR: Siddhartha by Hermann Hesse  Not Found**\n",
      "Working on book: Les Mis\n",
      "Finished downloading 52.0%\n",
      "Working on book: The History of Mary I\n",
      "Finished downloading 53.0%\n",
      "Working on book: The eleventh hour in the life of Julia Ward Howe by Maud Howe Elliott \n",
      "Finished downloading 54.0%\n",
      "Working on book: Leaves of Grass by Walt Whitman \n",
      "**ERROR: Leaves of Grass by Walt Whitman  Not Found**\n",
      "Working on book: Oliver Twist by Charles Dickens \n",
      "**ERROR: Oliver Twist by Charles Dickens  Not Found**\n",
      "Working on book: Leviathan by Thomas Hobbes \n",
      "**ERROR: Leviathan by Thomas Hobbes  Not Found**\n",
      "Working on book: Meditations by Emperor of Rome Marcus Aurelius \n",
      "**ERROR: Meditations by Emperor of Rome Marcus Aurelius  Not Found**\n",
      "Working on book: A Study in Scarlet by Arthur Conan Doyle \n",
      "Finished downloading 59.0%\n",
      "Working on book: Wuthering Heights by Emily Bront\n",
      "**ERROR: Wuthering Heights by Emily Bront Not Found**\n",
      "Working on book: Walden\n",
      "Finished downloading 61.0%\n",
      "Working on book: The Art of Money Getting\n",
      "**ERROR: The Art of Money Getting Not Found**\n",
      "Working on book: Dubliners by James Joyce \n",
      "Finished downloading 63.0%\n",
      "Working on book: The Hound of the Baskervilles by Arthur Conan Doyle \n",
      "Finished downloading 64.0%\n",
      "Working on book: Beowulf\n",
      "**ERROR: Beowulf Not Found**\n",
      "Working on book: The Souls of Black Folk by W\n",
      "**ERROR: The Souls of Black Folk by W Not Found**\n",
      "Working on book: Prestuplenie i nakazanie\n",
      "Finished downloading 67.0%\n",
      "Working on book: A Modest Proposal by Jonathan Swift \n",
      "**ERROR: A Modest Proposal by Jonathan Swift  Not Found**\n",
      "Working on book: The Drawings of Leonardo da Vinci by da Vinci Leonardo \n",
      "Finished downloading 69.0%\n",
      "Working on book: Forbidden Fruit by Anonymous \n",
      "**ERROR: Forbidden Fruit by Anonymous  Not Found**\n",
      "Working on book: Gulliver\n",
      "Finished downloading 71.0%\n",
      "Working on book: The Elements of Style by William Strunk \n",
      "**ERROR: The Elements of Style by William Strunk  Not Found**\n",
      "Working on book: Korean\n",
      "**ERROR: Korean Not Found**\n",
      "Working on book: On the origin of species by Charles Darwin \n",
      "**ERROR: On the origin of species by Charles Darwin  Not Found**\n",
      "Working on book: The Complete Works of William Shakespeare by William Shakespeare \n",
      "Finished downloading 75.0%\n",
      "Working on book: Notes from the Underground by Fyodor Dostoyevsky \n",
      "**ERROR: Notes from the Underground by Fyodor Dostoyevsky  Not Found**\n",
      "Working on book: Songs of Innocence\n",
      "Finished downloading 77.0%\n",
      "Working on book: The Tragedy of Romeo and Juliet by William Shakespeare \n",
      "**ERROR: The Tragedy of Romeo and Juliet by William Shakespeare  Not Found**\n",
      "Working on book: Also sprach Zarathustra\n",
      "Finished downloading 79.0%\n",
      "Working on book: Anna Karenina by graf Leo Tolstoy \n",
      "Finished downloading 80.0%\n",
      "Working on book: Autobiography of Benjamin Franklin by Benjamin Franklin \n",
      "**ERROR: Autobiography of Benjamin Franklin by Benjamin Franklin  Not Found**\n",
      "Working on book: Balaam and His Master by Joel Chandler Harris \n",
      "Finished downloading 82.0%\n",
      "Working on book: A Christmas Carol in Prose\n",
      "Finished downloading 83.0%\n",
      "Working on book: The Life and Adventures of Robinson Crusoe by Daniel Defoe \n",
      "Finished downloading 84.0%\n",
      "Working on book: Narrative of the Life of Frederick Douglass\n",
      "**ERROR: Narrative of the Life of Frederick Douglass Not Found**\n",
      "Working on book: Hard Times by Charles Dickens \n",
      "Finished downloading 86.0%\n",
      "Working on book: Early London by Walter Besant \n",
      "Finished downloading 87.0%\n",
      "Working on book: The Awakening\n",
      "Finished downloading 88.0%\n",
      "Working on book: Little Women by Louisa May Alcott \n",
      "**ERROR: Little Women by Louisa May Alcott  Not Found**\n",
      "Working on book: Memoirs Of Fanny Hill by John Cleland \n",
      "**ERROR: Memoirs Of Fanny Hill by John Cleland  Not Found**\n",
      "Working on book: The Confessions of St\n",
      "**ERROR: The Confessions of St Not Found**\n",
      "Working on book: The Boy Traders by Harry Castlemon \n",
      "Finished downloading 92.0%\n",
      "Working on book: Don Quijote by Miguel de Cervantes Saavedra \n",
      "**ERROR: Don Quijote by Miguel de Cervantes Saavedra  Not Found**\n",
      "Working on book: Anne of Green Gables by L\n",
      "Finished downloading 94.0%\n",
      "Working on book: The Divine Comedy by Dante\n",
      "**ERROR: The Divine Comedy by Dante Not Found**\n",
      "Working on book: The Time Machine by H\n",
      "Finished downloading 96.0%\n",
      "Working on book: The Works of Edgar Allan Poe \n",
      "Finished downloading 97.0%\n",
      "Working on book: The Wonderful Wizard of Oz by L\n",
      "**ERROR: The Wonderful Wizard of Oz by L Not Found**\n",
      "Working on book: Democracy in America \n",
      "**ERROR: Democracy in America  Not Found**\n",
      "Working on book: The King in Yellow by Robert W\n",
      "**ERROR: The King in Yellow by Robert W Not Found**\n",
      "----------------------------------------\n",
      "Finished downloading all books!\n",
      "----------------------------------------\n",
      "Please enter a folder location to save the Ebooks in:\n",
      "55 book(s) was/were successfully downloaded and saved to the location C:\\Users\\Tirtha\\Documents\\Personal\\Data Science related\\Python and other Notebooks\\WebDataAnalytics\\Ebooks\\\n",
      "45 books were not found on the server!\n"
     ]
    }
   ],
   "source": [
    "save_text_files(100,verbosity=1)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.2"
  },
  "latex_envs": {
   "LaTeX_envs_menu_present": true,
   "autoclose": false,
   "autocomplete": true,
   "bibliofile": "biblio.bib",
   "cite_by": "apalike",
   "current_citInitial": 1,
   "eqLabelWithNumbers": true,
   "eqNumInitial": 1,
   "hotkeys": {
    "equation": "Ctrl-E",
    "itemize": "Ctrl-I"
   },
   "labels_anchors": false,
   "latex_user_defs": false,
   "report_style_numbering": false,
   "user_envs_cfg": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}