TwitterDemo.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Working with Twitter Data\n",
    "This notebook briefly outlines how to:\n",
    "* Get an API key for Twitter\n",
    "* Read in information about a user\n",
    "* Search for a keyword\n",
    "\n",
    "You will need to install Pandas and Tweepy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# We might do some data manipulation\n",
    "import pandas as pd\n",
    "\n",
    "# import Tweepy, a helper library for connecting to the Twitter API\n",
    "import tweepy"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## API Keys\n",
    "To use Twitter we need to have API Keys ready, you will not be able to run this notebook without setting up a developer account with Twitter. Apply for access [here](https://developer.twitter.com/en/apply-for-access), be warned this process may take around a week.\n",
    "\n",
    "Tweepy has an auth tutorial [here](https://docs.tweepy.org/en/v3.10.0/auth_tutorial.html#auth-tutorial)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "consumer_key = 'REPLACEME'\n",
    "consumer_secret = 'REPLACEME'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Authenticate our requests\n",
    "auth = tweepy.AppAuthHandler(consumer_key, consumer_secret)\n",
    "\n",
    "# create our API object\n",
    "api = tweepy.API(auth)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we are authenticated we can start to make requests.\n",
    "For example we can get the user @Twitter, and list it's name, follower count and print out all friends."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "name:  Twitter\n",
      "followers:  59387062\n",
      "dinoman_j\n",
      "MrTeddyTedster\n",
      "austinpetsalive\n",
      "djarinculture\n",
      "ThisIsNoahEvans\n",
      "babyangleszaa\n",
      "PAVGOD\n",
      "BTS_twt\n",
      "OBR1ENS\n",
      "Clarissa_Laskey\n"
     ]
    }
   ],
   "source": [
    "# Get the User object for twitter...\n",
    "user = api.get_user('twitter')\n",
    "\n",
    "# Here we can access most public features of a Twitter profile, and some subtle ones.\n",
    "print('name: ', user.screen_name)\n",
    "print('followers: ',user.followers_count)\n",
    "for friend in user.friends():\n",
    "   print(friend.screen_name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we want to paginate through pages of content, we would have to write something to handle this logic. Tweepy handles this for us with the Cursor class."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "@RepAOC @Bill_Wilson_1 Tibetans yearn for liberation by a socialism-touting, tequila-peddling bar girl? Same strate… https://t.co/28KBmuy93j\n",
      "I've wanted to switch to 100% vegan for a while now but as money has been tight during the pandemic as my partner's… https://t.co/WXTQL6aPLy\n",
      "RT @Yield_Field_1: 🌿 @veganswap is an eco-friendly yield farming and #DeFi aggregator on #BinanceSmartChain.\n",
      "\n",
      "🌿Users can stake $VEGAN to ea…\n",
      "The language on Facebook makes me shiver: \n",
      "\"Reminder! Carl invited you to like The Vegan Gallery\"\n",
      "\n",
      "How can you invi… https://t.co/b3sUs3QuVd\n",
      "good morning to lefty vegan non binary tramps only\n"
     ]
    }
   ],
   "source": [
    "# To paginate through multiple pages of results, Tweepy has the Cursor object\n",
    "for tweet in tweepy.Cursor(api.search, q='Vegan').items(5):\n",
    "    print(tweet.text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally we should try to store these in some reliable storage. For now I will use a dataframe but you might want to move to something cloud-based here."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "@najoumi35 @Edelweiss_999 Vegan c'est simplement que c'est végétal/minéral. Pas bio\n",
      "@AntonioDesiena1 @Antonio79B @EdizioniLindau Non lo dice un sociologo. Lo dice l'evoluzione dell'essere umano. Al n… https://t.co/NL6xrVaLpt\n",
      "🍰 🧁 THE GREAT NUSH BAKE OFF Nush 🧁 🍰 Join Nush and @livias for a live bake-a-long of dreams at 2pm on Friday 19th M… https://t.co/dQi6Hrju6N\n",
      "RT @Ivy_MiddletonUK: All animals are the same, they all feel just like us.\n",
      "\n",
      "Affection, fear, cold, hot, hungry, happy, upset...and DESIRE T…\n",
      "Chickpea mayonnaise!\n",
      "\n",
      "\"By adding just a dozen chickpeas to three tablespoons of liquid and blending it together wit… https://t.co/u61MNSK71I\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Tweet</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>@najoumi35 @Edelweiss_999 Vegan c'est simpleme...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>@AntonioDesiena1 @Antonio79B @EdizioniLindau N...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>🍰 🧁 THE GREAT NUSH BAKE OFF Nush 🧁 🍰 Join Nush...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>RT @Ivy_MiddletonUK: All animals are the same,...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Chickpea mayonnaise!\\n\\n\"By adding just a doze...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                               Tweet\n",
       "0  @najoumi35 @Edelweiss_999 Vegan c'est simpleme...\n",
       "1  @AntonioDesiena1 @Antonio79B @EdizioniLindau N...\n",
       "2  🍰 🧁 THE GREAT NUSH BAKE OFF Nush 🧁 🍰 Join Nush...\n",
       "3  RT @Ivy_MiddletonUK: All animals are the same,...\n",
       "4  Chickpea mayonnaise!\\n\\n\"By adding just a doze..."
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# We should store these in a dataframe so we can analyse them\n",
    "tweets = []\n",
    "for tweet in tweepy.Cursor(api.search, q='Vegan').items(5):\n",
    "    print(tweet.text)\n",
    "    tweets.append(tweet.text)\n",
    "\n",
    "df = pd.DataFrame(tweets, columns=['Tweet'])\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Next Steps\n",
    "I suggest exploring the above [user](https://docs.tweepy.org/en/latest/api.html?highlight=user#user-methods) and [tweet](https://developer.twitter.com/en/docs/twitter-api/v1/data-dictionary/object-model/tweet) objects. There is a lot of information in there that could be useful."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}