03_logistic_regression.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Working with Images & Logistic Regression in PyTorch"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Working with Images\n",
    "\n",
    "In this notebook, I'll use my existing knowledge of PyTorch and linear regression to solve a different kind of problem: *image classification*.\n",
    "I'll use the famous MNIST Handwritten Digits Databases as our training dataset. It consists of 28px by 28px grayscale images of handwritten digits (0 to 9) and labels for each image indicating which digit it represents. Here are some sample images from the dataset:\n",
    "\n",
    "![mnist-sample](https://i.imgur.com/CAYnuo1.jpg)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "import torchvision\n",
    "from torchvision.datasets import MNIST\n",
    "\n",
    "# torchvision contains some utilities for working with image data. \n",
    "# It also provides helper classes to download and import popular datasets like MNIST automatically."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Download training dataset\n",
    "dataset = MNIST(root = 'data/', download = True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When this statement is executed for the first time, it downloads the data to the `data/` directory next to the notebook and creates a PyTorch `Dataset`. On subsequent executions, the download is skipped as the data is already downloaded. Let's check the size of the dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "60000"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(dataset)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The dataset has 60,000 images that we'll use to train the model. There is also an additional test set of 10,000 images used for evaluating models and reporting metrics in papers and reports. We can create the test dataset using the `MNIST` class by passing `train=False` to the constructor."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "10000"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_dataset = MNIST(root = 'data/', train = False)\n",
    "len(test_dataset)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(<PIL.Image.Image image mode=L size=28x28 at 0x7F17320F48E0>, 5)"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataset[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It's a pair, consisting of a 28x28px image and a label. The image is an object of the class `PIL.Image.Image`, which is a part of the Python imaging library `Pillow`. We can view the image within Jupyter using `matplotlib`, the de-facto plotting and graphing library for data science in Python."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The statement %matplotlib inline indicates to Jupyter that we want to plot the graphs within the notebook. Without this line, Jupyter will show the image in a popup. Statements starting with % are called magic commands and are used to configure the behavior of Jupyter itself. You can find a full list of magic commands here: https://ipython.readthedocs.io/en/stable/interactive/magics.html .\n",
    "\n",
    "Let's look at a couple of images from the dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Label: 5\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAN80lEQVR4nO3df6hcdXrH8c+ncf3DrBpTMYasNhuRWBWbLRqLSl2RrD9QNOqWDVgsBrN/GHChhEr6xyolEuqP0qAsuYu6sWyzLqgYZVkVo6ZFCF5j1JjU1YrdjV6SSozG+KtJnv5xT+Su3vnOzcyZOZP7vF9wmZnzzJnzcLife87Md879OiIEYPL7k6YbANAfhB1IgrADSRB2IAnCDiRxRD83ZpuP/oEeiwiPt7yrI7vtS22/aftt27d281oAesudjrPbniLpd5IWSNou6SVJiyJia2EdjuxAj/XiyD5f0tsR8U5EfCnpV5Ku6uL1APRQN2GfJekPYx5vr5b9EdtLbA/bHu5iWwC61M0HdOOdKnzjND0ihiQNSZzGA03q5si+XdJJYx5/R9L73bUDoFe6CftLkk61/V3bR0r6kaR19bQFoG4dn8ZHxD7bSyU9JWmKpAci4o3aOgNQq46H3jraGO/ZgZ7ryZdqABw+CDuQBGEHkiDsQBKEHUiCsANJEHYgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kARhB5Ig7EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUii4ymbcXiYMmVKsX7sscf2dPtLly5tWTvqqKOK686dO7dYv/nmm4v1u+66q2Vt0aJFxXU///zzYn3lypXF+u23316sN6GrsNt+V9IeSfsl7YuIs+toCkD96jiyXxQRH9TwOgB6iPfsQBLdhj0kPW37ZdtLxnuC7SW2h20Pd7ktAF3o9jT+/Ih43/YJkp6x/V8RsWHsEyJiSNKQJNmOLrcHoENdHdkj4v3qdqekxyTNr6MpAPXrOOy2p9o++uB9ST+QtKWuxgDUq5vT+BmSHrN98HX+PSJ+W0tXk8zJJ59crB955JHF+nnnnVesX3DBBS1r06ZNK6577bXXFutN2r59e7G+atWqYn3hwoUta3v27Cmu++qrrxbrL7zwQrE+iDoOe0S8I+kvauwFQA8x9AYkQdiBJAg7kARhB5Ig7EASjujfl9om6zfo5s2bV6yvX7++WO/1ZaaD6sCBA8X6jTfeWKx/8sknHW97ZGSkWP/www+L9TfffLPjbfdaRHi85RzZgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJxtlrMH369GJ948aNxfqcOXPqbKdW7XrfvXt3sX7RRRe1rH355ZfFdbN+/6BbjLMDyRF2IAnCDiRB2IEkCDuQBGEHkiDsQBJM2VyDXbt2FevLli0r1q+44opi/ZVXXinW2/1L5ZLNmzcX6wsWLCjW9+7dW6yfccYZLWu33HJLcV3UiyM7kARhB5Ig7EAShB1IgrADSRB2IAnCDiTB9ewD4JhjjinW200vvHr16pa1xYsXF9e9/vrri/W1a9cW6xg8HV/PbvsB2zttbxmzbLrtZ2y/Vd0eV2ezAOo3kdP4X0i69GvLbpX0bEScKunZ6jGAAdY27BGxQdLXvw96laQ11f01kq6uuS8ANev0u/EzImJEkiJixPYJrZ5oe4mkJR1uB0BNen4hTEQMSRqS+IAOaFKnQ287bM+UpOp2Z30tAeiFTsO+TtIN1f0bJD1eTzsAeqXtabzttZK+L+l429sl/VTSSkm/tr1Y0u8l/bCXTU52H3/8cVfrf/TRRx2ve9NNNxXrDz/8cLHebo51DI62YY+IRS1KF9fcC4Ae4uuyQBKEHUiCsANJEHYgCcIOJMElrpPA1KlTW9aeeOKJ4roXXnhhsX7ZZZcV608//XSxjv5jymYgOcIOJEHYgSQIO5AEYQeSIOxAEoQdSIJx9knulFNOKdY3bdpUrO/evbtYf+6554r14eHhlrX77ruvuG4/fzcnE8bZgeQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJxtmTW7hwYbH+4IMPFutHH310x9tevnx5sf7QQw8V6yMjIx1vezJjnB1IjrADSRB2IAnCDiRB2IEkCDuQBGEHkmCcHUVnnnlmsX7PPfcU6xdf3Plkv6tXry7WV6xYUay/9957HW/7cNbxOLvtB2zvtL1lzLLbbL9ne3P1c3mdzQKo30RO438h6dJxlv9LRMyrfn5Tb1sA6tY27BGxQdKuPvQCoIe6+YBuqe3XqtP841o9yfYS28O2W/8zMgA912nYfybpFEnzJI1IurvVEyNiKCLOjoizO9wWgBp0FPaI2BER+yPigKSfS5pfb1sA6tZR2G3PHPNwoaQtrZ4LYDC0HWe3vVbS9yUdL2mHpJ9Wj+dJCknvSvpxRLS9uJhx9sln2rRpxfqVV17ZstbuWnl73OHir6xfv75YX7BgQbE+WbUaZz9iAisuGmfx/V13BKCv+LoskARhB5Ig7EAShB1IgrADSXCJKxrzxRdfFOtHHFEeLNq3b1+xfskll7SsPf/888V1D2f8K2kgOcIOJEHYgSQIO5AEYQeSIOxAEoQdSKLtVW/I7ayzzirWr7vuumL9nHPOaVlrN47eztatW4v1DRs2dPX6kw1HdiAJwg4kQdiBJAg7kARhB5Ig7EAShB1IgnH2SW7u3LnF+tKlS4v1a665plg/8cQTD7mnidq/f3+xPjJS/u/lBw4cqLOdwx5HdiAJwg4kQdiBJAg7kARhB5Ig7EAShB1IgnH2w0C7sexFi8abaHdUu3H02bNnd9JSLYaHh4v1FStWFOvr1q2rs51Jr+2R3fZJtp+zvc32G7ZvqZZPt/2M7beq2+N63y6ATk3kNH6fpL+PiD+X9FeSbrZ9uqRbJT0bEadKerZ6DGBAtQ17RIxExKbq/h5J2yTNknSVpDXV09ZIurpXTQLo3iG9Z7c9W9L3JG2UNCMiRqTRPwi2T2ixzhJJS7prE0C3Jhx229+W9Iikn0TEx/a4c8d9Q0QMSRqqXoOJHYGGTGjozfa3NBr0X0bEo9XiHbZnVvWZknb2pkUAdWh7ZPfoIfx+Sdsi4p4xpXWSbpC0srp9vCcdTgIzZswo1k8//fRi/d577y3WTzvttEPuqS4bN24s1u+8886WtccfL//KcIlqvSZyGn++pL+V9LrtzdWy5RoN+a9tL5b0e0k/7E2LAOrQNuwR8Z+SWr1Bv7jedgD0Cl+XBZIg7EAShB1IgrADSRB2IAkucZ2g6dOnt6ytXr26uO68efOK9Tlz5nTUUx1efPHFYv3uu+8u1p966qli/bPPPjvkntAbHNmBJAg7kARhB5Ig7EAShB1IgrADSRB2IIk04+znnntusb5s2bJiff78+S1rs2bN6qinunz66acta6tWrSque8cddxTre/fu7agnDB6O7EAShB1IgrADSRB2IAnCDiRB2IEkCDuQRJpx9oULF3ZV78bWrVuL9SeffLJY37dvX7FeuuZ89+7dxXWRB0d2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUjCEVF+gn2SpIcknSjpgKShiPhX27dJuknS/1ZPXR4Rv2nzWuWNAehaRIw76/JEwj5T0syI2GT7aEkvS7pa0t9I+iQi7ppoE4Qd6L1WYZ/I/Owjkkaq+3tsb5PU7L9mAXDIDuk9u+3Zkr4naWO1aKnt12w/YPu4FusssT1se7irTgF0pe1p/FdPtL8t6QVJKyLiUdszJH0gKST9k0ZP9W9s8xqcxgM91vF7dkmy/S1JT0p6KiLuGac+W9KTEXFmm9ch7ECPtQp729N425Z0v6RtY4NefXB30EJJW7ptEkDvTOTT+Ask/Yek1zU69CZJyyUtkjRPo6fx70r6cfVhXum1OLIDPdbVaXxdCDvQex2fxgOYHAg7kARhB5Ig7EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUiCsANJ9HvK5g8k/c+Yx8dXywbRoPY2qH1J9NapOnv7s1aFvl7P/o2N28MRcXZjDRQMam+D2pdEb53qV2+cxgNJEHYgiabDPtTw9ksGtbdB7Uuit071pbdG37MD6J+mj+wA+oSwA0k0Enbbl9p+0/bbtm9toodWbL9r+3Xbm5uen66aQ2+n7S1jlk23/Yztt6rbcefYa6i322y/V+27zbYvb6i3k2w/Z3ub7Tds31Itb3TfFfrqy37r+3t221Mk/U7SAknbJb0kaVFEbO1rIy3YflfS2RHR+BcwbP+1pE8kPXRwai3b/yxpV0SsrP5QHhcR/zAgvd2mQ5zGu0e9tZpm/O/U4L6rc/rzTjRxZJ8v6e2IeCcivpT0K0lXNdDHwIuIDZJ2fW3xVZLWVPfXaPSXpe9a9DYQImIkIjZV9/dIOjjNeKP7rtBXXzQR9lmS/jDm8XYN1nzvIelp2y/bXtJ0M+OYcXCarer2hIb7+bq203j309emGR+YfdfJ9OfdaiLs401NM0jjf+dHxF9KukzSzdXpKibmZ5JO0egcgCOS7m6ymWqa8Uck/SQiPm6yl7HG6asv+62JsG+XdNKYx9+R9H4DfYwrIt6vbndKekyjbzsGyY6DM+hWtzsb7ucrEbEjIvZHxAFJP1eD+66aZvwRSb+MiEerxY3vu/H66td+ayLsL0k61fZ3bR8p6UeS1jXQxzfYnlp9cCLbUyX9QIM3FfU6STdU92+Q9HiDvfyRQZnGu9U042p43zU+/XlE9P1H0uUa/UT+vyX9YxM9tOhrjqRXq583mu5N0lqNntb9n0bPiBZL+lNJz0p6q7qdPkC9/ZtGp/Z+TaPBmtlQbxdo9K3ha5I2Vz+XN73vCn31Zb/xdVkgCb5BByRB2IEkCDuQBGEHkiDsQBKEHUiCsANJ/D+f1mbtgJ8kQQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "image, label = dataset[0]\n",
    "plt.imshow(image, cmap = 'gray')\n",
    "print('Label:', label)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Label: 3\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAANb0lEQVR4nO3df6gd9ZnH8c9ntVE0kSRK9GL91aioKCZrFMW6uJaUrCixYNcGWVxWuPmjShUhGyoYYVPQXeNKEAsparNLN6UQQ6WsNBLCuv5TEjWrMbFNNsT0JiHBDVrrP9H47B93Itfknjk3Z2bOnHuf9wsu55x5zsw8HPLJzDnz4+uIEICp7y/abgBAfxB2IAnCDiRB2IEkCDuQxOn9XJltfvoHGhYRHm96pS277UW2f297t+3lVZYFoFnu9Ti77dMk/UHSQkkjkrZIWhIRO0rmYcsONKyJLftNknZHxJ6IOCrpl5IWV1gegAZVCfuFkv445vVIMe1rbA/b3mp7a4V1Aaioyg904+0qnLSbHhFrJK2R2I0H2lRlyz4i6aIxr78p6UC1dgA0pUrYt0i6wvZltqdJ+oGkV+tpC0Ddet6Nj4gvbD8k6beSTpP0UkS8X1tnAGrV86G3nlbGd3agcY2cVANg8iDsQBKEHUiCsANJEHYgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kARhB5Ig7EAShB1IgrADSRB2IIm+DtmMZlxzzTUda3fddVfpvMPDw6X1LVu2lNbfeeed0nqZ5557rrR+9OjRnpeNk7FlB5Ig7EAShB1IgrADSRB2IAnCDiRB2IEkGMV1Eli6dGlp/ZlnnulYmz59et3t1OaOO+4orW/evLlPnUwtnUZxrXRSje29kj6VdEzSFxGxoMryADSnjjPo/joiPqphOQAaxHd2IImqYQ9JG22/ZXvck6xtD9veantrxXUBqKDqbvytEXHA9hxJr9v+ICLeGPuGiFgjaY3ED3RAmypt2SPiQPF4WNIGSTfV0RSA+vUcdttn255x/Lmk70raXldjAOrV83F229/S6NZcGv068B8R8ZMu87Ab34PZs2eX1nfu3NmxNmfOnLrbqc3HH39cWr/vvvtK6xs3bqyznSmj9uPsEbFH0vU9dwSgrzj0BiRB2IEkCDuQBGEHkiDsQBLcSnoSOHLkSGl9xYoVHWurVq0qnfess84qre/bt6+0fvHFF5fWy8ycObO0vmjRotI6h95ODVt2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUiCW0lPcdu2bSutX399+YWL27eX36Lg2muvPeWeJmru3Lml9T179jS27sms0yWubNmBJAg7kARhB5Ig7EAShB1IgrADSRB2IAmuZ5/iVq5cWVp//PHHS+vz5s2rs51TMm3atNbWPRWxZQeSIOxAEoQdSIKwA0kQdiAJwg4kQdiBJLiePbkLLrigtN7t3uzXXXddne18zfr160vr9957b2Prnsx6vp7d9ku2D9vePmbabNuv295VPM6qs1kA9ZvIbvzPJZ04NMdySZsi4gpJm4rXAAZY17BHxBuSThx/aLGktcXztZLuqbkvADXr9dz48yPioCRFxEHbczq90fawpOEe1wOgJo1fCBMRayStkfiBDmhTr4feDtkekqTi8XB9LQFoQq9hf1XSA8XzByT9up52ADSl62687XWSbpd0nu0RSSskPSXpV7YflLRP0vebbBK9u//++0vr3e4b3+R94bt58803W1v3VNQ17BGxpEPpOzX3AqBBnC4LJEHYgSQIO5AEYQeSIOxAElziOglcddVVpfUNGzZ0rF1++eWl855++uDeTZwhm3vDkM1AcoQdSIKwA0kQdiAJwg4kQdiBJAg7kMTgHmTFV66++urS+mWXXdaxNsjH0bt59NFHS+sPP/xwnzqZGtiyA0kQdiAJwg4kQdiBJAg7kARhB5Ig7EASk/cgbCJl16tL0rJlyzrWnn766dJ5zzzzzJ566oehoaG2W5hS2LIDSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBIcZ58CVq9e3bG2a9eu0nlnzpxZad3drpd//vnnO9bOOeecSuvGqem6Zbf9ku3DtrePmfak7f22txV/dzbbJoCqJrIb/3NJi8aZ/q8RMa/4+8962wJQt65hj4g3JB3pQy8AGlTlB7qHbL9b7ObP6vQm28O2t9reWmFdACrqNew/lTRX0jxJByWt6vTGiFgTEQsiYkGP6wJQg57CHhGHIuJYRHwp6WeSbqq3LQB16ynstsdee/g9Sds7vRfAYOh6nN32Okm3SzrP9oikFZJutz1PUkjaK2lpgz2igtdee63R5dvjDgX+lbLx4Z944onSeefNm1dav+SSS0rrH374YWk9m65hj4gl40x+sYFeADSI02WBJAg7kARhB5Ig7EAShB1IgktcUcm0adNK690Or5X5/PPPS+vHjh3redkZsWUHkiDsQBKEHUiCsANJEHYgCcIOJEHYgSQ4zo5KVq5c2diyX3yx/OLKkZGRxtY9FbFlB5Ig7EAShB1IgrADSRB2IAnCDiRB2IEkHBH9W5ndv5XV7Nxzz+1Ye/nll0vnXbduXaV6m4aGhkrrH3zwQWm9yrDMc+fOLa3v2bOn52VPZREx7v292bIDSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBJczz5Bq1ev7li7++67S+e98sorS+sHDhwore/fv7+0vnv37o61G264oXTebr0tW7astF7lOPqqVatK690+F5yarlt22xfZ3mx7p+33bf+omD7b9uu2dxWPs5pvF0CvJrIb/4WkxyLiakk3S/qh7WskLZe0KSKukLSpeA1gQHUNe0QcjIi3i+efStop6UJJiyWtLd62VtI9TTUJoLpT+s5u+1JJ8yX9TtL5EXFQGv0PwfacDvMMSxqu1iaAqiYcdtvTJa2X9EhE/Mke91z7k0TEGklrimVM2gthgMluQofebH9Do0H/RUS8Ukw+ZHuoqA9JOtxMiwDq0PUSV49uwtdKOhIRj4yZ/i+S/i8inrK9XNLsiCg9TjOZt+w333xzx9qzzz5bOu8tt9xSad179+4tre/YsaNj7bbbbiudd8aMGb209JVu/37KLoG98cYbS+f97LPPeuopu06XuE5kN/5WSX8n6T3b24ppP5b0lKRf2X5Q0j5J36+jUQDN6Br2iHhTUqcv6N+ptx0ATeF0WSAJwg4kQdiBJAg7kARhB5LgVtI16HapZtklqJL0wgsv1NlOXx05cqS0XnYLbjSDW0kDyRF2IAnCDiRB2IEkCDuQBGEHkiDsQBLcSroGjz32WGn9jDPOKK1Pnz690vrnz5/fsbZkyZJKy/7kk09K6wsXLqy0fPQPW3YgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSILr2YEphuvZgeQIO5AEYQeSIOxAEoQdSIKwA0kQdiCJrmG3fZHtzbZ32n7f9o+K6U/a3m97W/F3Z/PtAuhV15NqbA9JGoqIt23PkPSWpHsk/a2kP0fEMxNeGSfVAI3rdFLNRMZnPyjpYPH8U9s7JV1Yb3sAmnZK39ltXyppvqTfFZMesv2u7Zdsz+owz7Dtrba3VuoUQCUTPjfe9nRJ/yXpJxHxiu3zJX0kKST9k0Z39f+hyzLYjQca1mk3fkJht/0NSb+R9NuIeHac+qWSfhMR13ZZDmEHGtbzhTC2LelFSTvHBr344e6470naXrVJAM2ZyK/x35b035Lek/RlMfnHkpZImqfR3fi9kpYWP+aVLYstO9CwSrvxdSHsQPO4nh1IjrADSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUiCsANJEHYgCcIOJEHYgSQIO5BE1xtO1uwjSR+OeX1eMW0QDWpvg9qXRG+9qrO3SzoV+no9+0krt7dGxILWGigxqL0Nal8SvfWqX72xGw8kQdiBJNoO+5qW119mUHsb1L4keutVX3pr9Ts7gP5pe8sOoE8IO5BEK2G3vcj2723vtr28jR46sb3X9nvFMNStjk9XjKF32Pb2MdNm237d9q7icdwx9lrqbSCG8S4ZZrzVz67t4c/7/p3d9mmS/iBpoaQRSVskLYmIHX1tpAPbeyUtiIjWT8Cw/VeS/izp344PrWX7nyUdiYiniv8oZ0XEPw5Ib0/qFIfxbqi3TsOM/71a/OzqHP68F21s2W+StDsi9kTEUUm/lLS4hT4GXkS8IenICZMXS1pbPF+r0X8sfdeht4EQEQcj4u3i+aeSjg8z3upnV9JXX7QR9gsl/XHM6xEN1njvIWmj7bdsD7fdzDjOPz7MVvE4p+V+TtR1GO9+OmGY8YH57HoZ/ryqNsI+3tA0g3T879aI+EtJfyPph8XuKibmp5LmanQMwIOSVrXZTDHM+HpJj0TEn9rsZaxx+urL59ZG2EckXTTm9TclHWihj3FFxIHi8bCkDRr92jFIDh0fQbd4PNxyP1+JiEMRcSwivpT0M7X42RXDjK+X9IuIeKWY3PpnN15f/frc2gj7FklX2L7M9jRJP5D0agt9nMT22cUPJ7J9tqTvavCGon5V0gPF8wck/brFXr5mUIbx7jTMuFr+7Fof/jwi+v4n6U6N/iL/v5Ieb6OHDn19S9L/FH/vt92bpHUa3a37XKN7RA9KOlfSJkm7isfZA9Tbv2t0aO93NRqsoZZ6+7ZGvxq+K2lb8Xdn259dSV99+dw4XRZIgjPogCQIO5AEYQeSIOxAEoQdSIKwA0kQdiCJ/wehviHnQhygtgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "image, label = dataset[10]\n",
    "plt.imshow(image, cmap = 'gray')\n",
    "print('Label:', label)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It's evident that these images are relatively small in size, and recognizing the digits can sometimes be challenging even for the human eye. While it's useful to look at these images, there's just one problem here: PyTorch doesn't know how to work with images. We need to convert the images into tensors. We can do this by specifying a transform while creating our dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torchvision.transforms as transforms"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "PyTorch datasets allow us to specify one or more transformation functions that are applied to the images as they are loaded. The `torchvision.transforms` module contains many such predefined functions. We'll use the `ToTensor` transform to convert images into PyTorch tensors."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "# MNIST dataset (images and labels)\n",
    "\n",
    "dataset = MNIST(root = 'data/', train = True, transform = transforms.ToTensor())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([1, 28, 28]) 5\n"
     ]
    }
   ],
   "source": [
    "img_tensor, label = dataset[0]\n",
    "print(img_tensor.shape, label)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The image is now converted to a 1x28x28 tensor. The first dimension tracks color channels. The second and third dimensions represent pixels along the height and width of the image, respectively. Since images in the MNIST dataset are grayscale, there's just one channel. Other datasets have images with color, in which case there are three channels: red, green, and blue (RGB)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's look at some sample values inside the tensor."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[0.0039, 0.6039, 0.9922, 0.3529, 0.0000],\n",
      "        [0.0000, 0.5451, 0.9922, 0.7451, 0.0078],\n",
      "        [0.0000, 0.0431, 0.7451, 0.9922, 0.2745],\n",
      "        [0.0000, 0.0000, 0.1373, 0.9451, 0.8824],\n",
      "        [0.0000, 0.0000, 0.0000, 0.3176, 0.9412]])\n",
      "tensor(1.) tensor(0.)\n"
     ]
    }
   ],
   "source": [
    "print(img_tensor[0, 10:15, 10:15])\n",
    "print(torch.max(img_tensor), torch.min(img_tensor))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The values range from 0 to 1, with `0` representing black, `1` representing white, and the value in between different shades of gray. We can also plot the tensor as an image using `plt.imshow`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPUAAAD4CAYAAAA0L6C7AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAJRElEQVR4nO3dz2ucBR7H8c9n04qiCx7qQZrSiohsEVahFKEHoQjWKnpVqF7UXFaoIIge/QfEi5egYsFSEfQg6iIFFRGsGjUWu1GoPxaLQncprXpRaj97mGHpuknzzHSeeeb58n5BIJMZMh9K3n1mJuEZJxGAOv7U9QAAk0XUQDFEDRRD1EAxRA0Us6GNb2q7Ny+pb926tesJI9m0aVPXE0by7bffdj2hsVOnTnU9YSRJvNrX3cavtGzHXvX+Zs7i4mLXE0by4IMPdj1hJPv27et6QmMHDx7sesJI1oqah99AMUQNFEPUQDFEDRRD1EAxRA0UQ9RAMUQNFEPUQDFEDRRD1EAxRA0UQ9RAMUQNFEPUQDFEDRRD1EAxjaK2vcf2V7aP23687VEAxrdu1LbnJD0j6XZJ2yXda3t728MAjKfJkXqnpONJvknym6SXJN3d7iwA42oS9WZJ3593+cTwa//D9oLtJdtLkxoHYHRNThG82hkL/+8UpEkWJS1K/TpFMFBNkyP1CUlbzrs8L+mHduYAuFhNov5Y0nW2r7F9iaR7JL3W7iwA41r34XeSs7YflvSWpDlJzyc51voyAGNp9LY7Sd6U9GbLWwBMAH9RBhRD1EAxRA0UQ9RAMUQNFEPUQDFEDRRD1EAxRA0UQ9RAMUQNFEPUQDFEDRRD1EAxRA0UQ9RAMY1OkjCOpB/nHjxz5kzXE0p76KGHup7Q2KFDh7qe0Ni5c+fWvI4jNVAMUQPFEDVQDFEDxRA1UAxRA8UQNVAMUQPFEDVQDFEDxRA1UAxRA8UQNVAMUQPFEDVQDFEDxRA1UMy6Udt+3vZJ219MYxCAi9PkSP2CpD0t7wAwIetGneQ9SaemsAXABPCcGihmYmcTtb0gaWFS3w/AeCYWdZJFSYuSZLsf5wcGCuLhN1BMk19pHZL0gaTrbZ+w/UD7swCMa92H30nuncYQAJPBw2+gGKIGiiFqoBiiBoohaqAYogaKIWqgGKIGiiFqoBiiBoohaqAYogaKIWqgGKIGiiFqoBiiBopxMvnTifXpHGWXX3551xNG8sYbb3Q9YSS33HJL1xMau+2227qe0NiRI0d05swZr3YdR2qgGKIGiiFqoBiiBoohaqAYogaKIWqgGKIGiiFqoBiiBoohaqAYogaKIWqgGKIGiiFqoBiiBoohaqAYogaKWTdq21tsv2N7xfYx2/unMQzAeDY0uM1ZSY8m+dT2nyV9Yvtwkn+0vA3AGNY9Uif5Mcmnw89/lrQiaXPbwwCMp8mR+r9sb5N0k6QPV7luQdLCRFYBGFvjqG1fIekVSY8k+emP1ydZlLQ4vG1vThEMVNPo1W/bGzUI+mCSV9udBOBiNHn125Kek7SS5Kn2JwG4GE2O1Lsk3Sdpt+3l4cfelncBGNO6z6mTvC9p1bf3ADB7+IsyoBiiBoohaqAYogaKIWqgGKIGiiFqoBiiBoohaqAYogaKIWqgGKIGiiFqoBiiBoohaqAYogaKcTL5cwRy4sH2XHvttV1PGMny8nLXExo7ffp01xMa27t3r44ePbrqyUs4UgPFEDVQDFEDxRA1UAxRA8UQNVAMUQPFEDVQDFEDxRA1UAxRA8UQNVAMUQPFEDVQDFEDxRA1UAxRA8WsG7XtS21/ZPtz28dsPzmNYQDGs6HBbX6VtDvJL7Y3Snrf9t+THGl5G4AxrBt1Bicx+2V4cePwg3OQATOq0XNq23O2lyWdlHQ4yYftzgIwrkZRJ/k9yY2S5iXttH3DH29je8H2ku2lSY8E0NxIr34nOS3pXUl7VrluMcmOJDsmtA3AGJq8+n2V7SuHn18m6VZJX7Y9DMB4mrz6fbWkA7bnNPhP4OUkr7c7C8C4mrz6fVTSTVPYAmAC+IsyoBiiBoohaqAYogaKIWqgGKIGiiFqoBiiBoohaqAYogaKIWqgGKIGiiFqoBiiBoohaqAYogaKaXLmE8yQr7/+uusJI7n//vu7ntDYgQMHup7Q2IYNa6fLkRoohqiBYogaKIaogWKIGiiGqIFiiBoohqiBYogaKIaogWKIGiiGqIFiiBoohqiBYogaKIaogWKIGiiGqIFiGkdte872Z7Zfb3MQgIszypF6v6SVtoYAmIxGUduel3SHpGfbnQPgYjU9Uj8t6TFJ59a6ge0F20u2lyayDMBY1o3a9p2STib55EK3S7KYZEeSHRNbB2BkTY7UuyTdZfs7SS9J2m37xVZXARjbulEneSLJfJJtku6R9HaSfa0vAzAWfk8NFDPS2+4keVfSu60sATARHKmBYogaKIaogWKIGiiGqIFiiBoohqiBYogaKIaogWKIGiiGqIFiiBoohqiBYogaKIaogWKIGijGSSb/Te1/SfrnhL/tJkn/nvD3bFOf9vZpq9SvvW1t3ZrkqtWuaCXqNthe6tOZSvu0t09bpX7t7WIrD7+BYogaKKZPUS92PWBEfdrbp61Sv/ZOfWtvnlMDaKZPR2oADRA1UEwvora9x/ZXto/bfrzrPRdi+3nbJ21/0fWW9djeYvsd2yu2j9ne3/Wmtdi+1PZHtj8fbn2y601N2J6z/Znt16d1nzMfte05Sc9Iul3Sdkn32t7e7aoLekHSnq5HNHRW0qNJ/iLpZkl/m+F/218l7U7yV0k3Stpj++aONzWxX9LKNO9w5qOWtFPS8STfJPlNg3fevLvjTWtK8p6kU13vaCLJj0k+HX7+swY/fJu7XbW6DPwyvLhx+DHTr/Lanpd0h6Rnp3m/fYh6s6Tvz7t8QjP6g9dntrdJuknSh90uWdvwoeyypJOSDieZ2a1DT0t6TNK5ad5pH6L2Kl+b6f+h+8b2FZJekfRIkp+63rOWJL8nuVHSvKSdtm/oetNabN8p6WSST6Z9332I+oSkLeddnpf0Q0dbyrG9UYOgDyZ5tes9TSQ5rcG7r87yaxe7JN1l+zsNnjLutv3iNO64D1F/LOk629fYvkSDN75/reNNJdi2pOckrSR5qus9F2L7KttXDj+/TNKtkr7sdtXakjyRZD7JNg1+Zt9Osm8a9z3zUSc5K+lhSW9p8ELOy0mOdbtqbbYPSfpA0vW2T9h+oOtNF7BL0n0aHEWWhx97ux61hqslvWP7qAb/0R9OMrVfE/UJfyYKFDPzR2oAoyFqoBiiBoohaqAYogaKIWqgGKIGivkPGaruA1eRIiMAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# plot the image by passing in the 28x28 matrix\n",
    "\n",
    "plt.imshow(img_tensor[0, 10:15, 10:15], cmap = 'gray');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that we need to pass just the 28x28 matrix to plt.imshow, without a channel dimension. We also pass a color map (cmap=gray) to indicate that we want to see a grayscale image."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Training and Validation Datasets\n",
    "\n",
    "While building real-world machine learning models, it is quite common to split the dataset into three parts:\n",
    "\n",
    "1. Training set - used to train the model, i.e., compute the loss and adjust the model's weights using gradient descent.\n",
    "2. Validation set - used to evaluate the model during training, adjust hyperparameters (learning rate, etc.), and pick the best version of the model.\n",
    "3. Test set - used to compare different models or approaches and report the model's final accuracy.\n",
    "\n",
    "In the MNIST dataset, there are 60,000 training images and 10,000 test images. The test set is standardized so that different researchers can report their models' results against the same collection of images.\n",
    "\n",
    "Since there's no predefined validation set, we must manually split the 60,000 images into training and validation datasets. Let's set aside 10,000 randomly chosen images for validation. We can do this using the `random_spilt` method from PyTorch.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(50000, 10000)"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from torch.utils.data import random_split\n",
    "\n",
    "train_ds, valid_ds = random_split(dataset, [50000, 10000])\n",
    "len(train_ds), len(valid_ds)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It's essential to choose a random sample for creating a validation set. Training data is often sorted by the target labels, i.e., images of 0s, followed by 1s, followed by 2s, etc. If we create a validation set using the last 20% of images, it would only consist of 8s and 9s. In contrast, the training set would contain no 8s or 9s. Such a training-validation would make it impossible to train a useful model.\n",
    "\n",
    "We can now create data loaders to help us load the data in batches. We'll use a batch size of 128."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "from torch.utils.data import DataLoader\n",
    "\n",
    "batch_size = 128\n",
    "\n",
    "train_loader = DataLoader(train_ds, batch_size, shuffle=True)\n",
    "val_loader = DataLoader(valid_ds, batch_size)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We set `shuffle=True` for the training data loader to ensure that the batches generated in each epoch are different. This randomization helps generalize & speed up the training process. On the other hand, since the validation data loader is used only for evaluating the model, there is no need to shuffle the images."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}