Bhargav_solanki_assignment_11.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Please fill in your name and that of your teammate.\n",
    "\n",
    "You: Bhargav Solanki\n",
    "\n",
    "Teammate: Carmen Sangro Prieto"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Welcome to the eleventh lab. Deep Learning takes the already complex topic of Neural Networks and turns it up a notch. Several notches, in fact. It's hard to find exercises small enough to fit in a single assignment, let alone a *set* of exercises for all of these topics.\n",
    "\n",
    "So this week the assignment is particularly small, with only 15 points, and should not take you as long as usual to complete. What you should do if you are interested in building Deep Learning experience instead is take one of the Bonus Questions and solve it yourself. We will support fully any question on any of the topics.\n",
    "\n",
    "Willing to learn but unsure on the topic? Go for the **Transfer Learning** tutorial, it's the shortest and one of the most marketable skills. Basically you download a pre-trained network (on a huge dataset), cut the last (decision) layer(s), add your own (so you decide based on the last feature space), then train _only your new layer(s)_ on your specific task, which is fast and easy. While re-using the larger body that was pre-trained by someone else, likely with a larger budget."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### How to pass the lab?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Below you find the exercise questions. Each question awarding points is numbered and states the number of points like this: **[0pt]**. To answer a question, fill the cell below with your answer (markdown for text, code for implementation). Incorrect or incomplete answers are in principle worth 0 points: to assign partial reward is only up to teacher discretion. Over-complete answers do not award extra points (though they are appreciated and will be kept under consideration). Save your work frequently! (`ctrl+s`)\n",
    "\n",
    "**You need at least 10 points (out of 15 available) to pass** (66%)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1. History and implementations"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 1.1 **[1pt]** Mention 3 reasons why DL did not happen 30 years ago."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Technology 30 years ago was not up to the right standard to facilitate neural networks in order for NN and DL to work properly:\n",
    "\n",
    "1) There was a lack of datasets available to develop and train new models and algorithms.\n",
    "\n",
    "2) Storage of huge amounts of data (essential for DL) was not possible. It was also very expensive.\n",
    "\n",
    "3) There didn't exist computers strong enough to process the data.\n",
    "\n",
    "(Source: https://becominghuman.ai/what-is-deep-learning-2dd6192c1de7)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 1.2 **[1pt]** Explain the main pros/cons of GPU vs. CPU"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A **CPU** (central processing unit) is a generalized processor that is designed to carry out a wide variety of tasks. On the contrary **GPU** (graphics processing unit) is a specialized processing unit with enhanced mathematical computation capability, ideal for computer graphics and ML tasks.\n",
    "\n",
    "Advantages of CPUs include:\n",
    "\n",
    "1) **Flexibility**: CPUs are flexible and resilient and can handle a variety of tasks outside of graphics processing.\n",
    "\n",
    "2) **Precision**: CPUs can work on mid-range mathematical equations with a higher level of precision. CPUs can handle the computational depth and complexity more readily, becoming increasingly crucial for specific applications.\n",
    "\n",
    "3) **Access to Memory**: CPUs usually contain significant local cache memory, which means they can handle a larger set of linear instructions and, hence, more complex system and computational operations.\n",
    "\n",
    "4) **Cost and Availability**: CPUs are more readily available, more widely manufactured, and cost-effective for consumer and enterprise use. \n",
    "\n",
    "On the contrary, some disadvantages of CPUs are:\n",
    "\n",
    "1) **Parallel Processing**: CPUs cannot handle parallel processing like a GPU.\n",
    "\n",
    "2) **Compatibility**: Not every system or software is compatible with every processor.\n",
    "\n",
    "Regarding GPUs, the main advantages are:\n",
    "\n",
    "1) **High Data Throughput**: a GPU can push vast volumes of processed data through a workload, speeding up specific tasks beyond what a CPU can handle.\n",
    "\n",
    "2) **Massive Parallel Computing**: GPUs excel in extensive calculations with numerous similar operations, such as computing matrices or modeling complex systems.\n",
    "\n",
    "Finally, disadvantages of GPUs include:\n",
    "\n",
    "1) **Multitasking**: GPUs aren’t built for multitasking, so they don’t have much impact in areas like general-purpose computing.\n",
    "\n",
    "2) **Cost**: GPUs are still significantly more expensive than CPUs.\n",
    "\n",
    "3) **Power and Complexity**: While a GPU can handle large amounts of parallel computing and data throughput, they struggle when the processing requirements become more chaotic. Branching logic paths, sequential operations, and other approaches to computing impede the effectiveness of a GPU.\n",
    "\n",
    "(Source: https://www.weka.io/blog/cpu-vs-gpu/)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 1.3 **[1pt]** How do you implement Model Quantization? What are the advantages?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Model quantization: we store the parametrization of the model in a different format, a smaller data type, which automatically reduces the model's size. The main advantage is that, by reducing the size of the model with minimal loss, we can achieve a better performance. Also, the loading and passing around of the model on the hardware’s buses is also proportionally reduced in time.\n",
    "\n",
    "On top of that, with model quantization, we try ro use data where possible and reduce memory consumption."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2. Long Short-Term Memory"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 2.1 **[5pt]** Implement with Keras a neural network with one LSTM layer and one Dense (decision, output) layer to predict the next number in a sequence."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Generate a dataset from a simple sequence using `gen_dataset()` below. Play with it and print it to understand how it works. Basically, it generates a sequence of multiples of ten, then prepares subsequences of fixed size `x` paired with the next number in the sequence `y`.\n",
    "- Define a model using Sequential, then add an LSTM layer. You can start with 20 neurons, remember to pass `input_shape` because this is your first layer (you saw this last week with the first convolution layer).\n",
    "- For a shallow network, it is better if the LSTM layer uses activations supporting regression, such as `linear` or Rectified Linear Unit ([ReLU](https://machinelearningmastery.com/rectified-linear-activation-function-for-deep-learning-neural-networks/)).\n",
    "- The next layer needs to squeeze the LSTM processing into a single-digit decision. How many neurons you need for this? Which activation function will you choose for the output of this regression problem?\n",
    "- Next you need to compile, and for practical reasons I suggest you use the [Adam optimizer](https://keras.io/optimizers/#adam) (explained [here](https://machinelearningmastery.com/adam-optimization-algorithm-for-deep-learning/)) instead of good _old_ SGD. I use a learning rate of `1e-3` The MSE loss still works.\n",
    "- Finally you can fit it. On my machine 100 epochs don't take long, and I use `verbose=0` to avoid the long output."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[470, 480, 490]\n",
      "[[[470]\n",
      "  [480]\n",
      "  [490]]]\n"
     ]
    }
   ],
   "source": [
    "# Here's some code to get you started\n",
    "\n",
    "import numpy as np\n",
    "from keras.models import Sequential\n",
    "from keras.layers import Dense, LSTM\n",
    "from tensorflow.keras.optimizers import Adam\n",
    "\n",
    "# Let's predict data from a simple linear `y=10x` for starters\n",
    "sequence = list(range(10, 500, 10))    # this will *exclude* 500\n",
    "# We use a simple sliding window of size 3 and predict each 4th\n",
    "test_x, test_y = [470, 480, 490], 500  # so we can use 500 for testing\n",
    "\n",
    "# params: original data sequence, size of sliding window\n",
    "def gen_dataset(sequence=sequence, win_size=len(test_x)):\n",
    "    # I use a sequence of generators with different starting points\n",
    "    gen_x = [sequence[start:-1] for start in range(win_size)]\n",
    "    x = np.array(list(zip(*gen_x))) # then I simply zip them. Can you understand this?\n",
    "    y = np.array(sequence[win_size:])\n",
    "    x = x.reshape((*x.shape,1)) # Keras requires the \"color channels\" dimension    \n",
    "    return x, y\n",
    "\n",
    "# Let's not forget to reshape this too\n",
    "print(test_x)\n",
    "test_x = np.array(test_x).reshape(1,len(test_x),1)\n",
    "print(test_x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 40  50  60  70  80  90 100 110 120 130 140 150 160 170 180 190 200 210\n",
      " 220 230 240 250 260 270 280 290 300 310 320 330 340 350 360 370 380 390\n",
      " 400 410 420 430 440 450 460 470 480 490]\n",
      "1/1 [==============================] - 0s 92ms/step\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "array([[502.99747]], dtype=float32)"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from keras import Input\n",
    "train_x, train_y = gen_dataset()\n",
    "print(train_y)\n",
    "model = Sequential()\n",
    "shape = test_x.shape\n",
    "model.add(Input(shape=shape[1:]))\n",
    "# add LSTM with 20 nuerons\n",
    "model.add(LSTM(20, activation='relu'))\n",
    "# Add a Dense layer with 1 units.\n",
    "model.add(Dense(1,  activation=\"linear\"))\n",
    "model.compile(optimizer=Adam(learning_rate=1e-3), loss=\"mean_squared_error\")\n",
    "\n",
    "#fit model\n",
    "history = model.fit(train_x, train_y, validation_split=0.1,\n",
    "                    epochs=150, batch_size=10, verbose=0)\n",
    "model.predict(test_x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 2.2 **[3pt]** Change the model to use two LSTM layers instead of one, of 50 neurons each and ReLU activation, plus of course the output linear layer Dense. Visualize the model training progression with a metric of your choice (from [here](https://keras.io/api/metrics/regression_metrics/))."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- The first LSTM needs to pass a whole input sequence to the next one for a stack of LSTM to work, you need to use the `return_sequences` parameter.\n",
    "- Again copy & paste here works well. Here are some SE rules of thumb that maybe are less obvious/known than I expected:\n",
    "    - If you do something once, rough is ok.\n",
    "    - If you do it twice, copy+paste is ok.\n",
    "    - If you do it 3 times, it's time to write a function, the first 2 cases will tell you which parts are fixed and which parameters you need. Clean it up a bit.\n",
    "    - If you call something a lot (e.g. more than 5 times), you better make it readable, refactor names, create classes, and add documentation.\n",
    "    - You _never_ optimize \"by eye\", or just because \"you know how to do it\": when the program feels slow you run a profiler, and you optimize the objectively slowest part, then repeat until it's fast enough.\n",
    "    - You _never_ add infrastructure because it's the right thing to do, aim at elegance and readability instead, and rather avoid using languages that require early structure or are hard to refactor. E.g. C++ you need a distructor from the beginning, Python instead you can keep writing script until you _need_ a class. Important: don't script because lazy and less work, script to make it more compact and readable, and if you need a class invest some time into designing it on paper before writing it.\n",
    "- Remember that to visualize performance you need to compile with a metric, train with a validation split, and save the training history in a variable.\n",
    "- Remember also that classification and regression have different metrics: you cannot use accuracy here!\n",
    "- Since you have more neurons to train, and specifically one extra layer, training for more epochs is a good idea. Feel free to also generate more data if you like.\n",
    "- A bigger network does not necessarily mean better results, as should be clear by now."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[500.91272]], dtype=float32)"
      ]
     },
     "execution_count": 66,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "model = Sequential()\n",
    "shape = test_x.shape\n",
    "model.add(Input(shape=shape[1:]))\n",
    "# add LSTM with 50 nuerons\n",
    "model.add(LSTM(50, activation='relu', return_sequences=True))\n",
    "# add LSTM with 50 nuerons\n",
    "model.add(LSTM(50, activation='relu'))\n",
    "# Add a Dense layer with 1 unit.\n",
    "model.add(Dense(1,  activation=\"linear\"))\n",
    "model.compile(optimizer=Adam(learning_rate=1e-3), loss=\"mean_squared_error\")\n",
    "\n",
    "#fit model\n",
    "history = model.fit(train_x, train_y, validation_split=0.1,\n",
    "                    epochs=250, batch_size=10, verbose=0)\n",
    "model.predict(test_x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZIAAAEWCAYAAABMoxE0AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAjhUlEQVR4nO3df5xV9X3n8dd77tz5haD8kodhMJBIU8UqBqSm7uaRlEapaYrbaELTRJrlEVpjt8m26Rb7Y9vuY9nV7bamttGEVCvaRKUaV9pqUsWabh416GiNgsoyKsooEQRElJ8Dn/3jfC/eGS/jzNx75zpz3s/H4z7uuZ9zvme+37nAh+/3e875KiIwMzMbrqZGV8DMzEY3JxIzM6uKE4mZmVXFicTMzKriRGJmZlVxIjEzs6o4kZiNEEk3Sfrvgzx2i6Sfq/Y8ZiPBicTMzKriRGJmZlVxIjErk4aUfkfSE5LelHSDpGmS7pW0V9L9kiaWHf+LkjZKek3Sg5JOL9t3jqTHUrnbgbZ+P+sXJD2eyv6rpLOGWecvSOqWtEvSWknvSXFJukbSdkl7UpvOTPsukvRUqttLkr4yrF+YGU4kZpV8EvgY8BPAJ4B7gd8DppD9nflNAEk/AdwKfBmYCtwD/L2kFkktwP8BbgEmAX+Xzksq+0HgRuDXgMnAN4C1klqHUlFJPwv8T+BTwCnAC8BtafcFwIdTO04CPg3sTPtuAH4tIsYDZwIPDOXnmpVzIjF7u7+MiFci4iXg/wLrI+LfIuIgcBdwTjru08A/RsR9EXEY+N9AO/AzwHlAEfhqRByOiDuAR8p+xheAb0TE+og4EhGrgYOp3FD8CnBjRDyW6ncl8CFJM4HDwHjgJwFFxNMRsS2VOwycIWlCROyOiMeG+HPNjnEiMXu7V8q291f4fELafg9ZDwCAiDgKbAWmp30vRd+nor5Qtv1e4LfTsNZrkl4DZqRyQ9G/Dm+Q9TqmR8QDwF8BXwNekbRK0oR06CeBi4AXJH1f0oeG+HPNjnEiMRu+l8kSApDNSZAlg5eAbcD0FCs5tWx7K7AyIk4qe3VExK1V1mEc2VDZSwARcW1EzAPmkA1x/U6KPxIRi4GTyYbg1gzx55od40RiNnxrgI9LWiipCPw22fDUvwIPAb3Ab0pqlvRLwIKyst8Efl3ST6dJ8XGSPi5p/BDr8G3g85LmpvmV/0E2FLdF0rnp/EXgTeAAcCTN4fyKpBPTkNzrwJEqfg+Wc04kZsMUEZuAzwJ/CbxKNjH/iYg4FBGHgF8CfhXYTTaf8p2ysl1k8yR/lfZ3p2OHWod1wB8Cd5L1gt4PLEm7J5AlrN1kw187yeZxAD4HbJH0OvDrqR1mwyIvbGVmZtVwj8TMzKriRGJmZlVxIjEzs6o4kZiZWVWaG12BkTZlypSYOXNmo6thZjaqPProo69GxNRK+3KXSGbOnElXV1ejq2FmNqpIeuF4+zy0ZWZmValbIpH0gfSI7NLrdUlfljRJ0n2SNqf38kdyX5keh71J0oVl8XmSnkz7ri09dkJSq6TbU3x9elCdmZmNoLolkojYFBFzI2IuMA/YR/bk1BXAuoiYDaxLn5F0BtkduXOARcB1kgrpdNcDy4HZ6bUoxZcBuyPiNOAa4Op6tcfMzCobqTmShcCzEfGCpMXAR1J8NfAg8LvAYuC29Cjs5yV1AwskbQEmRMRDAJJuBi4mWyNiMfDH6Vx3AH8lSeHb9c2sxg4fPkxPTw8HDhxodFXqqq2tjc7OTorF4qDLjFQiWUK2ABDAtNKaCBGxTdLJKT4d+GFZmZ4UO5y2+8dLZbamc/VK2kP25NNX69EIM8uvnp4exo8fz8yZM+n7UOexIyLYuXMnPT09zJo1a9Dl6j7ZnlaK+0WyFeIGPLRCLAaID1Smfx2WS+qS1LVjx453qIaZ2dsdOHCAyZMnj9kkAiCJyZMnD7nXNRJXbf088FhElBYHekXSKQDpfXuK95Ct5VDSSbbWQk/a7h/vU0ZSM3AisKt/BSJiVUTMj4j5U6dWvAzazOwdjeUkUjKcNo5EIvll3hrWAlgLLE3bS4G7y+JL0pVYs8gm1R9Ow2B7JZ2Xrta6rF+Z0rkuAR6o1/xI15ZdXP3dZ/D0i5lZX3VNJJI6gI9Rtg4DcBXwMUmb076rACJiI9lCQU8B3wWuiIjSYjuXA39NtmbDs2QT7QA3AJPTxPxvka4Aq4cnevZw/YPPsuvNQ/X6EWZmx/Xaa69x3XXXDbncRRddxGuvvVb7CpWp62R7ROwjm/wuj+0ku4qr0vErgZUV4l3AmRXiB4BLa1LZdzBjUgcAW3fvZ/IJrSPxI83Mjiklki9+8Yt94keOHKFQKBynFNxzzz31rprvbB+sGZPaAXhx174G18TM8mjFihU8++yzzJ07l3PPPZePfvSjfOYzn+GnfuqnALj44ouZN28ec+bMYdWqVcfKzZw5k1dffZUtW7Zw+umn84UvfIE5c+ZwwQUXsH///prULXfP2hquGRNTj8SJxCz3/uTvN/LUy6/X9JxnvGcCf/SJOcfdf9VVV7FhwwYef/xxHnzwQT7+8Y+zYcOGY5fp3njjjUyaNIn9+/dz7rnn8slPfpLJk/sMCLF582ZuvfVWvvnNb/KpT32KO++8k89+tvpVlp1IBmlcazOTxrXQs9uJxMwab8GCBX3u9bj22mu56667ANi6dSubN29+WyKZNWsWc+fOBWDevHls2bKlJnVxIhmCGZM62LqrNl1BMxu9Buo5jJRx48Yd237wwQe5//77eeihh+jo6OAjH/lIxXtBWlvfmt8tFAo1G9ryHMkQzJjYzlb3SMysAcaPH8/evXsr7tuzZw8TJ06ko6ODZ555hh/+8IcVj6sX90iGYMakDr638cccORoUmsb+jUlm9u4xefJkzj//fM4880za29uZNm3asX2LFi3i61//OmeddRYf+MAHOO+880a0bk4kQzBjYgeHjwQ/fv0A009qb3R1zCxnvv3tb1eMt7a2cu+991bcV5oHmTJlChs2bDgW/8pXvlKzenloawhKlwD7yi0zs7c4kQzBtAltAOzYe7DBNTEze/dwIhmCKemO9lffcCIxy6M8PGtvOG10IhmCk9qLFJrkRGKWQ21tbezcuXNMJ5PSeiRtbW1DKufJ9iFoahKTxrXw6l4/uNEsbzo7O+np6WGsr2lUWiFxKJxIhmjKCa3ukZjlULFYHNKqgXnioa0hmnJCixOJmVkZJ5IhmnpCK6++4aEtM7MSJ5IhmjK+lR1vHBzTE25mZkPhRDJEU05o4VDvUfYe7G10VczM3hWcSIbo2L0kvinRzAxwIhmyUiLZ6bXbzcwAJ5Ihc4/EzKyvuiYSSSdJukPSM5KelvQhSZMk3Sdpc3qfWHb8lZK6JW2SdGFZfJ6kJ9O+ayUpxVsl3Z7i6yXNrGd7AKaMbwH8mBQzs5J690j+AvhuRPwkcDbwNLACWBcRs4F16TOSzgCWAHOARcB1kgrpPNcDy4HZ6bUoxZcBuyPiNOAa4Oo6t4dJHaVE4qEtMzOoYyKRNAH4MHADQEQciojXgMXA6nTYauDitL0YuC0iDkbE80A3sEDSKcCEiHgosmtub+5XpnSuO4CFpd5KvTQXmhjXUmDvAV+1ZWYG9e2RvA/YAfyNpH+T9NeSxgHTImIbQHo/OR0/HdhaVr4nxaan7f7xPmUiohfYA/Rd7R6QtFxSl6SuWjwnZ0J7kb0HDld9HjOzsaCeiaQZ+CBwfUScA7xJGsY6jko9iRggPlCZvoGIVRExPyLmT506deBaD8L4tmb3SMzMknomkh6gJyLWp893kCWWV9JwFel9e9nxM8rKdwIvp3hnhXifMpKagROBXTVvST/j24q87h6JmRlQx0QSET8Gtkr6QAotBJ4C1gJLU2wpcHfaXgssSVdizSKbVH84DX/tlXRemv+4rF+Z0rkuAR6IEXh2yQT3SMzMjqn3Y+T/E/AtSS3Ac8DnyZLXGknLgBeBSwEiYqOkNWTJphe4IiKOpPNcDtwEtAP3phdkE/m3SOom64ksqXN7gKxH8tyrb47EjzIze9erayKJiMeB+RV2LTzO8SuBlRXiXcCZFeIHSIloJHmOxMzsLb6zfRhKV235CcBmZk4kwzK+rZnDR4IDh482uipmZg3nRDIM49uKAL6XxMwMJ5JhmdCWTS297nkSMzMnkuGYkHokvpfEzMyJZFgmtGc9El+5ZWbmRDIspTmS1/e7R2Jm5kQyDOPb3CMxMytxIhmGCb5qy8zsGCeSYehoKVBokifbzcxwIhkWSZzQ6sekmJmBE8mwTWh3IjEzAyeSYRvfWvRVW2ZmOJEM27jWAvsOHXnnA83MxjgnkmFqKxbYf9iJxMzMiWSYOloK7HePxMzMiWS42t0jMTMDnEiGrb3FicTMDJxIhq292OyhLTMznEiGrb2lif2Hj3i5XTPLvbomEklbJD0p6XFJXSk2SdJ9kjan94llx18pqVvSJkkXlsXnpfN0S7pWklK8VdLtKb5e0sx6tqdce7HAkaPB4SNOJGaWbyPRI/loRMyNiPnp8wpgXUTMBtalz0g6A1gCzAEWAddJKqQy1wPLgdnptSjFlwG7I+I04Brg6hFoDwDtLdkTgD28ZWZ514ihrcXA6rS9Gri4LH5bRByMiOeBbmCBpFOACRHxUGTjSDf3K1M61x3AwlJvpd7ai1mO84S7meVdvRNJAP8k6VFJy1NsWkRsA0jvJ6f4dGBrWdmeFJuetvvH+5SJiF5gDzC5fyUkLZfUJalrx44dNWlYe0v2q3MiMbO8a67z+c+PiJclnQzcJ+mZAY6t1JOIAeIDlekbiFgFrAKYP39+TSY12ovZr27fIT+40czyra49koh4Ob1vB+4CFgCvpOEq0vv2dHgPMKOseCfwcop3Voj3KSOpGTgR2FWPtvTX3pINbR1wj8TMcq5uiUTSOEnjS9vABcAGYC2wNB22FLg7ba8FlqQrsWaRTao/nIa/9ko6L81/XNavTOlclwAPxAhdj1uaI/GDG80s7+o5tDUNuCvNfTcD346I70p6BFgjaRnwInApQERslLQGeAroBa6IiNK/0pcDNwHtwL3pBXADcIukbrKeyJI6tqePjtQj8VVbZpZ3dUskEfEccHaF+E5g4XHKrARWVoh3AWdWiB8gJaKR1uartszMAN/ZPmzukZiZZZxIhsn3kZiZZZxIhql01ZYTiZnlnRPJMLU2NyF5aMvMzIlkmCRli1s5kZhZzjmRVMGrJJqZOZFUpd3rtpuZOZFUwz0SMzMnkqp43XYzMyeSqrQXC37WlpnlnhNJFdpbCn76r5nlnhNJFXz5r5mZE0lV2ls8tGVm5kRShfaih7bMzJxIqtDhq7bMzJxIqlG6j2SEFmU0M3tXciKpQltLgQg42Hu00VUxM2sYJ5IqdHjddjMzJ5JqeE0SMzMnkqocW7fdPRIzy7G6JxJJBUn/Jukf0udJku6TtDm9Tyw79kpJ3ZI2SbqwLD5P0pNp37WSlOKtkm5P8fWSZta7PeU6WpoBJxIzy7eR6JF8CXi67PMKYF1EzAbWpc9IOgNYAswBFgHXSSqkMtcDy4HZ6bUoxZcBuyPiNOAa4Or6NqUvr9tuZlbnRCKpE/g48Ndl4cXA6rS9Gri4LH5bRByMiOeBbmCBpFOACRHxUGTX2d7cr0zpXHcAC0u9lZHQ3pL9+pxIzCzP6t0j+SrwX4Dy62OnRcQ2gPR+copPB7aWHdeTYtPTdv94nzIR0QvsASb3r4Sk5ZK6JHXt2LGjyia9pb1YGtrqrdk5zcxGm7olEkm/AGyPiEcHW6RCLAaID1SmbyBiVUTMj4j5U6dOHWR13pmv2jIzg+Y6nvt84BclXQS0ARMk/S3wiqRTImJbGrbano7vAWaUle8EXk7xzgrx8jI9kpqBE4Fd9WpQf8fmSA75hkQzy6+69Ugi4sqI6IyImWST6A9ExGeBtcDSdNhS4O60vRZYkq7EmkU2qf5wGv7aK+m8NP9xWb8ypXNdkn7GiD2vpNQj2eehLTPLsXr2SI7nKmCNpGXAi8ClABGxUdIa4CmgF7giIkpjRpcDNwHtwL3pBXADcIukbrKeyJKRagS81SPxE4DNLM9GJJFExIPAg2l7J7DwOMetBFZWiHcBZ1aIHyAlokYoFkShSZ4jMbNc853tVZBEh9dtN7OccyKpUpvXbTeznHMiqVJHi9dtN7N8cyKpUruHtsws5waVSCR9SdIEZW6Q9JikC+pdudGgrejlds0s3wbbI/mPEfE6cAEwFfg82WW8udfhORIzy7nBJpLSo0guAv4mIn5E5ceT5I6Htsws7wabSB6V9E9kieR7ksbT90GMudXW4qEtM8u3wd6QuAyYCzwXEfskTSIb3sq9jqKv2jKzfBtsj+RDwKaIeE3SZ4E/IHtke+61u0diZjk32ERyPbBP0tlk64u8QLbAVO61u0diZjk32ETSm56quxj4i4j4C2B8/ao1erS3FDjYe5QjR0fsocNmZu8qg00keyVdCXwO+Me0lnqxftUaPfwEYDPLu8Emkk8DB8nuJ/kx2RK3f1q3Wo0iXiXRzPJuUIkkJY9vASemJXQPRITnSChfJdGJxMzyabCPSPkU8DDZ2h+fAtZLuqSeFRst2jy0ZWY5N9j7SH4fODcitgNImgrcD9xRr4qNFi3NWS4+2Ov7M80snwY7R9JUSiLJziGUHdNaCtmv4fARJxIzy6fB9ki+K+l7wK3p86eBe+pTpdGl1CM55B6JmeXUYCfbfwdYBZwFnA2siojfHaiMpDZJD0v6kaSNkv4kxSdJuk/S5vQ+sazMlZK6JW2SdGFZfJ6kJ9O+ayUpxVsl3Z7i6yXNHPJvoEqlRHL4iO8jMbN8GvTwVETcGRG/FRH/OSLuGkSRg8DPRsTZZM/pWiTpPGAFsC4iZgPr0mcknQEsAeYAi4Dr0v0qkN1ZvxyYnV6LUnwZsDsiTgOuAa4ebHtqpZiGtg4d8WS7meXTgIlE0l5Jr1d47ZX0+kBlI/NG+lhMr9Ld8atTfDVwcdpeDNwWEQcj4nmgG1gg6RRgQkQ8lO6uv7lfmdK57gAWlnorI6U0R+KhLTPLqwHnSCKiqsegpB7Fo8BpwNciYr2kaRGxLZ1/m6ST0+HTgR+WFe9JscNpu3+8VGZrOlevpD3AZODVauo9FC3NWd465KEtM8upul55FRFHImIu0EnWuzhzgMMr9SRigPhAZfqeWFouqUtS144dO96h1kPTUshG39wjMbO8GpFLeCPiNeBBsrmNV9JwFem9dFlxDzCjrFgn8HKKd1aI9ykjqRk4EdhV4eevioj5ETF/6tSptWlU8tZkuxOJmeVT3RKJpKmSTkrb7cDPAc8Aa4Gl6bClwN1pey2wJF2JNYtsUv3hNAy2V9J5af7jsn5lSue6BHggzaOMmGIhDW25R2JmOTXY+0iG4xRgdZonaQLWRMQ/SHoIWCNpGfAi2WNXiIiNktYATwG9wBURUboU6nLgJqAduDe9AG4AbpHUTdYTWVLH9lTk+0jMLO/qlkgi4gngnArxncDC45RZCaysEO8C3ja/EhEHSImoUY4lEg9tmVlO+TEnVSo2uUdiZvnmRFKlpiZRLMiT7WaWW04kNVAsNLlHYma55URSAy3NTZ4jMbPcciKpgZZCk4e2zCy3nEhqoFho8sJWZpZbTiQ10NrsORIzyy8nkhpoafbQlpnllxNJDfiqLTPLMyeSGsh6JH6MvJnlkxNJDbS4R2JmOeZEUgPF5iYOeo7EzHLKiaQGWgpNHHaPxMxyyomkBlqa5TvbzSy3nEhqwHe2m1meOZHUQItvSDSzHHMiqQHfR2JmeeZEUgN++q+Z5ZkTSQ14aMvM8syJpAY82W5meVa3RCJphqR/lvS0pI2SvpTikyTdJ2lzep9YVuZKSd2SNkm6sCw+T9KTad+1kpTirZJuT/H1kmbWqz0DaSk0cTSg18nEzHKonj2SXuC3I+J04DzgCklnACuAdRExG1iXPpP2LQHmAIuA6yQV0rmuB5YDs9NrUYovA3ZHxGnANcDVdWzPcRWbs1+j50nMLI/qlkgiYltEPJa29wJPA9OBxcDqdNhq4OK0vRi4LSIORsTzQDewQNIpwISIeCgiAri5X5nSue4AFpZ6KyOppZD9Gg/3+sGNZpY/IzJHkoaczgHWA9MiYhtkyQY4OR02HdhaVqwnxaan7f7xPmUiohfYA0yu8POXS+qS1LVjx44ateotLalHcvDIkZqf28zs3a7uiUTSCcCdwJcj4vWBDq0QiwHiA5XpG4hYFRHzI2L+1KlT36nKQ1bqkfjKLTPLo7omEklFsiTyrYj4Tgq/koarSO/bU7wHmFFWvBN4OcU7K8T7lJHUDJwI7Kp9SwZW6pF4TRIzy6N6XrUl4Abg6Yj487Jda4GlaXspcHdZfEm6EmsW2aT6w2n4a6+k89I5L+tXpnSuS4AH0jzKiColEvdIzCyPmut47vOBzwFPSno8xX4PuApYI2kZ8CJwKUBEbJS0BniK7IqvKyKiNOlwOXAT0A7cm16QJapbJHWT9USW1LE9x1UsTbb7qi0zy6G6JZKI+AGV5zAAFh6nzEpgZYV4F3BmhfgBUiJqpGOT7e6RmFkO+c72GvBku5nlmRNJDbQ0Zx0vD22ZWR45kdRASyG7Ad89EjPLIyeSGii6R2JmOeZEUgPH5kicSMwsh5xIauDYVVuHnUjMLH+cSGqgrZjNkRzs9bO2zCx/nEhqoD0lkv2HnUjMLH+cSGqg1CM54KEtM8shJ5IaKDSJYkEccI/EzHLIiaRG2poLHtoys1xyIqmRtpaCh7bMLJecSGqkrdjEQfdIzCyHnEhqxENbZpZXTiQ10t5S8GS7meWSE0mNtDV7jsTM8smJpEZai00c8J3tZpZDTiQ10l4ssP+QE4mZ5Y8TSY20FQteatfMcsmJpEbaik2ebDezXKpbIpF0o6TtkjaUxSZJuk/S5vQ+sWzflZK6JW2SdGFZfJ6kJ9O+ayUpxVsl3Z7i6yXNrFdbBqOt6Mt/zSyf6tkjuQlY1C+2AlgXEbOBdekzks4AlgBzUpnrJBVSmeuB5cDs9CqdcxmwOyJOA64Brq5bSwahvejLf80sn+qWSCLiX4Bd/cKLgdVpezVwcVn8tog4GBHPA93AAkmnABMi4qGICODmfmVK57oDWFjqrTRCazG7/DerpplZfoz0HMm0iNgGkN5PTvHpwNay43pSbHra7h/vUyYieoE9wORKP1TSckldkrp27NhRo6b01VZMqyR6wt3McubdMtleqScRA8QHKvP2YMSqiJgfEfOnTp06zCoOrP3YmiQe3jKzfBnpRPJKGq4ivW9P8R5gRtlxncDLKd5ZId6njKRm4ETePpQ2Yry4lZnl1UgnkrXA0rS9FLi7LL4kXYk1i2xS/eE0/LVX0nlp/uOyfmVK57oEeCAaOEFRGtrylVtmljfN9TqxpFuBjwBTJPUAfwRcBayRtAx4EbgUICI2SloDPAX0AldEROlf5MvJrgBrB+5NL4AbgFskdZP1RJbUqy2D4aEtM8uruiWSiPjl4+xaeJzjVwIrK8S7gDMrxA+QEtG7QasTiZnl1Ltlsn3Ua2vOEomHtswsb5xIauTY5b+ebDeznHEiqZH2Fg9tmVk+OZHUSGloy2uSmFneOJHUSOk+kv2HPLRlZvniRFIjvvzXzPLKiaRGWtNku4e2zCxvnEhqpLW5CQkOeLldM8sZJ5IakURbc4EDfvqvmeWME0kNtRWb2O8eiZnljBNJDZ3U0cKufYcaXQ0zsxHlRFJDnRPb6dm1r9HVMDMbUU4kNdQ5sYOe3fsbXQ0zsxHlRFJDnRPb2fnmIfYd6m10VczMRowTSQ11TmwH4CX3SswsR5xIamjGpA4Atu7eRwMXazQzG1FOJDVU6pF89f7NnLtyHW8c9BCXmY19TiQ1NPWEVlqbm3iiZw+vvnGQH2ze0egqmZnVnRNJDUk61isBuO+p7Q2sjZnZyKjbmu15deqkDnbsPciH3j+Zf960nSNHg0KTGl0tM7O6GfU9EkmLJG2S1C1pRaPrc+VFp3PDr57LJ85+D7vePMSDm9wrMbOxbVT3SCQVgK8BHwN6gEckrY2IpxpVp5+YNh7I1iV539Rx/Ne7N3L4yFEmn9DKvFMn0uTeiZmNMaM6kQALgO6IeA5A0m3AYqBhiaSkrVjgTy85i0u+/hC//rePAdmj5tuKBYqFJsrzidK2UJ/PWax0zNsTkAY4R9/Y28+ht2302bScqPTnysauLy2czSfOfk/NzzvaE8l0YGvZ5x7gp/sfJGk5sBzg1FNPHZmaAfPeO4m7vng+EcGLu/ax4aU9HD4SHOw9CmT3mZRuNzn2zlv3n7wVo0KsLFrxuOgTi7cf3udeF9/1kkP+0nPnxPZiXc472hNJpf9Ove2vR0SsAlYBzJ8/f0T/+sydcRIA55w6kcVzp4/kjzYzGxGjfbK9B5hR9rkTeLlBdTEzy6XRnkgeAWZLmiWpBVgCrG1wnczMcmVUD21FRK+k3wC+BxSAGyNiY4OrZWaWK6M6kQBExD3APY2uh5lZXo32oS0zM2swJxIzM6uKE4mZmVXFicTMzKqivK3kJ2kH8MIwi08BXq1hdUaDPLYZ8tlutzkfhtvm90bE1Eo7cpdIqiGpKyLmN7oeIymPbYZ8ttttzod6tNlDW2ZmVhUnEjMzq4oTydCsanQFGiCPbYZ8ttttzoeat9lzJGZmVhX3SMzMrCpOJGZmVhUnkkGStEjSJkndklY0uj71ImmLpCclPS6pK8UmSbpP0ub0PrHR9ayGpBslbZe0oSx23DZKujJ975skXdiYWlfnOG3+Y0kvpe/6cUkXle0bC22eIemfJT0taaOkL6X4mP2uB2hzfb/riPDrHV5kj6h/Fngf0AL8CDij0fWqU1u3AFP6xf4XsCJtrwCubnQ9q2zjh4EPAhveqY3AGen7bgVmpT8HhUa3oUZt/mPgKxWOHSttPgX4YNoeD/y/1LYx+10P0Oa6ftfukQzOAqA7Ip6LiEPAbcDiBtdpJC0GVqft1cDFjatK9SLiX4Bd/cLHa+Ni4LaIOBgRzwPdZH8eRpXjtPl4xkqbt0XEY2l7L/A0MJ0x/F0P0ObjqUmbnUgGZzqwtexzDwN/OaNZAP8k6VFJy1NsWkRsg+wPKnByw2pXP8dr41j/7n9D0hNp6Ks0xDPm2ixpJnAOsJ6cfNf92gx1/K6dSAZHFWJj9brp8yPig8DPA1dI+nCjK9RgY/m7vx54PzAX2Ab8WYqPqTZLOgG4E/hyRLw+0KEVYqOy3RXaXNfv2olkcHqAGWWfO4GXG1SXuoqIl9P7duAusm7uK5JOAUjv2xtXw7o5XhvH7HcfEa9ExJGIOAp8k7eGNMZMmyUVyf5B/VZEfCeFx/R3XanN9f6unUgG5xFgtqRZklqAJcDaBtep5iSNkzS+tA1cAGwga+vSdNhS4O7G1LCujtfGtcASSa2SZgGzgYcbUL+aK/1jmvwHsu8axkibJQm4AXg6Iv68bNeY/a6P1+a6f9eNvspgtLyAi8iugHgW+P1G16dObXwf2RUcPwI2ltoJTAbWAZvT+6RG17XKdt5K1r0/TPY/smUDtRH4/fS9bwJ+vtH1r2GbbwGeBJ5I/6CcMsba/O/IhmmeAB5Pr4vG8nc9QJvr+l37ESlmZlYVD22ZmVlVnEjMzKwqTiRmZlYVJxIzM6uKE4mZmVXFicRsFJH0EUn/0Oh6mJVzIjEzs6o4kZjVgaTPSno4rf3wDUkFSW9I+jNJj0laJ2lqOnaupB+mB+rdVXqgnqTTJN0v6UepzPvT6U+QdIekZyR9K93NbNYwTiRmNSbpdODTZA/AnAscAX4FGAc8FtlDMb8P/FEqcjPwuxFxFtndx6X4t4CvRcTZwM+Q3ZkO2RNdv0y2lsT7gPPr3CSzATU3ugJmY9BCYB7wSOostJM9GPAocHs65m+B70g6ETgpIr6f4quBv0vPPJseEXcBRMQBgHS+hyOiJ31+HJgJ/KDurTI7DicSs9oTsDoiruwTlP6w33EDPZ9ooOGqg2XbR/DfY2swD22Z1d464BJJJ8OxNcLfS/b37ZJ0zGeAH0TEHmC3pH+f4p8Dvh/ZGhI9ki5O52iV1DGSjTAbLP9PxqzGIuIpSX9AttJkE9kTd68A3gTmSHoU2EM2jwLZo8y/nhLFc8DnU/xzwDck/bd0jktHsBlmg+an/5qNEElvRMQJja6HWa15aMvMzKriHomZmVXFPRIzM6uKE4mZmVXFicTMzKriRGJmZlVxIjEzs6r8f4R375a7YMClAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(history.history['loss'])\n",
    "plt.title('model loss')\n",
    "plt.ylabel('loss')\n",
    "plt.xlabel('epoch')\n",
    "plt.legend(['train', 'test'], loc='upper right')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### BONUS **[ZERO pt]** Change the data generated to something a bit more challenging than `y=10x`, e.g. a sinusoidal wave or another nonlinearity with more complexity. Run your code again and you should see a larger difference in performance between single and double LSTM layers. Remember that the network structure is only an upper bound for the function complexity."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3. Deep Convolutional Neural Networks"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 3.1 **[2pt]** Calculate the dimension of the feature space of the third layer of LeNet-5 (16 filters, slide 22). Explain your reasoning.\n",
    "\n",
    "- Remember it uses Valid Convolution for padding.\n",
    "- The filter size is really $(5\\times5\\times6)$ since it takes all channels at a time.\n",
    "- The number of Filters is the number of neurons."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "10 x 10 x 16\n",
    "Because, we have a 5 x 5 mask so for each of the 16 filters, we will have one output each of dimensions 10 x 10. How did we get 10 x 10, well take a 14 x 14 matrix and convolve witha filter of 5 x 5 and stride 1 we get a 10 x 10 x 16"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 3.2 **[2pt]** Explain in English the results of the Microsoft Tay Twitter chatbot experiment. Propose a safer alternative experiment protocol."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There were three main problems with Microsofts Bot.\n",
    "1) Garbage in garbage out :\n",
    "Twitter is full of people writing garbage, insult and sometimes news and also racist and hatred messages. The bot was trained on bad information. So naturally, it spat out garbage.\n",
    "2) Its a black box: \n",
    "There is no ethics or morals that decide what to keep and clearly, its not AI. It just learned what it read. \n",
    "\n",
    "The bot should have been subjected to have another ML model, that would judge the toxicity score of the output, before it even gave any output. The original model should try to beet toxicty threshold and the the Toxicity model, vice versa. OpenAI bots actually do that."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 4. Generative Adversarial Networks"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### BONUS **[ZERO pt]** GANs are amazing tools and a great topic, but they are complex enough that implementing a decent example would require a lab by itself. So here is a [great tutorial](https://colab.research.google.com/github/tensorflow/gan/blob/master/tensorflow_gan/examples/colab_notebooks/tfgan_tutorial.ipynb), if you choose to play with it share your progress on Moodle and we'll support you!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 5. Transfer Learning"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If I were to only do **one** Bonus Question in the entire course, and was interested in taking a job using Deep Learning after the university, I would do this one here. It is too much work to complete to _require_ everyone to do it, but is probably the most valuable exercise in this whole assignment if you wish to do it.\n",
    "\n",
    "Transfer Learning is easily the most useful and powerful technique to know when you first get a job that expects you to apply Deep Learning -- granted, IF you know how NNs work, as required for this course. It allows you to simply download enormous networks that have been trained on supercomputers using unbelievably large datasets, then specialize them your specific problem and use their results for free."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### BONUS **[ZERO pt]** Follow [this tutorial](https://keras.io/guides/transfer_learning/) on Transfer Learning."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# At the end of the exercise"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Bonus question with no points! Answering this will have no influence on your scoring, not at the assignment and not towards the exam score -- really feel free to ignore it with no consequence. But solving it will reward you with skills that will make the next lectures easier, give you real applications, and will be good practice towards the exam.\n",
    "\n",
    "The solution for this questions will not be included in the regular lab solutions pdf, but you are welcome to open a discussion on the Moodle: we will support your addressing it, and you may meet other students that choose to solve this, and find a teammate for the next assignment that is willing to do things for fun and not only for score :)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### BONUS **[ZERO pt]** You now know the basis for time series prediction using recurrent networks. Why don't you try your hand at predicting the evolution of the current COVID-19 situation? Specifically look at the Reproduction number, which is the base for the exponential growth of the infection. You can find the main data from JHU CSSE [here](https://github.com/CSSEGISandData/COVID-19), then the data for Switzerland [here](https://github.com/openZH/covid_19) (specifically Fribourg [here](https://github.com/openZH/covid_19/blob/master/fallzahlen_kanton_total_csv_v2/COVID19_Fallzahlen_Kanton_FR_total.csv)), some work from ETHZ [here](https://bsse.ethz.ch/cevo/research/sars-cov-2/real-time-monitoring-in-switzerland.html), and an example for advanced visualization [here](https://opensource.com/article/20/4/python-data-covid-19). Feel free to share your conclusions and opinions on it on the forum."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Final considerations"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- You now know how to implement LSTM networks using Keras: this allows you to tackle several Natural Language Processing tasks, which are highly marketable at this time.\n",
    "- You should now have a deeper understanding of convolutions, especially on how things that appear small and easy (such as padding and striding) can lead to quite complex changes of behavior. For example, can we apply \"same\" padding with an filter of an even shape (e.g. 4 x 4, 6 x 6 etc.)? Would it be possible to pad the input such that, using a stride > 1, we get a matrix with the same shape as the input? This reasoning is important because these \"sizes\" in the network are hyperparameters, which means that you are responsible to set them correctly."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}