28-Transfer_learning_feature_extraction.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"http://akhavanpour.ir/notebook/images/srttu.gif\" alt=\"SRTTU\" style=\"width: 150px;\"/>\n",
    "\n",
    "[![Azure Notebooks](https://notebooks.azure.com/launch.png)](https://notebooks.azure.com/import/gh/Alireza-Akhavan/class.vision)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Transfer Learning - Part 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "<div style=\"direction:rtl;text-align:right;font-family:Tahoma\">\n",
    "در این نوت‌بوک رویکرد نسخت Transfer Learning یعنی استفاده به عنوان یک Feature extractor بررسی خواهد شد.\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'2.1.6'"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import keras\n",
    "keras.__version__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div style=\"direction:rtl;text-align:right;font-family:Tahoma\">\n",
    "کدها  برگرفته از فصل پنج کتاب\n",
    "[Deep Learning with Python](https://www.manning.com/books/deep-learning-with-python?a_aid=keras&a_bid=76564dff)\n",
    "و گیت هاب نویسنده کتاب و توسعه دهنده کراس \n",
    "[François Chollet](http://nbviewer.jupyter.org/github/fchollet/deep-learning-with-python-notebooks/blob/master/5.3-using-a-pretrained-convnet.ipynb)\n",
    "است.\n",
    "</div>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Using a pre-trained convnet\n",
    "\n",
    "----\n",
    "\n",
    "A common and highly effective approach to deep learning on small image datasets is to leverage a pre-trained network. A pre-trained network \n",
    "is simply a saved network previously trained on a large dataset, typically on a large-scale image classification task. If this original \n",
    "dataset is large enough and general enough, then the spatial feature hierarchy learned by the pre-trained network can effectively act as a \n",
    "generic model of our visual world, and hence its features can prove useful for many different computer vision problems, even though these \n",
    "new problems might involve completely different classes from those of the original task. For instance, one might train a network on \n",
    "ImageNet (where classes are mostly animals and everyday objects) and then re-purpose this trained network for something as remote as \n",
    "identifying furniture items in images. Such portability of learned features across different problems is a key advantage of deep learning \n",
    "compared to many older shallow learning approaches, and it makes deep learning very effective for small-data problems.\n",
    "\n",
    "In our case, we will consider a large convnet trained on the ImageNet dataset (1.4 million labeled images and 1000 different classes). \n",
    "ImageNet contains many animal classes, including different species of cats and dogs, and we can thus expect to perform very well on our cat \n",
    "vs. dog classification problem.\n",
    "\n",
    "We will use the VGG16 architecture, developed by Karen Simonyan and Andrew Zisserman in 2014, a simple and widely used convnet architecture \n",
    "for ImageNet. Although it is a bit of an older model, far from the current state of the art and somewhat heavier than many other recent \n",
    "models, we chose it because its architecture is similar to what you are already familiar with, and easy to understand without introducing \n",
    "any new concepts. This may be your first encounter with one of these cutesie model names -- VGG, ResNet, Inception, Inception-ResNet, \n",
    "Xception... you will get used to them, as they will come up frequently if you keep doing deep learning for computer vision.\n",
    "\n",
    "There are two ways to leverage a pre-trained network: *feature extraction* and *fine-tuning*. We will cover both of them. Let's start with \n",
    "feature extraction."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Feature extraction\n",
    "\n",
    "Feature extraction consists of using the representations learned by a previous network to extract interesting features from new samples. \n",
    "These features are then run through a new classifier, which is trained from scratch.\n",
    "\n",
    "As we saw previously, convnets used for image classification comprise two parts: they start with a series of pooling and convolution \n",
    "layers, and they end with a densely-connected classifier. The first part is called the \"convolutional base\" of the model. In the case of \n",
    "convnets, \"feature extraction\" will simply consist of taking the convolutional base of a previously-trained network, running the new data \n",
    "through it, and training a new classifier on top of the output.\n",
    "\n",
    "![swapping FC classifiers](https://s3.amazonaws.com/book.keras.io/img/ch5/swapping_fc_classifier.png)\n",
    "\n",
    "Why only reuse the convolutional base? Could we reuse the densely-connected classifier as well? In general, it should be avoided. The \n",
    "reason is simply that the representations learned by the convolutional base are likely to be more generic and therefore more reusable: the \n",
    "feature maps of a convnet are presence maps of generic concepts over a picture, which is likely to be useful regardless of the computer \n",
    "vision problem at hand. On the other end, the representations learned by the classifier will necessarily be very specific to the set of \n",
    "classes that the model was trained on -- they will only contain information about the presence probability of this or that class in the \n",
    "entire picture. Additionally, representations found in densely-connected layers no longer contain any information about _where_ objects are \n",
    "located in the input image: these layers get rid of the notion of space, whereas the object location is still described by convolutional \n",
    "feature maps. For problems where object location matters, densely-connected features would be largely useless.\n",
    "\n",
    "Note that the level of generality (and therefore reusability) of the representations extracted by specific convolution layers depends on \n",
    "the depth of the layer in the model. Layers that come earlier in the model extract local, highly generic feature maps (such as visual \n",
    "edges, colors, and textures), while layers higher-up extract more abstract concepts (such as \"cat ear\" or \"dog eye\"). So if your new \n",
    "dataset differs a lot from the dataset that the original model was trained on, you may be better off using only the first few layers of the \n",
    "model to do feature extraction, rather than using the entire convolutional base.\n",
    "\n",
    "In our case, since the ImageNet class set did contain multiple dog and cat classes, it is likely that it would be beneficial to reuse the \n",
    "information contained in the densely-connected layers of the original model. However, we will chose not to, in order to cover the more \n",
    "general case where the class set of the new problem does not overlap with the class set of the original model."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's put this in practice by using the convolutional base of the VGG16 network, trained on ImageNet, to extract interesting features from \n",
    "our cat and dog images, and then training a cat vs. dog classifier on top of these features.\n",
    "\n",
    "The VGG16 model, among others, comes pre-packaged with Keras. You can import it from the `keras.applications` module. Here's the list of \n",
    "image classification models (all pre-trained on the ImageNet dataset) that are available as part of `keras.applications`:\n",
    "\n",
    "* Xception\n",
    "* InceptionV3\n",
    "* ResNet50\n",
    "* VGG16\n",
    "* VGG19\n",
    "* MobileNet\n",
    "\n",
    "Let's instantiate the VGG16 model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from keras.applications import VGG16\n",
    "\n",
    "conv_base = VGG16(weights='imagenet',\n",
    "                  include_top=False,\n",
    "                  input_shape=(150, 150, 3))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We passed three arguments to the constructor:\n",
    "\n",
    "* `weights`, to specify which weight checkpoint to initialize the model from\n",
    "* `include_top`, which refers to including or not the densely-connected classifier on top of the network. By default, this \n",
    "densely-connected classifier would correspond to the 1000 classes from ImageNet. Since we intend to use our own densely-connected \n",
    "classifier (with only two classes, cat and dog), we don't need to include it.\n",
    "* `input_shape`, the shape of the image tensors that we will feed to the network. This argument is purely optional: if we don't pass it, \n",
    "then the network will be able to process inputs of any size.\n",
    "\n",
    "Here's the detail of the architecture of the VGG16 convolutional base: it's very similar to the simple convnets that you are already \n",
    "familiar with."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "input_1 (InputLayer)         (None, 150, 150, 3)       0         \n",
      "_________________________________________________________________\n",
      "block1_conv1 (Conv2D)        (None, 150, 150, 64)      1792      \n",
      "_________________________________________________________________\n",
      "block1_conv2 (Conv2D)        (None, 150, 150, 64)      36928     \n",
      "_________________________________________________________________\n",
      "block1_pool (MaxPooling2D)   (None, 75, 75, 64)        0         \n",
      "_________________________________________________________________\n",
      "block2_conv1 (Conv2D)        (None, 75, 75, 128)       73856     \n",
      "_________________________________________________________________\n",
      "block2_conv2 (Conv2D)        (None, 75, 75, 128)       147584    \n",
      "_________________________________________________________________\n",
      "block2_pool (MaxPooling2D)   (None, 37, 37, 128)       0         \n",
      "_________________________________________________________________\n",
      "block3_conv1 (Conv2D)        (None, 37, 37, 256)       295168    \n",
      "_________________________________________________________________\n",
      "block3_conv2 (Conv2D)        (None, 37, 37, 256)       590080    \n",
      "_________________________________________________________________\n",
      "block3_conv3 (Conv2D)        (None, 37, 37, 256)       590080    \n",
      "_________________________________________________________________\n",
      "block3_pool (MaxPooling2D)   (None, 18, 18, 256)       0         \n",
      "_________________________________________________________________\n",
      "block4_conv1 (Conv2D)        (None, 18, 18, 512)       1180160   \n",
      "_________________________________________________________________\n",
      "block4_conv2 (Conv2D)        (None, 18, 18, 512)       2359808   \n",
      "_________________________________________________________________\n",
      "block4_conv3 (Conv2D)        (None, 18, 18, 512)       2359808   \n",
      "_________________________________________________________________\n",
      "block4_pool (MaxPooling2D)   (None, 9, 9, 512)         0         \n",
      "_________________________________________________________________\n",
      "block5_conv1 (Conv2D)        (None, 9, 9, 512)         2359808   \n",
      "_________________________________________________________________\n",
      "block5_conv2 (Conv2D)        (None, 9, 9, 512)         2359808   \n",
      "_________________________________________________________________\n",
      "block5_conv3 (Conv2D)        (None, 9, 9, 512)         2359808   \n",
      "_________________________________________________________________\n",
      "block5_pool (MaxPooling2D)   (None, 4, 4, 512)         0         \n",
      "=================================================================\n",
      "Total params: 14,714,688\n",
      "Trainable params: 14,714,688\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "conv_base.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The final feature map has shape `(4, 4, 512)`. That's the feature on top of which we will stick a densely-connected classifier.\n",
    "\n",
    "At this point, there are two ways we could proceed: \n",
    "\n",
    "* Running the convolutional base over our dataset, recording its output to a Numpy array on disk, then using this data as input to a \n",
    "standalone densely-connected classifier similar to those you have seen in the first chapters of this book. This solution is very fast and \n",
    "cheap to run, because it only requires running the convolutional base once for every input image, and the convolutional base is by far the \n",
    "most expensive part of the pipeline. However, for the exact same reason, this technique would not allow us to leverage data augmentation at \n",
    "all.\n",
    "* Extending the model we have (`conv_base`) by adding `Dense` layers on top, and running the whole thing end-to-end on the input data. This \n",
    "allows us to use data augmentation, because every input image is going through the convolutional base every time it is seen by the model. \n",
    "However, for this same reason, this technique is far more expensive than the first one.\n",
    "\n",
    "We will cover both techniques. Let's walk through the code required to set-up the first one: recording the output of `conv_base` on our \n",
    "data and using these outputs as inputs to a new model.\n",
    "\n",
    "We will start by simply running instances of the previously-introduced `ImageDataGenerator` to extract images as Numpy arrays as well as \n",
    "their labels. We will extract features from these images simply by calling the `predict` method of the `conv_base` model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found 2000 images belonging to 2 classes.\n",
      "Found 1000 images belonging to 2 classes.\n",
      "Found 1000 images belonging to 2 classes.\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "import numpy as np\n",
    "from keras.preprocessing.image import ImageDataGenerator\n",
    "\n",
    "base_dir = 'E:/dataset/catDog/catVsdog'\n",
    "\n",
    "train_dir = os.path.join(base_dir, 'train')\n",
    "validation_dir = os.path.join(base_dir, 'validation')\n",
    "test_dir = os.path.join(base_dir, 'test')\n",
    "\n",
    "datagen = ImageDataGenerator(rescale=1./255)\n",
    "batch_size = 20\n",
    "\n",
    "def extract_features(directory, sample_count):\n",
    "    features = np.zeros(shape=(sample_count, 4, 4, 512))\n",
    "    labels = np.zeros(shape=(sample_count))\n",
    "    generator = datagen.flow_from_directory(\n",
    "        directory,\n",
    "        target_size=(150, 150),\n",
    "        batch_size=batch_size,\n",
    "        class_mode='binary')\n",
    "    i = 0\n",
    "    for inputs_batch, labels_batch in generator:\n",
    "        features_batch = conv_base.predict(inputs_batch)\n",
    "        features[i * batch_size : (i + 1) * batch_size] = features_batch\n",
    "        labels[i * batch_size : (i + 1) * batch_size] = labels_batch\n",
    "        i += 1\n",
    "        if i * batch_size >= sample_count:\n",
    "            # Note that since generators yield data indefinitely in a loop,\n",
    "            # we must `break` after every image has been seen once.\n",
    "            break\n",
    "    return features, labels\n",
    "\n",
    "train_features, train_labels = extract_features(train_dir, 2000)\n",
    "validation_features, validation_labels = extract_features(validation_dir, 1000)\n",
    "test_features, test_labels = extract_features(test_dir, 1000)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The extracted features are currently of shape `(samples, 4, 4, 512)`. We will feed them to a densely-connected classifier, so first we must \n",
    "flatten them to `(samples, 8192)`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "train_features = np.reshape(train_features, (2000, 4 * 4 * 512))\n",
    "validation_features = np.reshape(validation_features, (1000, 4 * 4 * 512))\n",
    "test_features = np.reshape(test_features, (1000, 4 * 4 * 512))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "At this point, we can define our densely-connected classifier (note the use of dropout for regularization), and train it on the data and \n",
    "labels that we just recorded:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 2000 samples, validate on 1000 samples\n",
      "Epoch 1/30\n",
      "2000/2000 [==============================] - 7s 3ms/step - loss: 0.5870 - acc: 0.6915 - val_loss: 0.4264 - val_acc: 0.8390\n",
      "Epoch 2/30\n",
      "2000/2000 [==============================] - 6s 3ms/step - loss: 0.4143 - acc: 0.8025 - val_loss: 0.3636 - val_acc: 0.8510\n",
      "Epoch 3/30\n",
      "2000/2000 [==============================] - 6s 3ms/step - loss: 0.3546 - acc: 0.8470 - val_loss: 0.3194 - val_acc: 0.8890\n",
      "Epoch 4/30\n",
      "2000/2000 [==============================] - 5s 3ms/step - loss: 0.3211 - acc: 0.8630 - val_loss: 0.2976 - val_acc: 0.8920\n",
      "Epoch 5/30\n",
      "2000/2000 [==============================] - 6s 3ms/step - loss: 0.2848 - acc: 0.8825 - val_loss: 0.2869 - val_acc: 0.8910\n",
      "Epoch 6/30\n",
      "2000/2000 [==============================] - 6s 3ms/step - loss: 0.2650 - acc: 0.8965 - val_loss: 0.2785 - val_acc: 0.8900\n",
      "Epoch 7/30\n",
      "2000/2000 [==============================] - 6s 3ms/step - loss: 0.2425 - acc: 0.9110 - val_loss: 0.2660 - val_acc: 0.8990\n",
      "Epoch 8/30\n",
      "2000/2000 [==============================] - 6s 3ms/step - loss: 0.2315 - acc: 0.9170 - val_loss: 0.2619 - val_acc: 0.8970\n",
      "Epoch 9/30\n",
      "2000/2000 [==============================] - 6s 3ms/step - loss: 0.2204 - acc: 0.9135 - val_loss: 0.2563 - val_acc: 0.9030\n",
      "Epoch 10/30\n",
      "2000/2000 [==============================] - 6s 3ms/step - loss: 0.2036 - acc: 0.9205 - val_loss: 0.2631 - val_acc: 0.8930\n",
      "Epoch 11/30\n",
      "2000/2000 [==============================] - 6s 3ms/step - loss: 0.1943 - acc: 0.9285 - val_loss: 0.2455 - val_acc: 0.9030\n",
      "Epoch 12/30\n",
      "2000/2000 [==============================] - 6s 3ms/step - loss: 0.1874 - acc: 0.9335 - val_loss: 0.2489 - val_acc: 0.9020\n",
      "Epoch 13/30\n",
      "2000/2000 [==============================] - 6s 3ms/step - loss: 0.1808 - acc: 0.9355 - val_loss: 0.2447 - val_acc: 0.8980\n",
      "Epoch 14/30\n",
      "2000/2000 [==============================] - 5s 3ms/step - loss: 0.1730 - acc: 0.9385 - val_loss: 0.2504 - val_acc: 0.9000\n",
      "Epoch 15/30\n",
      "2000/2000 [==============================] - 5s 3ms/step - loss: 0.1646 - acc: 0.9390 - val_loss: 0.2412 - val_acc: 0.9050\n",
      "Epoch 16/30\n",
      "2000/2000 [==============================] - 5s 3ms/step - loss: 0.1527 - acc: 0.9505 - val_loss: 0.2382 - val_acc: 0.9050\n",
      "Epoch 17/30\n",
      "2000/2000 [==============================] - 5s 3ms/step - loss: 0.1530 - acc: 0.9470 - val_loss: 0.2386 - val_acc: 0.9030\n",
      "Epoch 18/30\n",
      "2000/2000 [==============================] - 5s 3ms/step - loss: 0.1420 - acc: 0.9495 - val_loss: 0.2382 - val_acc: 0.9020\n",
      "Epoch 19/30\n",
      "2000/2000 [==============================] - 5s 3ms/step - loss: 0.1383 - acc: 0.9525 - val_loss: 0.2485 - val_acc: 0.9010\n",
      "Epoch 20/30\n",
      "2000/2000 [==============================] - 5s 3ms/step - loss: 0.1330 - acc: 0.9525 - val_loss: 0.2373 - val_acc: 0.9040\n",
      "Epoch 21/30\n",
      "2000/2000 [==============================] - 5s 3ms/step - loss: 0.1284 - acc: 0.9525 - val_loss: 0.2396 - val_acc: 0.9000\n",
      "Epoch 22/30\n",
      "2000/2000 [==============================] - 5s 3ms/step - loss: 0.1207 - acc: 0.9580 - val_loss: 0.2498 - val_acc: 0.9000\n",
      "Epoch 23/30\n",
      "2000/2000 [==============================] - 5s 3ms/step - loss: 0.1165 - acc: 0.9620 - val_loss: 0.2403 - val_acc: 0.8990\n",
      "Epoch 24/30\n",
      "2000/2000 [==============================] - 5s 3ms/step - loss: 0.1110 - acc: 0.9665 - val_loss: 0.2476 - val_acc: 0.8980\n",
      "Epoch 25/30\n",
      "2000/2000 [==============================] - 5s 3ms/step - loss: 0.1036 - acc: 0.9720 - val_loss: 0.2490 - val_acc: 0.8990\n",
      "Epoch 26/30\n",
      "2000/2000 [==============================] - 5s 3ms/step - loss: 0.1025 - acc: 0.9670 - val_loss: 0.2380 - val_acc: 0.9040\n",
      "Epoch 27/30\n",
      "2000/2000 [==============================] - 5s 3ms/step - loss: 0.0968 - acc: 0.9665 - val_loss: 0.2388 - val_acc: 0.9040\n",
      "Epoch 28/30\n",
      "2000/2000 [==============================] - 5s 3ms/step - loss: 0.0970 - acc: 0.9675 - val_loss: 0.2394 - val_acc: 0.9080\n",
      "Epoch 29/30\n",
      "2000/2000 [==============================] - 5s 3ms/step - loss: 0.0897 - acc: 0.9755 - val_loss: 0.2442 - val_acc: 0.8990\n",
      "Epoch 30/30\n",
      "2000/2000 [==============================] - 5s 3ms/step - loss: 0.0868 - acc: 0.9725 - val_loss: 0.2409 - val_acc: 0.8990\n"
     ]
    }
   ],
   "source": [
    "from keras import models\n",
    "from keras import layers\n",
    "from keras import optimizers\n",
    "\n",
    "model = models.Sequential()\n",
    "model.add(layers.Dense(256, activation='relu', input_dim=4 * 4 * 512))\n",
    "model.add(layers.Dropout(0.5))\n",
    "model.add(layers.Dense(1, activation='sigmoid'))\n",
    "\n",
    "model.compile(optimizer=optimizers.RMSprop(lr=2e-5),\n",
    "              loss='binary_crossentropy',\n",
    "              metrics=['acc'])\n",
    "\n",
    "history = model.fit(train_features, train_labels,\n",
    "                    epochs=30,\n",
    "                    batch_size=20,\n",
    "                    validation_data=(validation_features, validation_labels))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Training is very fast, since we only have to deal with two `Dense` layers -- an epoch takes less than one second even on CPU.\n",
    "\n",
    "Let's take a look at the loss and accuracy curves during training:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAEICAYAAACzliQjAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XucFNWd9/HPjxEQEBVhTJSBGTQoDncYMQZUNKJIVNRg\nBDERjSEm6rqabB5vjxI2arLGVXfjJsGsTxJFCNFoSKJrvCtJzDIoF8GACCiDCCMgt0G5/Z4/TjX0\n3KuHnunp7u/79epX1+VU1amumV+dOnXqlLk7IiKSH9pkOgMiItJyFPRFRPKIgr6ISB5R0BcRySMK\n+iIieURBX0Qkjyjo5yEzKzCzbWbWM51pM8nMPmdmaW9/bGZnmtmqpPGlZnZKnLRN2NYvzOyWpi4v\nEsdBmc6ANM7MtiWNdgQ+BfZE49909+mprM/d9wCHpDttPnD349OxHjO7CrjM3UcmrfuqdKxbpCEK\n+lnA3fcF3agkeZW7P19fejM7yN13t0TeRBqjv8fWRdU7OcDMfmBmvzGzGWa2FbjMzE42s9fN7GMz\nW2tm/2FmbaP0B5mZm1lJNP5oNP8ZM9tqZn8zs16ppo3mn2Nmy8xss5n9p5n9xcwm1ZPvOHn8ppkt\nN7NNZvYfScsWmNl9ZrbBzFYAoxv4fW41s5k1pj1oZv8eDV9lZm9H+/NuVAqvb10VZjYyGu5oZo9E\neVsMDK2R9jYzWxGtd7GZnR9N7w/8BDglqjr7KOm3nZK0/NXRvm8ws6fM7Kg4v00qv3MiP2b2vJlt\nNLMPzex7Sdv5v9FvssXMys3s6Lqq0sxsTuI4R7/nq9F2NgK3mVlvM3sp2sZH0e92WNLyxdE+Vkbz\nHzCzg6M8n5CU7igzqzKzrvXtrzTC3fXJog+wCjizxrQfADuB8wgn8g7AicBJhKu5Y4BlwLVR+oMA\nB0qi8UeBj4AyoC3wG+DRJqQ9EtgKjI3m3QjsAibVsy9x8vh74DCgBNiY2HfgWmAxUAR0BV4Nf851\nbucYYBvQKWnd64GyaPy8KI0BZwA7gAHRvDOBVUnrqgBGRsM/Bl4GugDFwJIaab8CHBUdk0ujPHwm\nmncV8HKNfD4KTImGz4ryOAg4GPgv4MU4v02Kv/NhwDrgeqA9cCgwLJp3M7AA6B3twyDgCOBzNX9r\nYE7iOEf7thv4FlBA+Hs8Dvgi0C76O/kL8OOk/Xkr+j07RemHR/OmAXcmbec7wJOZ/j/M5k/GM6BP\niges/qD/YiPLfRf4bTRcVyD/WVLa84G3mpD2SuC1pHkGrKWeoB8zj59Pmv874LvR8KuEaq7EvDE1\nA1GNdb8OXBoNnwMsbSDtH4FrouGGgv77yccC+HZy2jrW+xbwpWi4saD/K+CupHmHEu7jFDX226T4\nO38VmFtPuncT+a0xPU7QX9FIHsYltgucAnwIFNSRbjiwErBofD5wUbr/r/Lpo+qd3LE6ecTM+pjZ\nn6LL9S3AVKBbA8t/mDRcRcM3b+tLe3RyPjz8l1bUt5KYeYy1LeC9BvIL8BgwIRq+NBpP5ONcM/t7\nVPXwMaGU3dBvlXBUQ3kws0lmtiCqovgY6BNzvRD2b9/63H0LsAnonpQm1jFr5HfuQQjudWloXmNq\n/j1+1sxmmdmaKA+/rJGHVR4aDVTj7n8hXDWMMLN+QE/gT03Mk6A6/VxSs7nizwkly8+5+6HA7YSS\nd3NaSyiJAmBmRvUgVdOB5HEtIVgkNNakdBZwppl1J1Q/PRblsQPwOHA3oerlcODPMfPxYX15MLNj\ngJ8Sqji6Ruv9R9J6G2te+gGhyiixvs6EaqQ1MfJVU0O/82rg2HqWq2/e9ihPHZOmfbZGmpr79yNC\nq7P+UR4m1chDsZkV1JOPXwOXEa5KZrn7p/WkkxgU9HNXZ2AzsD26EfbNFtjmH4EhZnaemR1EqCcu\nbKY8zgL+2cy6Rzf1/k9Did39Q0IVxC8JVTvvRLPaE+qZK4E9ZnYuoe45bh5uMbPDLTzHcG3SvEMI\nga+ScP77BqGkn7AOKEq+oVrDDODrZjbAzNoTTkqvuXu9V04NaOh3ng30NLNrzay9mR1qZsOieb8A\nfmBmx1owyMyOIJzsPiQ0GCgws8kknaAayMN2YLOZ9SBUMSX8DdgA3GXh5ngHMxueNP8RQnXQpYQT\ngBwABf3c9R3gcsKN1Z8Tbrg2K3dfB1wC/Dvhn/hY4E1CCS/defwp8AKwCJhLKK035jFCHf2+qh13\n/xi4AXiScDN0HOHkFccdhCuOVcAzJAUkd18I/Cfwv1Ga44G/Jy37HPAOsM7MkqtpEsv/D6Ea5slo\n+Z7AxJj5qqne39ndNwOjgC8TTkTLgNOi2fcATxF+5y2Em6oHR9V23wBuIdzU/1yNfavLHcAwwsln\nNvBEUh52A+cCJxBK/e8TjkNi/irCcf7U3f+a4r5LDYmbIyJpF12ufwCMc/fXMp0fyV5m9mvCzeEp\nmc5LttPDWZJWZjaa0FJmB6HJ3y5CaVekSaL7I2OB/pnOSy5Q9Y6k2whgBaEu+2zgQt14k6Yys7sJ\nzwrc5e7vZzo/uUDVOyIieUQlfRGRPNLq6vS7devmJSUlmc6GiEhWmTdv3kfu3lATaaAVBv2SkhLK\ny8sznQ0RkaxiZo09lQ6oekdEJK8o6IuI5BEFfRGRPNLq6vTrsmvXLioqKvjkk08ynRVpwMEHH0xR\nURFt29bXnYyIZFpWBP2Kigo6d+5MSUkJoeNGaW3cnQ0bNlBRUUGvXr0aX0BEMiIrqnc++eQTunbt\nqoDfipkZXbt21dWYSJLp06GkBNq0Cd/Tp2c6R1kS9AEF/CygYySy3/TpMHkyvPceuIfvyZPrDvwt\neXLImqAvIpKq5gimcdd5661QVVV9WlVVmF5zfXFPDumgoB/Dhg0bGDRoEIMGDeKzn/0s3bt33ze+\nc+fOWOu44oorWLp0aYNpHnzwQaa3hus/kRzQHME0lXW+X0/3cDWnxz05pE2mX9Jb8zN06FCvacmS\nJbWmNeTRR92Li93Nwvejj6a0eIPuuOMOv+eee2pN37t3r+/Zsyd9G8pSqR4rkYRU/m/jpC0udg+h\nufqnuLjpeUxlnXHTmtWdziy1vAHlno8vRm/JS6Xly5dTWlrKxIkT6du3L2vXrmXy5MmUlZXRt29f\npk6dui/tiBEjmD9/Prt37+bwww/npptuYuDAgZx88smsX78egNtuu437779/X/qbbrqJYcOGcfzx\nx/PXv4YXBm3fvp0vf/nLlJaWMm7cOMrKypg/f36tvN1xxx2ceOKJ9OvXj6uvvhqPelNdtmwZZ5xx\nBgMHDmTIkCGsWrUKgLvuuov+/fszcOBAbm22IoZI3VKt/46TNm5JOxWprPPOO6Fjx+rTOnYM05P1\nrOftzvVNP2Bxzgwt+TnQkn5znN2TJZf033nnHTcznzt37r75GzZscHf3Xbt2+YgRI3zx4sXu7j58\n+HB/8803fdeuXQ74008/7e7uN9xwg999993u7n7rrbf6fffdty/99773PXd3//3vf+9nn322u7vf\nfffd/u1vf9vd3efPn+9t2rTxN998s1Y+E/nYu3evjx8/ft/2hgwZ4rNnz3Z39x07dvj27dt99uzZ\nPmLECK+qqqq2bFOopC9N0Rwl6FRjQXNcPcRZ56OPunfsWH19HTumXkNBvpb0m+Ps3pBjjz2WsrKy\nfeMzZsxgyJAhDBkyhLfffpslS5bUWqZDhw6cc845AAwdOnRfabumiy66qFaaOXPmMH78eAAGDhxI\n375961z2hRdeYNiwYQwcOJBXXnmFxYsXs2nTJj766CPOO+88IDxM1bFjR55//nmuvPJKOnToAMAR\nRxyR+g8hcgBS+b+NmzZuSRviXz2ksk6AiRNh1SrYuzd8T6zjLccTJ8K0aVBcDGbhe9q0utOmQ84F\n/Za+VOrUqdO+4XfeeYcHHniAF198kYULFzJ69Og62623a9du33BBQQG7d++uc93t27dvNE1dqqqq\nuPbaa3nyySdZuHAhV155pdrPS6uWyv9t3LSpBNO4N1ObK0DHOTmkS84F/VTPxOm0ZcsWOnfuzKGH\nHsratWt59tln076N4cOHM2vWLAAWLVpU55XEjh07aNOmDd26dWPr1q088cQTAHTp0oXCwkL+8Ic/\nAOGht6qqKkaNGsXDDz/Mjh07ANi4cWPa8y25Jd1NIVP5v00lbdxgmsqVRksG6OaQc0G/pS+Vkg0Z\nMoTS0lL69OnD1772NYYPH572bVx33XWsWbOG0tJSvv/971NaWsphhx1WLU3Xrl25/PLLKS0t5Zxz\nzuGkk07aN2/69Once++9DBgwgBEjRlBZWcm5557L6NGjKSsrY9CgQdx3331pz7dkhzjBPNXGEnHW\nmcr/bXP8j7f4zdRMilPx35KfdDTZzGW7du3yHTt2uLv7smXLvKSkxHft2pXhXO2nY9X6xG0KGfeG\nYio3M9N1k7K5ZUs+G0LMG7mxAjEwGlgKLAduqmN+MfACsBB4GShKmrcHmB99Zje2LQX9hm3atMmH\nDBniAwYM8P79+/uzzz6b6SxVo2PVuqQSzJqjXXlzt6ZLp+Z8vqclpC3oAwXAu8AxQDtgAVBaI81v\ngcuj4TOAR5LmbYuTkcRHQT+76Vi1nHQ3MYwbzJtjnXLg4gb9OHX6w4Dl7r7C3XcCM4GxNdKUAi9G\nwy/VMV9EYoh7g7Q5HlCKW6+dyo3UvKorzxJxgn53YHXSeEU0LdkC4KJo+EKgs5l1jcYPNrNyM3vd\nzC6oawNmNjlKU15ZWZlC9kVyRyo3SOM2MUwl6MYN5qncSM1kazqpR2OXAsA44BdJ418FflIjzdHA\n74A3gQcIJ4bDo3ndo+9jgFXAsQ1tT9U72U3Hqumao9ok1RuUzVGvne115dmCNFbvrAF6JI0XRdOS\nTxwfuPtF7j4YuDWa9nH0vSb6XkG4yTs4pbOSSA6IU23THFUxqTZvbI426Nnerj3XxAn6c4HeZtbL\nzNoB44HZyQnMrJuZJdZ1M/BwNL2LmbVPpAGGA7WfJmrlTj/99FoPWt1///1861vfanC5Qw45BIAP\nPviAcePG1Zlm5MiRlJeXN7ie+++/n6qka/kxY8bw8ccfx8m6NKN01783R1UMKOhKDXEuB4AxwDJC\nK55bo2lTgfN9fxXQO1GaXwDto+lfABYR6vwXAV9vbFutsXrn5z//uU+aNKnatJNOOslfeeWVBpfr\n1KlTo+s+7bTTqnXYVpfi4mKvrKxsPKOtQKaPVTqku5OsuNU2raEqRrIX6Wyn35Kf1hj0N2zY4IWF\nhf7pp5+6u/vKlSu9R48evnfvXt+6daufccYZPnjwYO/Xr58/9dRT+5ZLBP2VK1d637593d29qqrK\nL7nkEu/Tp49fcMEFPmzYsH1B/+qrr/ahQ4d6aWmp33777e7u/sADD3jbtm29X79+PnLkSHevfhK4\n9957vW/fvt63b999PXSuXLnS+/Tp41dddZWXlpb6qFGj9vWgmWz27Nk+bNgwHzRokH/xi1/0Dz/8\n0N3dt27d6pMmTfJ+/fp5//79/fHHH3d392eeecYHDx7sAwYM8DPOOKPO3yrTx+pANccDSqk0W1Qg\nl6bK2aB//fXup52W3s/11zf+g37pS1/aF9Dvvvtu/853vuPu4QnZzZs3u7t7ZWWlH3vssb537153\nrzvo33vvvX7FFVe4u/uCBQu8oKBgX9BPdGm8e/duP+2003zBggXuXruknxgvLy/3fv36+bZt23zr\n1q1eWlrqb7zxhq9cudILCgr2dbl88cUX+yOPPFJrnzZu3Lgvrw899JDfeOON7u7+ve99z69P+lE2\nbtzo69ev96KiIl+xYkW1vNaU7UE/3x9QkuwVN+jnXN87zWXChAnMnDkTgJkzZzJhwgQgnDRvueUW\nBgwYwJlnnsmaNWtYt25dvet59dVXueyyywAYMGAAAwYM2Ddv1qxZDBkyhMGDB7N48eI6O1NLNmfO\nHC688EI6derEIYccwkUXXcRrr70GQK9evRg0aBBQf/fNFRUVnH322fTv35977rmHxYsXA/D8889z\nzTXX7EvXpUsXXn/9dU499VR69eoF5G73y3FvpjZX/btIczso0xlIVfRiqRY3duxYbrjhBt544w2q\nqqoYOnQoEDowq6ysZN68ebRt25aSkpImdWO8cuVKfvzjHzN37ly6dOnCpEmTDqg75ES3zBC6Zk70\noJnsuuuu48Ybb+T888/n5ZdfZsqUKU3eXms3fXpow/7++yEw33ln3Tc0e/YMN1rrmp7szjvDzdjk\ntvIN3UiFeNsXaW4q6cd0yCGHcPrpp3PllVfuK+UDbN68mSOPPJK2bdvy0ksv8V5dESPJqaeeymOP\nPQbAW2+9xcKFC4HQLXOnTp047LDDWLduHc8888y+ZTp37szWrVtrreuUU07hqaeeoqqqiu3bt/Pk\nk09yyimnxN6nzZs30717eM7uV7/61b7po0aN4sEHH9w3vmnTJj7/+c/z6quvsnLlSiC7ul9O5aGn\n5nhAKZFeLWikNVDQT8GECRNYsGBBtaA/ceJEysvL6d+/P7/+9a/p06dPg+v41re+xbZt2zjhhBO4\n/fbb910xDBw4kMGDB9OnTx8uvfTSat0yT548mdGjR3P66adXW9eQIUOYNGkSw4YN46STTuKqq65i\n8OD4j0FMmTKFiy++mKFDh9KtW7d902+77TY2bdpEv379GDhwIC+99BKFhYVMmzaNiy66iIEDB3LJ\nJZfE3k6mxX16FVLv4leBXLJOnIr/lvy0xtY7El86jlUqLVjipFWnX5IPiHkjN+vq9CW3JapiEiXz\nRFUM1C5Jx00bt55eJB+oekdaTJwnWFOpiombVq1nRPbLmqAfrl6kNWvoGDVHV8Bx02byFZoirY21\ntmBaVlbmNfuiWblyJZ07d6Zr166YWYZyJnXZsAHWrIGdOx2zDezYsZURI3rVSldSUncVS3FxuAma\narpU04rkOjOb5+5ljaXLijr9oqIiKioqUF/7rcv27SHou4cWLMuXH8yPflTEv/1b7VJ03FJ5Ku3f\nU0krIkFWBP22bdvuexJUWo/6Stq33lo76Me9mZrKg0x66EkkdVlRvSOtU5s2oZRfk1ko+Ser2dIG\nQqlcdesi6RG3eidrbuRK65NK/zO6mSrSOijoS5Ol2hRST7CKZJ6CvjSZSu8i2ScrbuRK6zVxooK8\nSDZRSV/qFPf9ryKSXVTSl1pS6f9GRLKLSvo5IN2l8lT6vxGR7KKgn+VSeUFI3JNDKv3fiEh2iRX0\nzWy0mS01s+VmdlMd84vN7AUzW2hmL5tZUdK8y83snehzeTozL/FL5amcHFJpfy8i2aXRJ3LNrABY\nBowCKoC5wAR3X5KU5rfAH939V2Z2BnCFu3/VzI4AyoEywIF5wFB331Tf9vREbmriPhWbSudkeno2\nu+3ZA8uXw1tvwaJF+7+3b4cePcLJu2fP6sM9e0LXruHvJlPcYf58WLIENm/e/9mypfp44rN1a91/\n+3Xp0gX69YP+/fd/H3cctGt34PnetSt0Orh6dbgaTv6sXg0ffBCOSRxlZfDcc03LRzo7XBsGLHf3\nFdGKZwJjgSVJaUqBG6Phl4CnouGzgefcfWO07HPAaGBGnJ2QxsXt0yaVKhv1aZNeO3fC738Ps2aF\n8cMO2/859NDq44lPhw7xAvDOnbB0afUAv2QJfPJJmG8Gn/tcCHSHHRaC0Jtvhvx8+mn1dXXoEE4E\nRUXxg2FhIXzhCzBiBJSWhkJIKrZsCUHu6afhmWdg7drq89u2rf17HXNMGO7cGQoK4m1n3brw2zzz\nDOzevX/dxx9f/URw/PFhfl0nmZonn/Xrw//HBx/UPvl07Rr+b3r1Cr9N27bx8llcHC/dgYgT9LsD\nq5PGK4CTaqRZAFwEPABcCHQ2s671LNu9ybnNI9Onxwu6cXuaTPXtUWp/f+BWrYKHHoL//u8QdI4+\nOgStRPDYvj292zv66BC8rrlmfxA74YTaT01DCFKVlbVLp6tXh1Lrtm3xtvnmm/DII2H48MPDCWD4\n8BDoTjwxnEhqbvftt0OQf/ppeO21EGQPPxzOPhvGjIGTTgrjhx4KBx+c3quPTz+tfZL8619hRoxi\naJs21U/ShYVw1lm1r5iKiqBTp/TlOd3S1WTzu8BPzGwS8CqwBoh5QQNmNhmYDNBTFccpNZmMWypv\nDd0Qb90KCxaEf5yePcM/Tqr/0Il/2uRqi3fege7dq5fY+vYNJcGWtmdPCGY/+1koVZrBuefC1VeH\nAJFcMt29u/6qi0RJvTFt2uwvyXftGj+fZnDkkeEzdGhq+5jMHVasgDlz4C9/Cd9PPx3mtW0b1j18\nOAwYAH//e5iXqE7s3x+++90Q6E8+GQ5qgQbk7duHvAwYUH36li2weDEsWxZONHVdiR1ySGarv9Il\nTp3+ycAUdz87Gr8ZwN3vrif9IcA/3L3IzCYAI939m9G8nwMvu3u951XV6Tffy0GmT4dbbgknh+Li\n5q+yWbOmejBYsKD6fYbOnRuuY3bfH9gTQX7ZsuqX5336QO/eYVtvvVW99FxSUv1E0K8ffOYz9Qfa\n5Onbt4eSXM08ffazdVdhrF0bSvTTpoXS8lFHwVVXhU++lWM2bAil58Rxnzs3VEN17AhnnhmC/Jgx\n4beV9Elnnf5coLeZ9SKU4McDl9bYWDdgo7vvBW4GHo5mPQvcZWZdovGzovnSgHQ3mXSHhQtD4DQL\npZeRI6FbtxBA01HC2rs3lJSSg3zixNWxI3z+83DbbTBsWLjaqFmtMG9eqG6oT69eIXBfcEH47t8/\nBPvkuudER241b2Am1+M2pkOHUKrr1ClUydSs5mjbNlxVJJ8Ili4NdeS7d8OoUXD//XDeefHrcXNN\n165h/887L4x/8kk4WR93XChFS2bF6k/fzMYA9wMFwMPufqeZTQXK3X22mY0D7ia00HkVuMbdP42W\nvRK4JVrVne7+/xralkr66Svpv/tuqKt87LFQj1pQEILSkUfCU0+Fku2RR8LFF8OECeESO+6NuKqq\nUIJLBPm//jWUkCGUhkeMCJ/hw2HgwHgBcMcOqKjYfyLYuzeUzvv2DZfWTZW42bloUSiF1nXjNHEZ\nn5xP97BPddV7J4bXrAnLXXFFqD7r3bvp+RQ5EHFL+nqJSit0IE0mP/ggtBJ57LEQlAFOOSUE9XHj\nQpUFhNLXM8+Ek8If/hDGi4th/PiQdsCA6vWX69eH4J4oxc+bt7/0XFq6/+bdiBGhVJ4LdZ9xJJri\nxW1FItJcFPSzXF2tdy69NJwI6qqT/vBDePJJePnlUEIdPDgE70suabxOecuWUD0xYwb8+c8hkJWW\nwoUXhpPInDnhZimEG2Ennri/FP+FL8ARRzT7zyEijVDQz2J79sArr4QgXF5e/UZjQ3XTvXuHQD9h\nQrjB2RSVlfD442Hbr70W6mcTpfjhw0NrjPbtm7ZuEWk+CvpZxj1Ux8yYAb/5TWgN0qkTnHpqKEnX\nVQ+dXBfdpUu4wZjOapWtW3OnmZpIrktn6x1pRkuWhEA/Y0a48dquXWjONmFCaN9d14M1LSUT7dxF\npHkp6KfJ3r2h5UljF05PPQU/+lEoybdtG/rtaNMGzjgjtKG/6KLwNKKISHNQ0E+DzZth7NhQD5+K\nXbtC4L/vvvDovIhIc1PQP0Dr18Po0aEN+F13hTbq9fmXfwntxJPt2gX33KOgLyItQ0H/ALz/fnjY\nafVqmD0bzjmn4fRf/3r96xERaQl6c1YT/eMfoQnjunWha9jGAj7o5SQiknkK+k0wb154ynXnzlCP\nP3x4vOXuvLN2a5yW7ulSRPKbgn6KXnkFTj89tKGfMyf0KxP33bMTJ4auFIqLQ9v34mK9jUpEWpbq\n9FPwhz+EzsmOOSZU6XTvnlrf94lpCvIikikq6cf06KOhL5oBA+DVV0PAh/gvJhcRaQ0U9GP4yU/g\nq1+F006DF14I/dAnpLvvexGR5qSg34h//Ve47rrw8o4//al21wRqkSMi2URBvwE/+xncfjtcfjn8\n9rd1v/VHLXJEJJso6NfjlVdCCX/MmPDu0/peKagWOSKSTdS1ch1WrgwvCikshNdfD10Xi4i0ZnG7\nVlZJv4Zt20LnaXv2hLdJKeCLSC5RO/0ke/fC174GixeH98ced1ymcyQikl4K+kmmTg3vmb3vPjjr\nrEznRkQk/VS9E3niCfj+92HSJLj++kznRkSkecQK+mY22syWmtlyM7upjvk9zewlM3vTzBaa2Zho\neomZ7TCz+dHnZ+negXRYsCBU65x8cmimqXfCikiuajTom1kB8CBwDlAKTDCz0hrJbgNmuftgYDzw\nX0nz3nX3QdHn6jTlO23Wr4fzzw8vH//d76B9+/3z4nakJiKSLeLU6Q8Dlrv7CgAzmwmMBZYkpXHg\n0Gj4MOCDdGayuezcCePGhcA/Z071t16l2pGaiEg2iFO90x1YnTReEU1LNgW4zMwqgKeB65Lm9Yqq\nfV4xs1Pq2oCZTTazcjMrr6ysjJ/7A+AeHr567TV4+GEYOrT6fHWkJiK5KF03cicAv3T3ImAM8IiZ\ntQHWAj2jap8bgcfM7NCaC7v7NHcvc/eywsLCNGWpYT/9aXhy9uabYcKE2vPVkZqI5KI4QX8N0CNp\nvCialuzrwCwAd/8bcDDQzd0/dfcN0fR5wLtAxlu/P/cc/NM/wXnnwQ9+UHcadaQmIrkoTtCfC/Q2\ns15m1o5wo3Z2jTTvA18EMLMTCEG/0swKoxvBmNkxQG9gRboy3xSPPQZf+hKUloY+8tvU8wuoIzUR\nyUWNBn133w1cCzwLvE1opbPYzKaa2flRsu8A3zCzBcAMYJKHTn1OBRaa2XzgceBqd9/YHDvSGHe4\n665wE/YLXwgdqh1aq6JpP3WkJiK5KC86XNu9G779bXjoIbj00nDjNrlppohItlOHa5GtW0Pd/UMP\nhZY3jz6qgC8i+Sun+95ZswbOPRcWLQpB/6qrMp0jEZHMytmgv2hReAHKxx/DH/8Io0dnOkciIpmX\nk9U7zz8PI0aErpJfe00BX0QkIeeC/i9/CeecE1rbvP46DBqU6RyJiLQeORP03WHKFLjiChg5MpTw\ne/RobCkDOgwPAAAMNklEQVQRkfySM0F/2TL44Q9Df/hPP63XHIqI1CVnbuQefzzMmxeetFV/+CIi\ndcuZoA/Qt2+mcyAi0rrlTPWOiIg0TkFfRCSPKOiLiOQRBX0RkTyioC8ikkcU9EVE8oiCvohIHlHQ\nFxHJIwr6IiJ5REFfRCSPKOiLiOQRBX0RkTwSK+ib2WgzW2pmy83spjrm9zSzl8zsTTNbaGZjkubd\nHC231MzOTmfmRUQkNY32smlmBcCDwCigAphrZrPdfUlSstuAWe7+UzMrBZ4GSqLh8UBf4GjgeTM7\nzt33pHtHRESkcXFK+sOA5e6+wt13AjOBsTXSOHBoNHwY8EE0PBaY6e6fuvtKYHm0PhERyYA4Qb87\nsDppvCKalmwKcJmZVRBK+delsGyLmj4dSkqgTZvwPX16JnMjItKy0nUjdwLwS3cvAsYAj5hZ7HWb\n2WQzKzez8srKyjRlqbbp02HyZHjvvfBO3ffeC+MK/CKSL+IE5jVA8ivGi6Jpyb4OzAJw978BBwPd\nYi6Lu09z9zJ3LyssLIyf+xTdeitUVVWfVlUVpouI5IM4QX8u0NvMeplZO8KN2dk10rwPfBHAzE4g\nBP3KKN14M2tvZr2A3sD/pivzqXr//dSmi4jkmkaDvrvvBq4FngXeJrTSWWxmU83s/CjZd4BvmNkC\nYAYwyYPFhCuAJcD/ANdksuVOz56pTRcRyTXm7pnOQzVlZWVeXl7eLOtO1OknV/F07AjTpsHEic2y\nSRGRFmFm89y9rLF0efVE7sSJIcAXF4NZ+FbAF5F80ujDWblm4kQFeRHJX3lV0hcRyXcK+iIieURB\nX0Qkjyjoi4jkEQV9EZE8oqAvIpJHFPRFRPKIgr6ISB5R0BcRySMK+iIieURBX0Qkjyjoi4jkEQV9\nEZE8oqAvIpJHFPRFRPKIgr6ISB5R0BcRySMK+iIieURBX0Qkjyjoi4jkkVhB38xGm9lSM1tuZjfV\nMf8+M5sffZaZ2cdJ8/YkzZudzsyLiEhqDmosgZkVAA8Co4AKYK6ZzXb3JYk07n5DUvrrgMFJq9jh\n7oPSl2UREWmqOCX9YcByd1/h7juBmcDYBtJPAGakI3MiIpJecYJ+d2B10nhFNK0WMysGegEvJk0+\n2MzKzex1M7ugnuUmR2nKKysrY2ZdRERSle4bueOBx919T9K0YncvAy4F7jezY2su5O7T3L3M3csK\nCwvTnCUREUmIE/TXAD2SxouiaXUZT42qHXdfE32vAF6men2/iIi0oDhBfy7Q28x6mVk7QmCv1QrH\nzPoAXYC/JU3rYmbto+FuwHBgSc1lRUSkZTTaesfdd5vZtcCzQAHwsLsvNrOpQLm7J04A44GZ7u5J\ni58A/NzM9hJOMD9MbvUjIiIty6rH6MwrKyvz8vLyTGdDRCSrmNm86P5pg/RErohIHlHQFxHJIwr6\nIiJ5REFfRCSPKOiLiOQRBX0RkTyioC8ikkcU9EVE8oiCvohIHlHQFxHJIwr6IiJ5REFfRCSPKOiL\niOQRBX0RkTyioC8ikkcU9EVE8oiCvohIHlHQFxHJIwr6IiJ5REFfRCSPKOiLiOQRBX0RkTwSK+ib\n2WgzW2pmy83spjrm32dm86PPMjP7OGne5Wb2TvS5PJ2ZFxGR1BzUWAIzKwAeBEYBFcBcM5vt7ksS\nadz9hqT01wGDo+EjgDuAMsCBedGym9K6FyIiEkuckv4wYLm7r3D3ncBMYGwD6ScAM6Lhs4Hn3H1j\nFOifA0YfSIZFRKTp4gT97sDqpPGKaFotZlYM9AJeTGVZM5tsZuVmVl5ZWRkn3yIi0gTpvpE7Hnjc\n3fekspC7T3P3MncvKywsTHOWREQkIU7QXwP0SBoviqbVZTz7q3ZSXVZERJpZnKA/F+htZr3MrB0h\nsM+umcjM+gBdgL8lTX4WOMvMuphZF+CsaJqIiGRAo6133H23mV1LCNYFwMPuvtjMpgLl7p44AYwH\nZrq7Jy270cz+lXDiAJjq7hvTuwsiIhKXJcXoVqGsrMzLy8sznQ0RkaxiZvPcvayxdHoiV0Qkjyjo\ni4jkEQV9EZE8oqAvIpJHFPRFRPKIgr6ISB5R0BcRySMK+iIieURBX0Qkjyjoi4jkEQV9EZE8oqAv\nIpJHFPRFRPKIgr6ISB5R0BcRySMK+iIieURBX0Qkjyjoi4jkEQV9EZE8oqAvIpJHFPRFRPJIrKBv\nZqPNbKmZLTezm+pJ8xUzW2Jmi83ssaTpe8xsfvSZna6Mi4hI6g5qLIGZFQAPAqOACmCumc129yVJ\naXoDNwPD3X2TmR2ZtIod7j4ozfkWEZEmiFPSHwYsd/cV7r4TmAmMrZHmG8CD7r4JwN3XpzebIiKS\nDnGCfndgddJ4RTQt2XHAcWb2FzN73cxGJ8072MzKo+kX1LUBM5scpSmvrKxMaQcSpk+HkhJo0yZ8\nT5/epNWIiOS0Rqt3UlhPb2AkUAS8amb93f1joNjd15jZMcCLZrbI3d9NXtjdpwHTAMrKyjzVjU+f\nDpMnQ1VVGH/vvTAOMHFiU3dJRCT3xCnprwF6JI0XRdOSVQCz3X2Xu68ElhFOArj7muh7BfAyMPgA\n81zLrbfuD/gJVVVhuoiI7Bcn6M8FeptZLzNrB4wHarbCeYpQysfMuhGqe1aYWRcza580fTiwhDR7\n//3UpouI5KtGg7677wauBZ4F3gZmuftiM5tqZudHyZ4FNpjZEuAl4F/cfQNwAlBuZgui6T9MbvWT\nLj17pjZdRCRfmXvKVejNqqyszMvLy1NapmadPkDHjjBtmur0RSQ/mNk8dy9rLF1OPJE7cWII8MXF\nYBa+FfBFRGpLV+udjJs4UUFeRKQxOVHSFxGReBT0RUTyiIK+iEgeUdAXEckjCvoiInmk1bXTN7NK\n4L0DWEU34KM0Zac1yLX9gdzbp1zbH8i9fcq1/YHa+1Ts7oWNLdTqgv6BMrPyOA8oZItc2x/IvX3K\ntf2B3NunXNsfaPo+qXpHRCSPKOiLiOSRXAz60zKdgTTLtf2B3NunXNsfyL19yrX9gSbuU87V6YuI\nSP1ysaQvIiL1UNAXEckjORP0zWy0mS01s+VmdlOm85MOZrbKzBaZ2XwzS+0lA62AmT1sZuvN7K2k\naUeY2XNm9k703SWTeUxVPfs0xczWRMdpvpmNyWQeU2FmPczsJTNbYmaLzez6aHpWHqcG9iebj9HB\nZva/ZrYg2qfvR9N7mdnfo5j3m+jNho2vLxfq9M2sgPBe3lGE9/XOBSY0x1u6WpKZrQLK3D0rHyox\ns1OBbcCv3b1fNO3fgI3u/sPo5NzF3f9PJvOZinr2aQqwzd1/nMm8NYWZHQUc5e5vmFlnYB5wATCJ\nLDxODezPV8jeY2RAJ3ffZmZtgTnA9cCNwO/cfaaZ/QxY4O4/bWx9uVLSHwYsd/cV7r4TmAmMzXCe\n8p67vwpsrDF5LPCraPhXhH/IrFHPPmUtd1/r7m9Ew1sJr0TtTpYepwb2J2t5sC0abRt9HDgDeDya\nHvsY5UrQ7w6sThqvIMsPdMSBP5vZPDObnOnMpMln3H1tNPwh8JlMZiaNrjWzhVH1T1ZUhdRkZiXA\nYODv5MBxqrE/kMXHyMwKzGw+sB54DngX+Dh6hzmkEPNyJejnqhHuPgQ4B7gmqlrIGR7qFrO/fhF+\nChwLDALWAvdmNjupM7NDgCeAf3b3LcnzsvE41bE/WX2M3H2Puw8Cigg1G32auq5cCfprgB5J40XR\ntKzm7mui7/XAk4SDne3WRfWuifrX9RnOzwFz93XRP+Ve4CGy7DhF9cRPANPd/XfR5Kw9TnXtT7Yf\nowR3/xh4CTgZONzMEq+8jR3zciXozwV6R3ez2wHjgdkZztMBMbNO0Y0ozKwTcBbwVsNLZYXZwOXR\n8OXA7zOYl7RIBMfIhWTRcYpuEv438La7/3vSrKw8TvXtT5Yfo0IzOzwa7kBosPI2IfiPi5LFPkY5\n0XoHIGqCdT9QADzs7ndmOEsHxMyOIZTuIbzA/rFs2yczmwGMJHQBuw64A3gKmAX0JHSh/RV3z5ob\no/Xs00hCtYEDq4BvJtWHt2pmNgJ4DVgE7I0m30KoB8+649TA/kwge4/RAMKN2gJCQX2Wu0+NYsRM\n4AjgTeAyd/+00fXlStAXEZHG5Ur1joiIxKCgLyKSRxT0RUTyiIK+iEgeUdAXEckjCvoiInlEQV9E\nJI/8f8VPE/svVwa/AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fd5529c2e80>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAEICAYAAACktLTqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XuUFNW59/Hvw3XkElAgKtdBReV+cUR9EQGvqBHEIAHx\nAieG6IkxiTFLlhpjiKyDxqDR8HokeTVGR5GjUfEWNEcimkTkEgQRCYiAAwgDCoKAOMzz/rG7h2bo\nmeme6Zme7v591qrVXdW7q3d1zTy1+6ldu8zdERGR7NIg3RUQEZHUU3AXEclCCu4iIllIwV1EJAsp\nuIuIZCEFdxGRLKTgLnGZWUMz221mnVNZNp3M7AQzS3nfXzM718zWxcyvMrPBiZStxmf9wcxure77\nK1nvXWb2x1SvV9KnUborIKlhZrtjZpsBXwEHIvPfd/fCZNbn7geAFqkumwvc/aRUrMfMrgWudPeh\nMeu+NhXrluyn4J4l3L0suEZahte6+18rKm9mjdy9pC7qJiJ1T2mZHBH52f20mT1lZruAK83sDDN7\nx8x2mNlmM3vAzBpHyjcyMzez/Mj8E5HXXzWzXWb2TzPrmmzZyOsXmtm/zWynmT1oZn83swkV1DuR\nOn7fzNaY2edm9kDMexua2X1mtt3M1gLDK/l+bjOzWeWWzTCz6ZHn15rZysj2fBRpVVe0riIzGxp5\n3szMHo/UbQVwSrmyt5vZ2sh6V5jZiMjy3sDvgMGRlNe2mO/2zpj3XxfZ9u1m9ryZHZvId1MVMxsV\nqc8OM3vDzE6Kee1WM9tkZl+Y2Ycx23q6mS2JLN9iZr9O9POkFri7piybgHXAueWW3QXsBy4hHNSP\nAE4FTiP8gjsO+DdwQ6R8I8CB/Mj8E8A2oABoDDwNPFGNst8EdgEjI6/dBHwNTKhgWxKp4wtAKyAf\n+Cy67cANwAqgI9AGmB/+5ON+znHAbqB5zLq3AgWR+UsiZQw4G9gL9Im8di6wLmZdRcDQyPN7gb8B\nRwJdgA/KlR0DHBvZJ1dE6nB05LVrgb+Vq+cTwJ2R5+dH6tgPyAP+L/BGIt9NnO2/C/hj5Hn3SD3O\njuyjW4FVkec9gfXAMZGyXYHjIs8XAuMiz1sCp6X7fyGXJ7Xcc8vb7v6iu5e6+153X+juC9y9xN3X\nAjOBIZW8/xl3X+TuXwOFhKCSbNlvAUvd/YXIa/cRDgRxJVjH/3L3ne6+jhBIo581BrjP3YvcfTsw\nrZLPWQu8TzjoAJwHfO7uiyKvv+juaz14A/hfIO5J03LGAHe5++fuvp7QGo/93NnuvjmyT54kHJgL\nElgvwHjgD+6+1N33AZOBIWbWMaZMRd9NZcYCc9z9jcg+mkY4QJwGlBAOJD0jqb2PI98dhIN0NzNr\n4+673H1BgtshtUDBPbd8EjtjZieb2ctm9qmZfQFMAdpW8v5PY57vofKTqBWVbR9bD3d3Qks3rgTr\nmNBnEVqclXkSGBd5fkVkPlqPb5nZAjP7zMx2EFrNlX1XUcdWVgczm2Bm70XSHzuAkxNcL4TtK1uf\nu38BfA50iCmTzD6raL2lhH3Uwd1XAT8l7IetkTTfMZGiE4EewCoze9fMLkpwO6QWKLjnlvLdAB8m\ntFZPcPdvAHcQ0g61aTMhTQKAmRmHBqPyalLHzUCnmPmqumrOBs41sw6EFvyTkToeATwD/BchZdIa\neC3BenxaUR3M7DjgIeB6oE1kvR/GrLeqbpubCKme6PpaEtI/GxOoVzLrbUDYZxsB3P0Jdx9ESMk0\nJHwvuPsqdx9LSL39BnjWzPJqWBepJgX33NYS2Al8aWbdge/XwWe+BAwws0vMrBHwI6BdLdVxNvBj\nM+tgZm2AWyor7O6fAm8DfwRWufvqyEtNgSZAMXDAzL4FnJNEHW41s9YWrgO4Iea1FoQAXkw4zn2P\n0HKP2gJ0jJ5AjuMp4Ltm1sfMmhKC7FvuXuEvoSTqPMLMhkY++2eE8yQLzKy7mQ2LfN7eyFRK2ICr\nzKxtpKW/M7JtpTWsi1STgntu+ylwDeEf92HCic9a5e5bgO8A04HtwPHAvwj98lNdx4cIufHlhJN9\nzyTwnicJJ0jLUjLuvgP4CfAc4aTkaMJBKhG/IPyCWAe8CvwpZr3LgAeBdyNlTgJi89SvA6uBLWYW\nm16Jvv8vhPTIc5H3dybk4WvE3VcQvvOHCAee4cCISP69KXAP4TzJp4RfCrdF3noRsNJCb6x7ge+4\n+/6a1keqx0LKUyQ9zKwhIQ0w2t3fSnd9RLKFWu5S58xseCRN0RT4OaGXxbtprpZIVlFwl3Q4E1hL\n+Ml/ATDK3StKy4hINSgtIyKShdRyFxHJQmkbOKxt27aen5+fro8XEclIixcv3ubulXUfBtIY3PPz\n81m0aFG6Pl5EJCOZWVVXWgNKy4iIZCUFdxGRLKTgLiKShXQnJpEc8fXXX1NUVMS+ffvSXRVJQF5e\nHh07dqRx44qGFqqcgrtIjigqKqJly5bk5+cTBuOU+srd2b59O0VFRXTt2rXqN8SRUWmZwkLIz4cG\nDcJjYVK3fBbJbfv27aNNmzYK7BnAzGjTpk2NfmVlTMu9sBAmTYI9e8L8+vVhHmB8jcfBE8kNCuyZ\no6b7KmNa7rfddjCwR+3ZE5aLiMihMia4b9iQ3HIRqV+2b99Ov3796NevH8cccwwdOnQom9+/P7Fh\n3ydOnMiqVasqLTNjxgwKU5SzPfPMM1m6dGlK1lXXMiYt07lzSMXEWy4iqVdYGH4Zb9gQ/s+mTq1Z\nCrRNmzZlgfLOO++kRYsW3HzzzYeUcXfcnQYN4rc7H3300So/5wc/+EH1K5lFMqblPnUqNGt26LJm\nzcJyEUmt6Dmu9evB/eA5rtroxLBmzRp69OjB+PHj6dmzJ5s3b2bSpEkUFBTQs2dPpkyZUlY22pIu\nKSmhdevWTJ48mb59+3LGGWewdetWAG6//Xbuv//+svKTJ09m4MCBnHTSSfzjH/8A4Msvv+Tb3/42\nPXr0YPTo0RQUFFTZQn/iiSfo3bs3vXr14tZbbwWgpKSEq666qmz5Aw88AMB9991Hjx496NOnD1de\neWXKv7NEZEzLPdpiSGVLQkTiq+wcV238z3344Yf86U9/oqCgAIBp06Zx1FFHUVJSwrBhwxg9ejQ9\nevQ45D07d+5kyJAhTJs2jZtuuolHHnmEyZMnH7Zud+fdd99lzpw5TJkyhb/85S88+OCDHHPMMTz7\n7LO89957DBgwoNL6FRUVcfvtt7No0SJatWrFueeey0svvUS7du3Ytm0by5cvB2DHjh0A3HPPPaxf\nv54mTZqULatrGdNyh/BHtW4dlJaGRwV2kdpR1+e4jj/++LLADvDUU08xYMAABgwYwMqVK/nggw8O\ne88RRxzBhRdeCMApp5zCunXr4q77sssuO6zM22+/zdixYwHo27cvPXv2rLR+CxYs4Oyzz6Zt27Y0\nbtyYK664gvnz53PCCSewatUqbrzxRubOnUurVq0A6NmzJ1deeSWFhYXVvgippjIquItI3ajoXFZt\nneNq3rx52fPVq1fz29/+ljfeeINly5YxfPjwuP29mzRpUva8YcOGlJSUxF1306ZNqyxTXW3atGHZ\nsmUMHjyYGTNm8P3vfx+AuXPnct1117Fw4UIGDhzIgQMHUvq5iVBwF5HDpPMc1xdffEHLli35xje+\nwebNm5k7d27KP2PQoEHMnj0bgOXLl8f9ZRDrtNNOY968eWzfvp2SkhJmzZrFkCFDKC4uxt25/PLL\nmTJlCkuWLOHAgQMUFRVx9tlnc88997Bt2zb2lM9x1YGEcu5mNhz4LdAQ+IO7T4tTZgxwJ+DAe+5+\nRQrrKSJ1KJ3nuAYMGECPHj04+eST6dKlC4MGDUr5Z/zwhz/k6quvpkePHmVTNKUST8eOHfnVr37F\n0KFDcXcuueQSLr74YpYsWcJ3v/td3B0z4+6776akpIQrrriCXbt2UVpays0330zLli1Tvg1VqfIe\nqmbWEPg3cB5QBCwExrn7BzFlugGzgbPd/XMz+6a7b61svQUFBa6bdYjUnZUrV9K9e/d0V6NeKCkp\noaSkhLy8PFavXs3555/P6tWradSofvUxibfPzGyxuxdU8JYyiWzJQGCNu6+NrHgWMBKI/R3zPWCG\nu38OUFVgFxFJp927d3POOedQUlKCu/Pwww/Xu8BeU4lsTQfgk5j5IuC0cmVOBDCzvxNSN3e6+1/K\nr8jMJgGTADrr6iMRSZPWrVuzePHidFejVqXqhGojoBswFBgH/N7MWpcv5O4z3b3A3Qvatavy/q4i\nIlJNiQT3jUCnmPmOkWWxioA57v61u39MyNF3S00VRUQkWYkE94VANzPramZNgLHAnHJlnie02jGz\ntoQ0zdoU1lNERJJQZXB39xLgBmAusBKY7e4rzGyKmY2IFJsLbDezD4B5wM/cfXttVVpERCqXUM7d\n3V9x9xPd/Xh3nxpZdoe7z4k8d3e/yd17uHtvd59Vm5UWkcwzbNiwwy5Iuv/++7n++usrfV+LFi0A\n2LRpE6NHj45bZujQoVTVtfr+++8/5GKiiy66KCXjvtx5553ce++9NV5PqukKVRGpE+PGjWPWrEPb\nfbNmzWLcuHEJvb99+/Y888wz1f788sH9lVdeoXXrw/p9ZA0FdxGpE6NHj+bll18uuzHHunXr2LRp\nE4MHDy7rdz5gwAB69+7NCy+8cNj7161bR69evQDYu3cvY8eOpXv37owaNYq9e/eWlbv++uvLhgv+\nxS9+AcADDzzApk2bGDZsGMOGDQMgPz+fbdu2ATB9+nR69epFr169yoYLXrduHd27d+d73/sePXv2\n5Pzzzz/kc+JZunQpp59+On369GHUqFF8/vnnZZ8fHQI4OmDZm2++WXazkv79+7Nr165qf7fxZFev\nfRFJyI9/DKm+wVC/fhCJi3EdddRRDBw4kFdffZWRI0cya9YsxowZg5mRl5fHc889xze+8Q22bdvG\n6aefzogRIyq8j+hDDz1Es2bNWLlyJcuWLTtkyN6pU6dy1FFHceDAAc455xyWLVvGjTfeyPTp05k3\nbx5t27Y9ZF2LFy/m0UcfZcGCBbg7p512GkOGDOHII49k9erVPPXUU/z+979nzJgxPPvss5WOz371\n1Vfz4IMPMmTIEO644w5++ctfcv/99zNt2jQ+/vhjmjZtWpYKuvfee5kxYwaDBg1i9+7d5OXlJfFt\nV00tdxGpM7GpmdiUjLtz66230qdPH84991w2btzIli1bKlzP/Pnzy4Jsnz596NOnT9lrs2fPZsCA\nAfTv358VK1ZUOSjY22+/zahRo2jevDktWrTgsssu46233gKga9eu9OvXD6h8WGEI48vv2LGDIUOG\nAHDNNdcwf/78sjqOHz+eJ554ouxK2EGDBnHTTTfxwAMPsGPHjpRfIauWu0gOqqyFXZtGjhzJT37y\nE5YsWcKePXs45ZRTACgsLKS4uJjFixfTuHFj8vPz4w7zW5WPP/6Ye++9l4ULF3LkkUcyYcKEaq0n\nKjpcMIQhg6tKy1Tk5ZdfZv78+bz44otMnTqV5cuXM3nyZC6++GJeeeUVBg0axNy5czn55JOrXdfy\n1HIXkTrTokULhg0bxn/8x38cciJ1586dfPOb36Rx48bMmzeP9fFumBzjrLPO4sknnwTg/fffZ9my\nZUAYLrh58+a0atWKLVu28Oqrr5a9p2XLlnHz2oMHD+b5559nz549fPnllzz33HMMHjw46W1r1aoV\nRx55ZFmr//HHH2fIkCGUlpbyySefMGzYMO6++2527tzJ7t27+eijj+jduze33HILp556Kh9++GHS\nn1kZtdxFpE6NGzeOUaNGHdJzZvz48VxyySX07t2bgoKCKluw119/PRMnTqR79+5079697BdA3759\n6d+/PyeffDKdOnU6ZLjgSZMmMXz4cNq3b8+8efPKlg8YMIAJEyYwcOBAAK699lr69+9faQqmIo89\n9hjXXXcde/bs4bjjjuPRRx/lwIEDXHnllezcuRN358Ybb6R169b8/Oc/Z968eTRo0ICePXuW3VUq\nVaoc8re2aMhfkbqlIX8zT02G/FVaRkQkCym4i4hkIQV3kRySrjSsJK+m+0rBXSRH5OXlsX37dgX4\nDODubN++vUYXNqm3jEiO6NixI0VFRRQXF6e7KpKAvLw8OnbsWO33K7iL5IjGjRvTtWvXdFdD6ojS\nMiIiWUjBXUQkCym4i4hkIQV3EZEspOAuIpKFFNxFRLKQgruISBZScBcRyUIK7iIiWUjBXUQkCym4\ni4hkIQV3EZEspOAuIpKFFNxFRLJQQsHdzIab2SozW2Nmk+O8PsHMis1saWS6NvVVFRGRRFU5nruZ\nNQRmAOcBRcBCM5vj7h+UK/q0u99QC3UUEZEkJdJyHwiscfe17r4fmAWMrN1qiYhITSQS3DsAn8TM\nF0WWlfdtM1tmZs+YWad4KzKzSWa2yMwW6VZfIiK1J1UnVF8E8t29D/A68Fi8Qu4+090L3L2gXbt2\nKfpoEREpL5HgvhGIbYl3jCwr4+7b3f2ryOwfgFNSUz0REamORIL7QqCbmXU1sybAWGBObAEzOzZm\ndgSwMnVVFBGRZFXZW8bdS8zsBmAu0BB4xN1XmNkUYJG7zwFuNLMRQAnwGTChFussIiJVMHdPywcX\nFBT4okWL0vLZIiKZyswWu3tBVeV0haqISBZScBcRyUIK7iIiWUjBXUQkCym4i4hkIQV3EZEspOAu\nIpKFFNxFRLKQgruISBZScBcRyUIZGdy/+qrqMoWFkJ8PDRqEx8LC2q6ViEj9kXHB/Xe/g86dYe/e\nissUFsKkSbB+PbiHx0mTFOBFJHdkXHDv3h22boXnn6+4zG23wZ49hy7bsycsFxHJBRkX3IcNgy5d\n4I9/rLjMhg3JLRcRyTYZF9wbNIBrroHXX4dPPolfpnPn5JaLiGSbjAvuEIK7O/zpT/FfnzoVmjU7\ndFmzZmG5iEguyMjgftxxMGRISM3Eu9fI+PEwc2ZI35iFx5kzw3IRkVyQkcEdYOJEWLMG3n47/uvj\nx8O6dVBaGh4V2EUkl2RscB89Glq0qPzEqohIrsrY4N68OYwZA7Nnw5dfprs2IiL1S8YGdwipmd27\n4Zln0l0TEZH6JaOD+6BBcMIJ8Oij6a6JiEj9ktHB3QwmTIA334S1a9NdGxGR+iOjgzvA1VeHIP/Y\nY+muiYhI/ZHxwb1TJzjvvBDcS0vTXRsRkfoh44M7hNTM+vUwb166ayIiUj9kRXC/9FJo1UonVkVE\norIiuB9xBIwbB3/+M+zcme7aiIikX0LB3cyGm9kqM1tjZpMrKfdtM3MzK0hdFRMzcWK4gcfs2XX9\nySIi9U+Vwd3MGgIzgAuBHsA4M+sRp1xL4EfAglRXMhGnngo9eig1IyICibXcBwJr3H2tu+8HZgEj\n45T7FXA3sC+F9UtYtM/7P/8JH36YjhqIiNQfiQT3DkDsbTGKIsvKmNkAoJO7v5zCuiXtqqugYUMN\nJiYiUuMTqmbWAJgO/DSBspPMbJGZLSouLq7pRx/mmGPgwgvh8cfhwIGUr15EJGMkEtw3Ap1i5jtG\nlkW1BHoBfzOzdcDpwJx4J1Xdfaa7F7h7Qbt27apf60pMnAibNsFrr9XK6kVEMkIiwX0h0M3MuppZ\nE2AsMCf6orvvdPe27p7v7vnAO8AId19UKzWuwre+BW3a6MSqiOS2KoO7u5cANwBzgZXAbHdfYWZT\nzGxEbVcwWU2ahLsuvfACfPZZumsjIpIeCeXc3f0Vdz/R3Y9396mRZXe4+5w4ZYemq9UeNXEi7N8P\nTz6ZzlqIiKRPVlyhWl6/fmFSrxkRyVVZGdwhtN4XL4bly9NdExGRupe1wf2KK6BxY5g+Pd01ERGp\ne1kb3Nu2hR//OKRm/v73dNdGRKRuZW1wB7jjjnAzj+uug6+/jl+msBDy86FBg/BYWFiXNRQRqR1Z\nHdxbtIAHH4T334ff/vbw1wsLYdKkcKMP9/A4aZICvIhkPnP3tHxwQUGBL1pUNz0mR4yAN96AlStD\nSz4qPz8E9PK6dIF16+qkaiIiSTGzxe5e5bDqWd1yj3rggXB/1R/96NDlGzbEL1/RchGRTJETwT0/\nP+Tfn3sOXnrp4PLOneOXr2i5iEimyIngDnDTTeFmHj/8IezZE5ZNnQrNmh1arlmzsFxEJJPlTHBv\n0gQeeijk0u+6KywbPx5mzgw5drPwOHNmWC4iksly4oRqrAkTwpgzS5eGlryISCbRCdUK/PrXoYvk\nf/5n6P4oIpKNci64t2sH06bBm2+GOzaJiGSjnAvuANdeC6efDjffrDHfRSQ75WRwb9AA/vu/Q2C/\n9dZ010ZEJPVyMrgD9O0LN94Yese88066ayMiklo5G9wBfvlLaN8+DCxWUpLu2oiIpE5OB/eWLcOA\nYu+9B7/7XbprIyKSOjkd3AEuuwwuvBBuuw2efjrdtRERSY2cD+5m8Mgj4Z6rY8fCDTfAV1+lu1Yi\nIjWT88Ed4Jhj4G9/C10jZ8yAQYPg44/TXSsRkepTcI9o3Dhcvfr88/DRR9C/P7zwwqFldNcmEckU\nCu7ljBwJS5ZAt25w6aXw05+GW/Tprk0ikklybuCwRH31VUjT/O53cMYZ4QYeGzceXk53bRKRuqSB\nw2qoadNw/9VZs2D58viBHXTXJhGpnxTcq/Cd78DixSEnH4/u2iQi9ZGCewJOPBEefhgaNjx0ue7a\nJCL1lYJ7giZOhMceg7Ztw7xZuABq3LjDy6pXjYikW0LB3cyGm9kqM1tjZpPjvH6dmS03s6Vm9raZ\nZeU9jsaPh+LikH+/6CJ44gkYOjR0nYxSrxoRqQ+qDO5m1hCYAVwI9ADGxQneT7p7b3fvB9wDTE95\nTeuR9u3hxRfh0UfDuDR9+oSLn0pLwzAG0RtwR+3ZE5aLiNSVRFruA4E17r7W3fcDs4CRsQXc/YuY\n2eZA1t/Azizcj3XFChg8OAxbcN55oaUej3rViEhdSiS4dwA+iZkviiw7hJn9wMw+IrTcb4y3IjOb\nZGaLzGxRcXFxdepb73TsCK++GsaFf/fdEPTjUa8aEalLKTuh6u4z3P144Bbg9grKzHT3AncvaNeu\nXao+Ou3M4Hvfg/ffh+7dD39dvWpEpK4lEtw3Ap1i5jtGllVkFnBpTSqVqbp0CRc8XXPNwRZ8y5bw\nq1+Fk7EiInUlkeC+EOhmZl3NrAkwFpgTW8DMusXMXgysTl0VM0uDBvDHP8KaNSHI79sHP/sZjB4N\n//xnumsnIrmiyuDu7iXADcBcYCUw291XmNkUMxsRKXaDma0ws6XATcA1tVbjDHHccSHIr1sHt9wC\nb7wB/+f/hOnZZ+HAgXTXUESymQYOqyO7d4dgf999sHZtCP4/+Um4OKp583TXTkQyhQYOq2datAjd\nJf/9b3jmGTj6aPjhD6FTJ7j11nAytrQ03bUUkWyh4F7HGjaEb38b/vGPMJ19NkybBr17Q7t2YQz5\n6dNh0SIoKUl3bUUkUzVKdwVy2RlnhFb8J5+EnPz8+WGK3gGqRYtwy7+zzgrTqaeGoYhFRKqi4J5m\nhYVhaIING8KFTlOnwrBh8NZbB4N9dOiCpk3DCdkRI8Ido7p2TW/dRaT+0gnVNIoOMhY7Fk2zZuFq\n19h+8du3w9tvh0D/2mshPw9hTJtLLw1Tv34VXx0rItkj0ROqCu5plJ8ffyyaqm7d99FHIXXz/PPw\n97+HE7GdO4fW/KWXhrFuKrq5iIhkNgX3DNCgQRgWuDyzxHvOFBfDSy+FQP/aa+GiqSOPhIsvhnPO\ngdNPDzcbaaBT5yJZQcE9A1S35V6RL78MAf6FF8KQxJ99Fpa3bg2nnRam008Pj0cdVZOai0i6KLhn\ngERz7tVRWgoffggLFsA774Qpti/9iSceDPS9e0NeXkjlNG4MTZocfB67LLpcRNJHwT1DxOstEy+w\nJ1quMrt2hf7z0WD/zjuwdWty6zjpJLjgAjj//HAXKl1dK1K3FNyzSG218KO3AVy1Cr7+Okz79x98\nXn7asycMfvbmmyG336QJnHlmCPQXXBB672Rbbn/btnA+Y+7ccM3B9dfrWgNJLwX3LJLq3HxN7dsX\n+uHPnRty/MuXh+VHHx3uRnXBBeGiqw4dwhW5mWbbNnjuOfif/wkXlx04EG6Mvm1b+M7vuguuuCL7\nDmSSGRTcs0gqetXUpk2bQpB/7TV4/fUQBCHU+5hjQpCvbGrRIjV99EtKwgBtX34Z0kWtWiW+3ngB\n/fjj4fLLw9S/P/z1r2GEz3/9C/r2DcNGXHCBri+IKimBvXvDPQzqsw0bwq/VvXurnvbvDwf0Xr3C\nuanjjkv/QV3BPYsk23JPRX6+ukpLQ/BbuBA2bjx82rHj8PeYhTRTvOmIIw4+b9gwBO5du0IQ3737\n0Of79h263saNQ4u7XbuDj+Wn7dvDEBDz5oWAfsIJBwN6vAvDSkvh6afD9/vxx+Fq4rvvDkND1IaS\nkvB9bt0KAwbAscfWzudUV2lpuMDuqafC9/j55+GAd9VV4UrqZs3SWz93WL364NXe8+dXfJ/jKLPw\nd3fEEeFv6NNPD77WrBn07Hkw2PfuHZ4ffXTdHeQV3LNIMjn32uyBkwpffhla+tFgv2kTfPFFaCXt\n2RN/ir5WUhJa+dGpZcv4z5s3D8G+uPjwadu2ww8w0YA+ZkxokSfyT7p/Pzz8MEyZEtZ5+eXhINqt\nW9Xvrcy+feHAGA1E//hH2JaoDh1g4MCD0ymnhF8odckdliwJAf3pp6GoKATCESPCPYWjy1q2DIPk\nXXUVDBlSNym60tLQKyw2mG/ZEl5r1+7gOE39+oW/k2gQj055eeFcUuzfwO7d8MEHIf0Ynd5//9DO\nCG3bhlZ9s2ZhHdF1xT6PXXb++eEcVXUouGeZRFvj9S0/Xx99/XUIyMXF0KhRuO9tdVtdX3wBv/lN\nmL76KtxL96c/DdcRNG0apsqC2u7d4SR1NBAtWBDWA6FFGA1G7duHgPruu2Fas+bgOk4+OQT6U08N\nU/v2IbC2bJnagPrhhyGgP/VUaA03bhxa6ePGhcDeokUoV1oaTro//nhoze/aFQ5K48eHQN+rV/Xr\ncOBA2G/gx0FMAAAKnElEQVSbNh06bd4c/jcWLAi/HiAcaIYMOfgdnnRSalvXW7ceDPTLl4fP37cv\nTHv3HvoYfR5Noz78cGiEVYeCe46q7/n5bPXpp+FeuTNnHj5Uc8OGBwN906ahZRgN+qtXh4DVoEFI\nu0QD0ZlnQps2FX/eZ5+Fbq3RYP/uuwdbqLGOOOJgoC8/RVNdjRqFx+gUO9+oUTgYvvIKLF0a/o6G\nDQsB/bLLqr4Ybu9emDMHnngC/vKX8N307RsC/bHHHgx8+/aFg1rsfHT64ouDQXzLlvh3MfvmN8NB\n7ZRTDn6HXbrUr/Mh7mH79+0LB8a8vOqtR8E9RyXTck9nbj5brVkTTrxGg1Vl09dfh1b3WWeF0T5r\nciLSPaRCFi8OLdtduw6doucnYqc9e0KgLCkJj/GeR8PDaaeFgD5mTPXz/sXFIWXz+OPhYBRPNN8d\nTV/k5YVfBO3bH5yOPfbQ+aOPzq2L6xTcc1SiOff6npuX+sE9/OJLdb68qCgcAPPywq+YaCBv1Kh+\ntbbrIwX3HJZIi1y5eZHMpOAulVJuXiQz6QbZUqnOnZNbLiKZRcE9R02devgFJs2aheXxFBaGVE6D\nBuGxsLC2aygiNaHgnqPGjw8nT6Pdxbp0qfhkavTk6/r1BwcbmzRJAV6kPlPOXaqkk68i9Ydy7pIy\nGzYkt1xE0k/BXaqkk68imUfBXaqU7MlXEUm/hIK7mQ03s1VmtsbMJsd5/SYz+8DMlpnZ/5pZl9RX\nVdIl2ZOv6lUjkn5VnlA1s4bAv4HzgCJgITDO3T+IKTMMWODue8zsemCou3+nsvXqhGr20ZAGIrUv\nlSdUBwJr3H2tu+8HZgEjYwu4+zx3j/5LvwN0TLbCkvluu+3QwA5h/rbb4pdXK1+k9iQS3DsAn8TM\nF0WWVeS7wKvxXjCzSWa2yMwWFRcXJ15LyQjJ9KpR33mR2pXSE6pmdiVQAPw63uvuPtPdC9y9oF27\ndqn8aKkHkulVk2wrX0SSk0hw3wh0ipnvGFl2CDM7F7gNGOHuX6WmepJJkulVk2wrX+kbkeQkEtwX\nAt3MrKuZNQHGAnNiC5hZf+BhQmDfGmcdkgOS6VWTaCtf6RuR6klo+AEzuwi4H2gIPOLuU81sCrDI\n3eeY2V+B3sDmyFs2uPuIytap3jK5LdGeNRr6QORQKR1+wN1fcfcT3f14d58aWXaHu8+JPD/X3Y92\n936RqdLALpJoKz/ZoQ+UwhEJdIWqpM348aH1XVoaHmuSvoHkUjg6CEi2U3CXei2Zk7SJ9sBRHl9y\ngYK71GvJnKRNNIWjbpiSCxqluwIiVRk/PrHhCzp3jn/ytXwKR0MYSy5Qy12yRqIpHA1hLLlAwV2y\nRqIpHA1hLLlAwV2ySiI9cJLJ40PiPWvUA0fqE91DVaQSiV5speGOpa4kehGTgrtIJRK9QlZX0kpd\n0Q2yRVIg0Z41upJW6hsFd5FKJNqzRlfSSn2j4C5SiUR71uhKWqlvFNxFKpFozxpdSSv1jU6oitSx\nRE++NmgQWuzlmYWunuUVFobAv2FDSAdNnaqeOtlIJ1RF6qnauJJWKRwpT8FdpI7VxpW0SuFIeUrL\niNRjiaZakk3hSOZSWkYkCyQynAIkn8JR98rsp+AukgUSTeEkm5vXgSBzKbiLZIFE8/jJ5OZ1sVVm\nU85dJIckk5tPtMtmsoOmqctmzSjnLiKHSSY3XxsXW6nLZt1RcBfJIcl0r0z0QJDMoGnqsll3FNxF\nckgywyTUxsVWyRwIlMevGQV3kRyTaPfK2rjYKtEDgdI3NafgLiIVSvVtCxM9ECSbvlEr/3AK7iJS\nY6n+NZBs+qY2umxm+gFDXSFFpN5J5raFtdFlsz7fE1ddIUUkYyWTx6+NLpvZ0KsnoeBuZsPNbJWZ\nrTGzyXFeP8vMlphZiZmNTn01RSSXJJPHr40um9nQq6fK4G5mDYEZwIVAD2CcmfUoV2wDMAF4MtUV\nFJHclGgevza6bNZWr566PBAk0nIfCKxx97Xuvh+YBYyMLeDu69x9GaDBRUWkTtVGl83a6NVT1907\nEwnuHYBPYuaLIsuSZmaTzGyRmS0qLi6uzipERA6T6i6btdGrp67z+I1qZ7XxuftMYCaE3jJ1+dki\nIuPHJ97bJZGynTvH76lT06tzUyGRlvtGoFPMfMfIMhGRnFYbV+emSiLBfSHQzcy6mlkTYCwwp3aq\nIyKSOWrj6txUqTK4u3sJcAMwF1gJzHb3FWY2xcxGAJjZqWZWBFwOPGxmK2qnuiIi9Uuqr85NFV2h\nKiKSQXSFqohIDlNwFxHJQgruIiJZSMFdRCQLKbiLiGShtPWWMbNiIM61XQlpC2xLYXXqg2zbpmzb\nHsi+bcq27YHs26Z429PF3dtV9ca0BfeaMLNFiXQFyiTZtk3Ztj2QfduUbdsD2bdNNdkepWVERLKQ\ngruISBbK1OA+M90VqAXZtk3Ztj2QfduUbdsD2bdN1d6ejMy5i4hI5TK15S4iIpVQcBcRyUIZF9zN\nbLiZrTKzNWY2Od31qSkzW2dmy81sqZll5DCZZvaImW01s/djlh1lZq+b2erI45HprGMyKtieO81s\nY2Q/LTWzi9JZx2SZWSczm2dmH5jZCjP7UWR5Ru6nSrYnY/eTmeWZ2btm9l5km34ZWd7VzBZEYt7T\nkftqVL2+TMq5m1lD4N/AeYR7uS4Exrn7B2mtWA2Y2TqgwN0z9sILMzsL2A38yd17RZbdA3zm7tMi\nB+Ej3f2WdNYzURVsz53Abne/N511qy4zOxY41t2XmFlLYDFwKTCBDNxPlWzPGDJ0P5mZAc3dfbeZ\nNQbeBn4E3AT82d1nmdl/A++5+0NVrS/TWu4DgTXuvtbd9wOzgJFprlPOc/f5wGflFo8EHos8f4zw\nj5cRKtiejObum919SeT5LsKNdzqQofupku3JWB7sjsw2jkwOnA08E1me8D7KtODeAfgkZr6IDN+h\nhJ33mpktNrNJ6a5MCh3t7psjzz8Fjk5nZVLkBjNbFknbZET6Ih4zywf6AwvIgv1Ubnsgg/eTmTU0\ns6XAVuB14CNgR+SOeJBEzMu04J6NznT3AcCFwA8iKYGs4iH3lzn5v/geAo4H+gGbgd+ktzrVY2Yt\ngGeBH7v7F7GvZeJ+irM9Gb2f3P2Au/cDOhIyFSdXd12ZFtw3Ap1i5jtGlmUsd98YedwKPEfYodlg\nSyQvGs2Pbk1zfWrE3bdE/vFKgd+Tgfspksd9Fih09z9HFmfsfoq3PdmwnwDcfQcwDzgDaG1mjSIv\nJRzzMi24LwS6Rc4eNwHGAnPSXKdqM7PmkZNBmFlz4Hzg/crflTHmANdEnl8DvJDGutRYNABGjCLD\n9lPkZN3/A1a6+/SYlzJyP1W0PZm8n8ysnZm1jjw/gtBxZCUhyI+OFEt4H2VUbxmASNem+4GGwCPu\nPjXNVao2MzuO0FoHaAQ8mYnbY2ZPAUMJw5NuAX4BPA/MBjoThnYe4+4ZcZKygu0ZSvip78A64Psx\nuep6z8zOBN4ClgOlkcW3EvLUGbefKtmecWTofjKzPoQTpg0JDe/Z7j4lEidmAUcB/wKudPevqlxf\npgV3ERGpWqalZUREJAEK7iIiWUjBXUQkCym4i4hkIQV3EZEspOAuIpKFFNxFRLLQ/wfrVYYmygVP\nVgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fd550e7c828>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "acc = history.history['acc']\n",
    "val_acc = history.history['val_acc']\n",
    "loss = history.history['loss']\n",
    "val_loss = history.history['val_loss']\n",
    "\n",
    "epochs = range(len(acc))\n",
    "\n",
    "plt.plot(epochs, acc, 'bo', label='Training acc')\n",
    "plt.plot(epochs, val_acc, 'b', label='Validation acc')\n",
    "plt.title('Training and validation accuracy')\n",
    "plt.legend()\n",
    "\n",
    "plt.figure()\n",
    "\n",
    "plt.plot(epochs, loss, 'bo', label='Training loss')\n",
    "plt.plot(epochs, val_loss, 'b', label='Validation loss')\n",
    "plt.title('Training and validation loss')\n",
    "plt.legend()\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "We reach a validation accuracy of about 90%, much better than what we could achieve in the previous section with our small model trained from \n",
    "scratch. However, our plots also indicate that we are overfitting almost from the start -- despite using dropout with a fairly large rate. \n",
    "This is because this technique does not leverage data augmentation, which is essential to preventing overfitting with small image datasets.\n",
    "\n",
    "Now, let's review the second technique we mentioned for doing feature extraction, which is much slower and more expensive, but which allows \n",
    "us to leverage data augmentation during training: extending the `conv_base` model and running it end-to-end on the inputs. Note that this \n",
    "technique is in fact so expensive that you should only attempt it if you have access to a GPU: it is absolutely intractable on CPU. If you \n",
    "cannot run your code on GPU, then the previous technique is the way to go.\n",
    "\n",
    "Because models behave just like layers, you can add a model (like our `conv_base`) to a `Sequential` model just like you would add a layer. \n",
    "So you can do the following:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from keras import models\n",
    "from keras import layers\n",
    "\n",
    "model = models.Sequential()\n",
    "model.add(conv_base)\n",
    "model.add(layers.Flatten())\n",
    "model.add(layers.Dense(256, activation='relu'))\n",
    "model.add(layers.Dense(1, activation='sigmoid'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is what our model looks like now:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "vgg16 (Model)                (None, 4, 4, 512)         14714688  \n",
      "_________________________________________________________________\n",
      "flatten_1 (Flatten)          (None, 8192)              0         \n",
      "_________________________________________________________________\n",
      "dense_3 (Dense)              (None, 256)               2097408   \n",
      "_________________________________________________________________\n",
      "dense_4 (Dense)              (None, 1)                 257       \n",
      "=================================================================\n",
      "Total params: 16,812,353\n",
      "Trainable params: 16,812,353\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see, the convolutional base of VGG16 has 14,714,688 parameters, which is very large. The classifier we are adding on top has 2 \n",
    "million parameters.\n",
    "\n",
    "Before we compile and train our model, a very important thing to do is to freeze the convolutional base. \"Freezing\" a layer or set of \n",
    "layers means preventing their weights from getting updated during training. If we don't do this, then the representations that were \n",
    "previously learned by the convolutional base would get modified during training. Since the `Dense` layers on top are randomly initialized, \n",
    "very large weight updates would be propagated through the network, effectively destroying the representations previously learned.\n",
    "\n",
    "In Keras, freezing a network is done by setting its `trainable` attribute to `False`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "This is the number of trainable weights before freezing the conv base: 30\n"
     ]
    }
   ],
   "source": [
    "print('This is the number of trainable weights '\n",
    "      'before freezing the conv base:', len(model.trainable_weights))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "conv_base.trainable = False"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "This is the number of trainable weights after freezing the conv base: 4\n"
     ]
    }
   ],
   "source": [
    "print('This is the number of trainable weights '\n",
    "      'after freezing the conv base:', len(model.trainable_weights))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With this setup, only the weights from the two `Dense` layers that we added will be trained. That's a total of four weight tensors: two per \n",
    "layer (the main weight matrix and the bias vector). Note that in order for these changes to take effect, we must first compile the model. \n",
    "If you ever modify weight trainability after compilation, you should then re-compile the model, or these changes would be ignored.\n",
    "\n",
    "Now we can start training our model, with the same data augmentation configuration that we used in our previous example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found 2000 images belonging to 2 classes.\n",
      "Found 1000 images belonging to 2 classes.\n",
      "Epoch 1/30\n",
      "74s - loss: 0.4465 - acc: 0.7810 - val_loss: 0.2056 - val_acc: 0.9120\n",
      "Epoch 2/30\n",
      "72s - loss: 0.2738 - acc: 0.8905 - val_loss: 0.1239 - val_acc: 0.9550\n",
      "Epoch 3/30\n",
      "72s - loss: 0.2088 - acc: 0.9145 - val_loss: 0.1194 - val_acc: 0.9560\n",
      "Epoch 4/30\n",
      "72s - loss: 0.1835 - acc: 0.9280 - val_loss: 0.1025 - val_acc: 0.9550\n",
      "Epoch 5/30\n",
      "72s - loss: 0.1642 - acc: 0.9330 - val_loss: 0.0903 - val_acc: 0.9680\n",
      "Epoch 6/30\n",
      "72s - loss: 0.1360 - acc: 0.9410 - val_loss: 0.0794 - val_acc: 0.9740\n",
      "Epoch 7/30\n",
      "72s - loss: 0.1426 - acc: 0.9465 - val_loss: 0.0968 - val_acc: 0.9560\n",
      "Epoch 8/30\n",
      "72s - loss: 0.1013 - acc: 0.9580 - val_loss: 0.1411 - val_acc: 0.9430\n",
      "Epoch 9/30\n",
      "72s - loss: 0.1177 - acc: 0.9500 - val_loss: 0.2105 - val_acc: 0.9310\n",
      "Epoch 10/30\n",
      "72s - loss: 0.0949 - acc: 0.9620 - val_loss: 0.0900 - val_acc: 0.9710\n",
      "Epoch 11/30\n",
      "72s - loss: 0.0915 - acc: 0.9655 - val_loss: 0.1204 - val_acc: 0.9630\n",
      "Epoch 12/30\n",
      "72s - loss: 0.0782 - acc: 0.9645 - val_loss: 0.0995 - val_acc: 0.9650\n",
      "Epoch 13/30\n",
      "72s - loss: 0.0717 - acc: 0.9755 - val_loss: 0.1269 - val_acc: 0.9580\n",
      "Epoch 14/30\n",
      "72s - loss: 0.0670 - acc: 0.9715 - val_loss: 0.0994 - val_acc: 0.9680\n",
      "Epoch 15/30\n",
      "71s - loss: 0.0718 - acc: 0.9735 - val_loss: 0.0558 - val_acc: 0.9790\n",
      "Epoch 16/30\n",
      "72s - loss: 0.0612 - acc: 0.9780 - val_loss: 0.0870 - val_acc: 0.9690\n",
      "Epoch 17/30\n",
      "71s - loss: 0.0693 - acc: 0.9765 - val_loss: 0.0972 - val_acc: 0.9720\n",
      "Epoch 18/30\n",
      "71s - loss: 0.0596 - acc: 0.9785 - val_loss: 0.0832 - val_acc: 0.9730\n",
      "Epoch 19/30\n",
      "71s - loss: 0.0497 - acc: 0.9800 - val_loss: 0.1160 - val_acc: 0.9610\n",
      "Epoch 20/30\n",
      "71s - loss: 0.0546 - acc: 0.9780 - val_loss: 0.1057 - val_acc: 0.9660\n",
      "Epoch 21/30\n",
      "71s - loss: 0.0568 - acc: 0.9825 - val_loss: 0.2012 - val_acc: 0.9500\n",
      "Epoch 22/30\n",
      "71s - loss: 0.0493 - acc: 0.9830 - val_loss: 0.1384 - val_acc: 0.9610\n",
      "Epoch 23/30\n",
      "71s - loss: 0.0328 - acc: 0.9905 - val_loss: 0.1281 - val_acc: 0.9640\n",
      "Epoch 24/30\n",
      "71s - loss: 0.0524 - acc: 0.9860 - val_loss: 0.0846 - val_acc: 0.9760\n",
      "Epoch 25/30\n",
      "71s - loss: 0.0422 - acc: 0.9845 - val_loss: 0.1002 - val_acc: 0.9670\n",
      "Epoch 26/30\n",
      "71s - loss: 0.0617 - acc: 0.9825 - val_loss: 0.0858 - val_acc: 0.9760\n",
      "Epoch 27/30\n",
      "71s - loss: 0.0568 - acc: 0.9830 - val_loss: 0.0889 - val_acc: 0.9700\n",
      "Epoch 28/30\n",
      "71s - loss: 0.0296 - acc: 0.9915 - val_loss: 0.1406 - val_acc: 0.9620\n",
      "Epoch 29/30\n",
      "71s - loss: 0.0432 - acc: 0.9890 - val_loss: 0.1535 - val_acc: 0.9650\n",
      "Epoch 30/30\n",
      "71s - loss: 0.0354 - acc: 0.9885 - val_loss: 0.1832 - val_acc: 0.9510\n"
     ]
    }
   ],
   "source": [
    "from keras.preprocessing.image import ImageDataGenerator\n",
    "\n",
    "train_datagen = ImageDataGenerator(\n",
    "      rescale=1./255,\n",
    "      rotation_range=40,\n",
    "      width_shift_range=0.2,\n",
    "      height_shift_range=0.2,\n",
    "      shear_range=0.2,\n",
    "      zoom_range=0.2,\n",
    "      horizontal_flip=True,\n",
    "      fill_mode='nearest')\n",
    "\n",
    "# Note that the validation data should not be augmented!\n",
    "test_datagen = ImageDataGenerator(rescale=1./255)\n",
    "\n",
    "train_generator = train_datagen.flow_from_directory(\n",
    "        # This is the target directory\n",
    "        train_dir,\n",
    "        # All images will be resized to 150x150\n",
    "        target_size=(150, 150),\n",
    "        batch_size=20,\n",
    "        # Since we use binary_crossentropy loss, we need binary labels\n",
    "        class_mode='binary')\n",
    "\n",
    "validation_generator = test_datagen.flow_from_directory(\n",
    "        validation_dir,\n",
    "        target_size=(150, 150),\n",
    "        batch_size=20,\n",
    "        class_mode='binary')\n",
    "\n",
    "model.compile(loss='binary_crossentropy',\n",
    "              optimizer=optimizers.RMSprop(lr=2e-5),\n",
    "              metrics=['acc'])\n",
    "\n",
    "history = model.fit_generator(\n",
    "      train_generator,\n",
    "      steps_per_epoch=100,\n",
    "      epochs=30,\n",
    "      validation_data=validation_generator,\n",
    "      validation_steps=50,\n",
    "      verbose=2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "model.save('cats_and_dogs_small_3.h5')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's plot our results again:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAEICAYAAACzliQjAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xl4FdX5wPHvCwIh7CQIArKoVERljaAVBFzRirjgAnGv\nxVpRsdr+VKhQFbu41KVWxYpLjSBqqdAKFAMWlaqEQkBABBUwbIZFZFMIeX9/nElyE+9N5ib35m7v\n53nuk3tnzsw9M3PzzplzzpwRVcUYY0xqqBPrDBhjjKk9FvSNMSaFWNA3xpgUYkHfGGNSiAV9Y4xJ\nIRb0jTEmhVjQT0EiUldE9ohIh0imjSUROUZEIt7/WETOFJF1AZ9Xi8gAP2mr8V1/FZF7qru8MX4c\nFusMmKqJyJ6Aj+nA98Ah7/ONqpoTzvpU9RDQONJpU4GqHhuJ9YjIDcCVqjooYN03RGLdxlTGgn4C\nUNXSoOuVJG9Q1XdCpReRw1S1qDbyZkxV7PcYX6x6JwmIyAMi8pqITBGR3cCVInKKiHwoIt+IyGYR\neUJE6nnpDxMRFZFO3udXvPmzRGS3iPxXRDqHm9abf66IfCYiu0TkSRH5QESuDZFvP3m8UUTWishO\nEXkiYNm6IvInEdkuIl8AQyrZP2NFZGqFaU+JyKPe+xtEZJW3PZ97pfBQ6yoQkUHe+3QR+ZuXtxVA\nnwppx4nIF956V4jIBd70E4E/AwO8qrNtAft2QsDyP/e2fbuI/ENEjvCzb8LZzyX5EZF3RGSHiGwR\nkV8HfM9vvH3yrYjkiUjbYFVpIvJ+yXH29ucC73t2AONEpIuIzPe+Y5u335oFLN/R28ZCb/7jIpLm\n5fm4gHRHiMg+EckItb2mCqpqrwR6AeuAMytMewA4AAzFncgbAicB/XBXc0cBnwGjvfSHAQp08j6/\nAmwDsoB6wGvAK9VIeziwGxjmzfslcBC4NsS2+MnjW0AzoBOwo2TbgdHACqA9kAEscD/noN9zFLAH\naBSw7q+BLO/zUC+NAKcD+4Hu3rwzgXUB6yoABnnvHwbeBVoAHYGVFdJeBhzhHZORXh5ae/NuAN6t\nkM9XgAne+7O9PPYE0oC/APP87Jsw93MzYCtwG9AAaAr09ebdDeQDXbxt6Am0BI6puK+B90uOs7dt\nRcBNQF3c7/FHwBlAfe938gHwcMD2fOLtz0Ze+lO9eZOAiQHfcwcwPdb/h4n8inkG7BXmAQsd9OdV\nsdydwOve+2CB/JmAtBcAn1Qj7fXAewHzBNhMiKDvM48nB8z/O3Cn934BrpqrZN55FQNRhXV/CIz0\n3p8LrK4k7T+Bm733lQX9DYHHAvhFYNog6/0E+In3vqqg/xLwYMC8prh2nPZV7Zsw9/NVwKIQ6T4v\nyW+F6X6C/hdV5GF4yfcCA4AtQN0g6U4FvgTE+7wUuDjS/1ep9LLqneTxVeAHEekqIv/yLte/Be4D\nMitZfkvA+31U3ngbKm3bwHyo+y8tCLUSn3n09V3A+kryC/AqMMJ7P9L7XJKP80XkI6/q4RtcKbuy\nfVXiiMryICLXiki+V0XxDdDV53rBbV/p+lT1W2An0C4gja9jVsV+PhIX3IOpbF5VKv4e24jINBHZ\n6OXhxQp5WKeu00A5qvoB7qqhv4icAHQA/lXNPBmsTj+ZVOyu+CyuZHmMqjYF7sWVvKNpM64kCoCI\nCOWDVEU1yeNmXLAoUVWX0mnAmSLSDlf99KqXx4bAG8DvcFUvzYF/+8zHllB5EJGjgKdxVRwZ3no/\nDVhvVd1LN+GqjErW1wRXjbTRR74qqmw/fwUcHWK5UPP2enlKD5jWpkKaitv3B1yvsxO9PFxbIQ8d\nRaRuiHy8DFyJuyqZpqrfh0hnfLCgn7yaALuAvV5D2I218J3/BHqLyFAROQxXT9wqSnmcBowRkXZe\no97/VZZYVbfgqiBexFXtrPFmNcDVMxcCh0TkfFzds9883CMizcXdxzA6YF5jXOArxJ3/foYr6ZfY\nCrQPbFCtYArwUxHpLiINcCel91Q15JVTJSrbzzOADiIyWkQaiEhTEenrzfsr8ICIHC1OTxFpiTvZ\nbcF1GKgrIqMIOEFVkoe9wC4RORJXxVTiv8B24EFxjeMNReTUgPl/w1UHjcSdAEwNWNBPXncA1+Aa\nVp/FNbhGlapuBS4HHsX9Ex8NLMGV8CKdx6eBXGA5sAhXWq/Kq7g6+tKqHVX9BrgdmI5rDB2OO3n5\nMR53xbEOmEVAQFLVZcCTwMdemmOBjwKWnQusAbaKSGA1Tcnys3HVMNO95TsA2T7zVVHI/ayqu4Cz\ngEtwJ6LPgIHe7IeAf+D287e4RtU0r9ruZ8A9uEb9YypsWzDjgb64k88M4M2APBQB5wPH4Ur9G3DH\noWT+Otxx/l5VF4a57aaCksYRYyLOu1zfBAxX1fdinR+TuETkZVzj8IRY5yXR2c1ZJqJEZAiup8x+\nXJe/g7jSrjHV4rWPDANOjHVekoFV75hI6w98gavLPge4yBreTHWJyO9w9wo8qKobYp2fZGDVO8YY\nk0KqLOmLyGQR+VpEPgkxX7zbrdeKyDIR6R0w7xoRWeO9rolkxo0xxoSvypK+iJyGu338ZVU9Icj8\n84BbcHdE9gMeV9V+XteuPNzt+gosBvqo6s7Kvi8zM1M7depUjU0xxpjUtXjx4m2qWlkXacBHQ66q\nLhBvsK0QhuFOCAp86PVZPgIYBMxV1R0AIjIXNyjWlMq+r1OnTuTl5VWVLWOMMQFEpKq70oHINOS2\no/wt1wXetFDTf0BERnkj+OUVFhZGIEvGGGOCiYveO6o6SVWzVDWrVasqr06MMcZUUySC/kbKjz/S\n3psWaroxxpgYiUTQnwFc7fXiORnYpaqbgTnA2SLSQkRa4EYunBOB7zPGGFNNVTbkisgUXKNspogU\n4MbQqAegqs8Ab+N67qzFDe96nTdvh4jcjxsXBeC+kkZdY4wxseGn986IKuYrcHOIeZOBydXLmjHG\nxJ+cHBg7FjZsgA4dYOJEyK7uUHgxEBcNucaY5JWTA506QZ067m9OTqxzVH05OTBqFKxfD6ru76hR\nibVNFvSNMVGTDEEy0NixsG9f+Wn79rnpicKCvjEmapIhSAbaEGLIt1DT/V7l1ObVkA2tbIyJmnCD\nZLzr0MFdrQSbXlHJVU7JSa/kKgfKtwH4TRcpVtI3xkRNsGAYanoi1P1PnAjp6eWnpae76RX5vcqp\n7ashC/rGmKjxGyQTpe4/OxsmTYKOHUHE/Z00KXiJ3O9VTm1fDVnQN8ZEjd8gGW5pN5ZXBdnZsG4d\nFBe7v6GqYPxe5YRzNRQJFvSNMWELJ+j6CZLhlHajdVUQ6ROJ36uccKqMIkJV4+rVp08fNcbExiuv\nqHbsqCri/r7ySvA06emqLuS6V3p68LR+dexYfn0lr44da5bWz/ZEa5vC/X4/6SoD5KmPGBvzIF/x\nZUHfmNjwG/jCCbqR/m5VFxiDfb9I9dcZjW2qbX6DvlXvGJOgIl0d4bdePRoNj+E0kPqtAw+nnSDZ\nupZWxoK+MXEknJt5Il2v7TfwRavh0W8Dqd868HACeW03psaSBX1jqinSJe1wAnk4pVi/+fQb+Gq9\n4bECv1cF4QTyWG9TrfJTB1SbL6vTN7G2fr3qpZeqzp8fOk24DX9+GurCqVeORr12uGlr2vAYbdE4\nRvEMa8g1qSDS/6hffqnaqZP7z6hfX/W114KnC7cHiZ/g4zeQh/P94TZQJnrgqyjZtqcyFvRN0ot0\nN7vPP1ft0EG1RQvVuXNVTz3VBYvHHvth2lgH6GicSExi8xv0rU7fJKxIjlmydi0MHAh798Kdd8IN\nN8AHH0BaGowZA7/+tWtgLBFOfbHfBsVw6pWjUa8da0VFcPXVcPrpsDFCT9PeuRMuvBD69oU5c9wp\nL+X5OTPU5stK+mWKi1WLimKdi/gVqVLs6tWqbduqZmaqPvjgD0vQhx3m/mZnq37/vVsmWn3AI10d\nEa2bjiKtuFj1uutc/ho0UG3TRvWDD2q2zk8+UT3mGNV69VSPPNKte/Bg1Y8/jkye4w1WvZO4iotV\nZ85UPfFE1R/9SHXPnljnKD5FIpiuXOkCzOGHqy5fHnqdzZu7v2eeqfrtt5WvM9h3+w28RUWqW7eq\nfvddDXeOj22PF8XFqr/8pdsv48e7YH300S5YT5pUvXVOn67auLFq69aq77/v9ufjj7sTO6gOH+5O\n9snEgn6CWrhQdcCAsuAFqr/6VaxzFZ/8BtNQ6X7/exfs27RxwV+18quHF15QrVtXtVcv1c2bw8vr\nyy+rHnGEW1dGhuoVV7hAl53tTiTdu7sAVaeOS9Ohg+q6dRHZTXHvgQfcNt96qzsBqKru2KF6zjlu\n+s9/XnaFVZVDh1Tvvdct17evakFB+fm7drn5jRq5Y3njjaqbNkV2e2LFgn6CWblS9cIL3RFp3Vr1\nL39RPXBA9ac/dT/O/PxY5zC0b79V3bgxNt9dk66Qdeq4QPzpp1WnLbl6ePttd8Lo3LnqkuKXX7qS\n6qWXukBfcZ0l6zn5ZNVhw1RHjVIdN0714YfdlUWXLq7Un8z+/Ge3L66+2gXsQEVFqv/3f25+//6q\nW7ZUvq5du1QvuMClv/Za1f37Q6fdskV19GhXdZeernrPParffFPz7YklC/oJ4quvXGCvU0e1SRPV\n++9X3b27bP62be6S9OSTf/hPES+uvtpdij/8cHzmMVTpHVQ/+6x8Wj9XDx9/rNqqlTsuH35YNn3H\nDtU333Ql06OPLlu+bVvVa65xpf0PPlBdu7b8MQ7mgw9UGzZU7dlTdefOiO2KuPLKK27/XHCB6sGD\nodNNmeL2Rbt2oevjV69W7drVFZCeeKLsiqEqa9eqjhjh8tGypeojj7jCViKyoB/ntm931TZpaa4/\n+JgxqoWFwdO+9JI7Us8+W7t59KOoyHVxbNrU5fGss+LvcjlU6b1t2+Dp/Vw9rFmjetRR7oRw662u\nKqGkaqZxY9Xzz3d1yCtX+g9AFc2e7U6m/fur7t1bvXXEq5kzXYAePLjyEnmJJUvcsWjQwP0/BPrn\nP93vLzOz8hvqKrN4serZZ7vjd9JJ7vgmGgv6caq4uOzyXUT1qqtcNUBVywwe7Jap6hK3tv33v+5X\nNGWK6jPPuBJZZqbqjBmxzlmZYKX3tLSaN2hu2eICRN26qqec4uqK33svsiXFadPc7+Tcc/3Xa8e7\nd991+z8rq6xR3I/CQvd/AKq33eb288SJbv/06hWZNpA33nCFmEaNVCdPrv4JOxYs6Mep8ePdXj/3\n3PDq6T/91F0RZGdHLWvVMmGC+6fbts19XrnSVUmA6i9+obpvX1naWPYi+dWvyqp52rWL3HcXF0e/\nFD5pksv35ZcnfhfevDxXjXnccaGvbCtz8KC7KoayhvERIyJ7DDZsUB00yK37sstctV0isKAfh0qq\naa67rnoliJJeCXPnRj5v1XXyyar9+pWf9t13ZV3wunVzJ7dYjuvy4ouuwa5nz/irevLrj3/U0p4s\n4f52vvvO/fZefz06efNr1Sp3Fdixo2vLqomXXnI9rx56KDql8aIi1d/9zv1ujjxS9T//ifx3RJoF\n/Tgzf76rnz399Opfpu/f72426dLFXz1otG3f7uqx7703+Pw5c1x3yPr13SVzVX3qDxxw3Sjr1/d3\ncqhKcbG72QpUzzjD9e5IZHfd5bbl7rv9pd+2zXUMaN1aS3sr1fSGp+pav161fXuXl0SqL//4Y/c/\nV6eO61kVTtVdQYErcIwZE34X3+qwoB9HVq1y9fHHHVfznhhz57qjFirQ1qbXXnN5adMmdKn8669V\nhw4NHvBLXldf7UrhFYN9VTdcVaaoSPXmm92yI0cmR314cbHrVw6u5B/K6tXuiqBhQ5f2nHNU33rL\nDSR31FHh1aPXNL8rV7reNEcf7f4Hli6tne+OpN27y+4W7tfP9fgJ5ttvXVvWrbe6//XA3++Pfxz9\n36AF/Tjx9dfuH+3ww1W/+CIy6xw50gXIwP7lsTBw4A+Dc7BSeXGx6w4XKqC3bas6ZIjqr39d+cnB\nb5/1/ftVL7nELXPHHfHZjbS6iopc3T6oPvdc2fTiYtdAOnSoOwHXr696/fXuLuMSCxa4eTfcEL38\nbdmimpPj+sm3b1927Lp0id1VRqS89ppqs2aud9bLL7v2hQ8+cO1a/fuXDdfRsKE70T70kDvJTZ2q\npVVz0WRBPw7s3+96daSlle/PXVNbtrhS06BBsetdUFzseq34LZW/8orbD4HpGjRQffrp8ulCda8E\nV0X0179WHsR37Ci7o/nRRyO5xfHj++9dRwAR1VdfdUG2Tx+3zRkZqr/5TeheXiVVRG+9FZm87N2r\nOmuWO7l2717+WA0f7roZR6qwEw/Wry/7fZVcSYm4nkh3362amxu86rXkJrPAE3WkWdCPsUOHykpk\nb7wR+fU/84xbd8U+y7Vl+fLQwTnUgGd+GmhDNfj+4Q+qp53mPg8YUDZsQqANG1SPP961nUyZEsmt\njT9797qhn0v20bHHut9EYG+pYL7/3nVvbNWqZt1/Dx1y3SZLquTq13ftVQ8+qLpoUeL3MqpMUZHq\nn/7kSu7TppX1XKtqmbPPdvspkgXAQBb0Y+yee7TKuteaOHTIXUVkZvr70alGtlfMQw+FDvrh1r/7\nzeehQ6rPP+9KkfXquRJtSalq+XLXFbNJE1faSgU7d7peUjNnhleFtWKFu+r6yU+qd6VYVOSqb8D9\nnT07+W4ei4bt292wG23bRqdh14J+DD3/vNuzo0ZFt/olP99Vsfz0p1WnjfQQu2ee6epsYzFs79at\nqldeqaV1xU884aq7jjgiMRsKY+Hxx93+e+aZ8JYrKnI3FIKryzbhyc93/yP9+0e+YdeCfozMnesa\ndM4+u3bG8PjVr9xRXLCg8nThPjavMnv2uMvUO+6I7Q1Xc+eWjXFz7LGpMyplJBw65E7c6ek/HH8o\nlIMHy8apuf/+6OYvmU2Z4vbhL34R2fVa0K+h4mJXbfLpp24EST8B/JNP3BggJ5xQeyP27dnjgm23\nbpWXHCL52Lx//cst++9/VzvbEbNvn+tJsX17rHOSeAoKXFVZv36VD3im6n7/l17qjvvvflc7+Utm\nJYW155+P3Dr9Bv3DavtJXfFg71748EPYsgW2bg3++vpr9/i2QBkZ0Lp12evww8veZ2S4x+qlp8O/\n/gXNmtXOtjRqBH/+Mwwd6h7p99hjwdN16ADr1wefHq7Zs6FhQxgwIPxlI61hQ7jqqljnIjG1awfP\nPAOXX+4eyTh+fPB0Bw7AiBHw97/DQw+5x0mamnnwQViyBG66CU44wT3OsbaIO0HEj6ysLM3Ly4vq\nd9x0k/uxl6hfv3wwDwzqmZmwZ0/ok8Pu3WXrSU+HBQugT5+oZj+oMWPg8cfhySdh9Ogfzs/JgVGj\nyj9TNj09+HNVq3LssXD00fD22zXLs4kPV10FU6a4ZwL361d+3oEDcNll8NZb8Kc/ud+ZiYzt2yEr\nCw4ehMWLXcypCRFZrKpZVSb0czlQm6/aqN7p3NnVZ376qesBUZPG1n373CiZH37ougzGSlGRuzGn\nTh031Gwwkah//+ILd1n6+OM1ya2JJ998457U1aVL+UdzfvedGyIaVJ98Mnb5S2ZLlrj+/gMG1LwN\nkEjW6QNDgNXAWuCuIPM7ArnAMuBdoH3AvEPAUu81o6rvinbQLwlaTzwR1a+Jid27VXv3dsPCLllS\ns3WFOkE8/bTbf8n2fNFU9+677liX3DW6f7+7AQzcU9xM9Lz6qtvPt9xSs/VELOgDdYHPgaOA+kA+\n0K1CmteBa7z3pwN/C5i3x09GSl7RDvrPPee2esWKqH5NzGzc6EYFbNu2+iMZVta9c9gwN4ZLIo0z\nbvwpaVx8/fWyB4pU98HkJjx33OH29wsvVH8dkQz6pwBzAj7fDdxdIc0K4EjvvQDfBsyLq6A/YoQb\nICyZg1Z+vrtJqWfP6g2uFap7Z4cObr3RHkPExMZ335UNpSAS2Z4lpnIHD7o7mvv2rf5YUX6Dfh0f\n7QPtgK8CPhd40wLlAxd77y8CmohIhvc5TUTyRORDEbkw2BeIyCgvTV5hYaGPLFWPKsybB6efDiJR\n+5qY694dXn8dli+HK674YS+kqmzYEHr67t1wzjk1z6OJPw0auAb/Y4+FF1+E66+PdY5Sx2GHuf/Z\nefOgjp+oXAORWv2dwEARWQIMBDbi6vIBOqprUR4JPCYiR1dcWFUnqWqWqma1atUqQln6oZUrXY+b\nM86I2lfERE4OdOrkfiydOrnP55wDTz3letiMGeNOeH6F6sbZtKn7cZ5+eiRybeLRCSfAp5/C1VfH\nOiepp2VL1wU72vz0098IHBnwub03rZSqbsIr6YtIY+ASVf3Gm7fR+/uFiLwL9MK1EdS63Fz3N5mC\nVsWumOvXu88AN94Ia9fCww/DMcf47243cWLw7p0tWkDPni74G2MSk5+S/iKgi4h0FpH6wBXAjMAE\nIpIpIiXruhuY7E1vISINStIApwIrI5X5cM2bB0cd5UrDyWLs2PLBGdznsWPd+z/8AS6+GH75S9fX\n2o/sbNd/v2NHVw3WsaO7KWf9ehgyJLL5N8bUriqDvqoWAaOBOcAqYJqqrhCR+0TkAi/ZIGC1iHwG\ntAYmetOPA/JEJB+YD/xeVWMS9IuK4N13k6uUD5XXv4Or8vnb3+Ckk2DkSPB731t2NqxbB8XF7m+T\nJm661ecbk9h8DcOgqm8Db1eYdm/A+zeAN4IstxA4sYZ5jIglS2DXruSrz/czvEJ6OsyY4e62HDoU\nPvoo/OEX5sxxdyj37Fmz/BpjYivK7cTxo6Q+f/Dg2ObDr2CNs8FMnOiCeqD0dDc9UOvWrlF3/344\n7zzYscN/XoqLXdA/++zo9ywwxkRXyvwLz5vneibUdHyL2lDSOLt+vet1U9I4GyzwB6t/DzWeTrdu\nMH06rFnjAv+ePf7ys2QJbNtm9fnGJIOUCPrffw/vv5849flVNc5WVLH+vbIB1AYPhtdeg0WL4KKL\n3L6pyuzZ7u9ZZ/nJvTEmnqVE0P/wQ1etkSj1+VU1ztbUhRfC5Mnwzjuucbeqm7fmzHEjhx5+eGS+\n3xgTOykR9HNzXV30aafFOif+hGpkrc7Y96Fcc40be//vf3dVR6Fu3tq1CxYutF47xiSLlAn6WVnQ\nvHmsc+KP38bZmrrtNrj3XnjhBfdgjGCBf948OHTI6vONSRZJH/R374aPP06cqh0Ir3G2piZMgFtu\ngUcfdU/zqWj2bNdH/+STI//dxpjal/RB/733XJ11vDTi+u2KGU7jbE2IuGqeq66CcePgL38pm6fq\n6vPPPBPq1YvO9xtjalfSPyN33jz3OMRTT411TiofJydaQd2POnXg+edd/f3o0a4abORIWL3a5fHu\nu2OXN2NMZCV9ST83F378Y/cA7VgLtytmbapXz3XlHDjQjbD4z3+6Uj5YI64xySSpg/727bB0afzU\n50e7K2ZNpaW54Rp69YJLL4Wnn4auXZNrgDpjUl1SB/35893feKnPr42umDXVpAnMmuUC/erVVso3\nJtkkddCfNw8aN3YjTEZTpMfJibXMTJg7F4YPd2PyG2OSR1I35ObmuhuyotnzJJzG2ZLPY8e6Kp0O\nHVzAj2Ujbijt27vHtxljkotoOM/RqwVZWVma53fQ90oUFMCRR8Ijj7gHiERLp07Bhzbu2NF1tTTG\nmNogIou9R9NWKmmrd+bNc3+jXZ8f742zxhgTKKmDfkYGdO8e3e9JhMZZY4wpkZRBX9XV5w8eHP2H\nfiRK46wxxkCSBv21a12dfm30z6/NcXKMMaamkrL3TsmjEWvrpqzsbAvyxpjEkJQl/dxc1+XwmGNi\nnRNjjIkvSRf0i4vdnbhnnOGqW4wxxpRJuqC/bJkbcydehl4wxph4knRBv7b65xtjTCJKuqCfmws/\n+pGr0zfGGFNeUgX9gwdhwYLI9drxO5CaMcYkiqTqsrloEezZE5mqnXh9ypUxxtREUpX0S+rzBw+u\n+bri+SlXxhhTXUkV9HNzoWdPN+ZOTdlAasaYZJQ0QX//fli4MHL1+TaQmjEmGSVN0N+1C664As4/\nPzLrs4HUjDHJKGkactu0gZdeitz6EukpV8YY41fSBP1osIHUjDHJJmmqd4wxxlTNgr4xxqQQC/rG\nGJNCLOgbY0wKsaBvjDEpxFfQF5EhIrJaRNaKyF1B5ncUkVwRWSYi74pI+4B514jIGu91TSQzb4wx\nJjxVBn0RqQs8BZwLdANGiEi3CskeBl5W1e7AfcDvvGVbAuOBfkBfYLyItIhc9o0xxoTDT0m/L7BW\nVb9Q1QPAVGBYhTTdAG+4M+YHzD8HmKuqO1R1JzAXGFLzbBtjjKkOP0G/HfBVwOcCb1qgfOBi7/1F\nQBMRyfC5LCIySkTyRCSvsLDQb96NMcaEKVINuXcCA0VkCTAQ2Agc8ruwqk5S1SxVzWrVqlWEsmSM\nMaYiP8MwbASODPjc3ptWSlU34ZX0RaQxcImqfiMiG4FBFZZ9twb5NcYYUwN+SvqLgC4i0llE6gNX\nADMCE4hIpoiUrOtuYLL3fg5wtoi08Bpwz/amGWOMiYEqg76qFgGjccF6FTBNVVeIyH0icoGXbBCw\nWkQ+A1oDE71ldwD3404ci4D7vGnGGGNiQFQ11nkoJysrS/Py8mKdDWOMSSgislhVs6pKl3J35Obk\nQKdOUKeO+5uTE+scGWNM7Ump8fRzcmDUqLIHnq9f7z6DjZtvjEkNKVXSHzu2LOCX2LfPTTfGmFSQ\nUkF/w4bwphtjTLJJqaDfoUN4040xJtmkVNCfOBHS08tPS093040xJhWkVNDPzoZJk6BjRxBxfydN\nskZcY0zqSKneO+ACvAV5Y0yqSqmSvjHGpDoL+sYYk0Is6BtjTAqxoG+MMSnEgr4xxqQQC/rGGJNC\nLOgbY0wKsaBvjDEpxIK+McakEAv6xhiTQizoG2NMCrGgb4wxKcSCvjHGpBAL+sYYk0Is6BtjTAqx\noG+MMSnEgr4xxqQQC/rGGJNCLOgbY0wKsaBvjDEpxIK+McakEAv6xhiTQizoG2NMCrGgb4wxKcSC\nvjHGpBBbJzS3AAARfUlEQVQL+sYYk0Is6BtjTAqxoG+MMSnEgr4xxqQQC/rGGJNCfAV9ERkiIqtF\nZK2I3BVkfgcRmS8iS0RkmYic503vJCL7RWSp93om0htgjDHGv8OqSiAidYGngLOAAmCRiMxQ1ZUB\nycYB01T1aRHpBrwNdPLmfa6qPSObbWOMMdXhp6TfF1irql+o6gFgKjCsQhoFmnrvmwGbIpdFY4wx\nkeIn6LcDvgr4XOBNCzQBuFJECnCl/FsC5nX2qn3+IyIDgn2BiIwSkTwRySssLPSfe2OMMWGJVEPu\nCOBFVW0PnAf8TUTqAJuBDqraC/gl8KqINK24sKpOUtUsVc1q1apVhLJkjDGmIj9BfyNwZMDn9t60\nQD8FpgGo6n+BNCBTVb9X1e3e9MXA58CPapppY4wx1eMn6C8CuohIZxGpD1wBzKiQZgNwBoCIHIcL\n+oUi0sprCEZEjgK6AF9EKvPGGGPCU2XvHVUtEpHRwBygLjBZVVeIyH1AnqrOAO4AnhOR23GNuteq\nqorIacB9InIQKAZ+rqo7orY1xhhjKiWqGus8lJOVlaV5eXmxzoYxxiQUEVmsqllVpbM7co0xJoVY\n0DfGmBRiQd8YY1KIBX1jjEkhFvSNMSaFWNA3xpgUYkHfGGNSiAV9Y4xJIRb0jTEmhVjQN8aYFGJB\n3xhjUogFfWOMSSEW9I0xJoVY0DfGmBRiQd8YY1KIBX1jjEkhFvSNMSaFWNA3xpgUYkHfGGNSiAV9\nY4xJIRb0jTEmhVjQN8aYFGJB3xhjUogFfWOMSSEW9I0xJoVY0DfGmBRiQd8YY1KIBX1jjEkhFvSN\nMSaFWNA3xpgUYkHfGGNSyGGxzoAxJn4cPHiQgoICvvvuu1hnxYSQlpZG+/btqVevXrWWt6BvjClV\nUFBAkyZN6NSpEyIS6+yYClSV7du3U1BQQOfOnau1DqveMcaU+u6778jIyLCAH6dEhIyMjBpdiVnQ\nN8aUYwE/vtX0+FjQN8aYFGJB3xhTbTk50KkT1Knj/ubk1Gx927dvp2fPnvTs2ZM2bdrQrl270s8H\nDhzwtY7rrruO1atXV5rmqaeeIqemmU1Q1pBrjKmWnBwYNQr27XOf1693nwGys6u3zoyMDJYuXQrA\nhAkTaNy4MXfeeWe5NKqKqlKnTvAy6wsvvFDl99x8883Vy2AS8FXSF5EhIrJaRNaKyF1B5ncQkfki\nskRElonIeQHz7vaWWy0i50Qy88aY2Bk7tizgl9i3z02PtLVr19KtWzeys7M5/vjj2bx5M6NGjSIr\nK4vjjz+e++67rzRt//79Wbp0KUVFRTRv3py77rqLHj16cMopp/D1118DMG7cOB577LHS9HfddRd9\n+/bl2GOPZeHChQDs3buXSy65hG7dujF8+HCysrJKT0iBxo8fz0knncQJJ5zAz3/+c1QVgM8++4zT\nTz+dHj160Lt3b9atWwfAgw8+yIknnkiPHj0YG42dVYUqg76I1AWeAs4FugEjRKRbhWTjgGmq2gu4\nAviLt2w37/PxwBDgL976jDEJbsOG8KbX1Keffsrtt9/OypUradeuHb///e/Jy8sjPz+fuXPnsnLl\nyh8ss2vXLgYOHEh+fj6nnHIKkydPDrpuVeXjjz/moYceKj2BPPnkk7Rp04aVK1fym9/8hiVLlgRd\n9rbbbmPRokUsX76cXbt2MXv2bABGjBjB7bffTn5+PgsXLuTwww9n5syZzJo1i48//pj8/HzuuOOO\nCO0d//yU9PsCa1X1C1U9AEwFhlVIo0BT730zYJP3fhgwVVW/V9UvgbXe+owxCa5Dh/Cm19TRRx9N\nVlZW6ecpU6bQu3dvevfuzapVq4IG/YYNG3LuuecC0KdPn9LSdkUXX3zxD9K8//77XHHFFQD06NGD\n448/Puiyubm59O3blx49evCf//yHFStWsHPnTrZt28bQoUMBd0NVeno677zzDtdffz0NGzYEoGXL\nluHviBryE/TbAV8FfC7wpgWaAFwpIgXA28AtYSyLiIwSkTwRySssLPSZdWNMLE2cCOnp5aelp7vp\n0dCoUaPS92vWrOHxxx9n3rx5LFu2jCFDhgTtu16/fv3S93Xr1qWoqCjouhs0aFBlmmD27dvH6NGj\nmT59OsuWLeP666+P+7uZI9V7ZwTwoqq2B84D/iYivtetqpNUNUtVs1q1ahWhLBljoik7GyZNgo4d\nQcT9nTSp+o244fj2229p0qQJTZs2ZfPmzcyZMyfi33Hqqacybdo0AJYvXx70SmL//v3UqVOHzMxM\ndu/ezZtvvglAixYtaNWqFTNnzgTcTW/79u3jrLPOYvLkyezfvx+AHTt2RDzfVfHTe2cjcGTA5/be\ntEA/xdXZo6r/FZE0INPnssaYBJWdXTtBvqLevXvTrVs3unbtSseOHTn11FMj/h233HILV199Nd26\ndSt9NWvWrFyajIwMrrnmGrp168YRRxxBv379Sufl5ORw4403MnbsWOrXr8+bb77J+eefT35+PllZ\nWdSrV4+hQ4dy//33RzzvlZGSluaQCUQOAz4DzsAF7EXASFVdEZBmFvCaqr4oIscBubhqnG7Aq7h6\n/Lbe9C6qeijU92VlZWleXl6NNsoYUz2rVq3iuOOOi3U24kJRURFFRUWkpaWxZs0azj77bNasWcNh\nh8W+p3uw4yQii1U1K8QiparMvaoWichoYA5QF5isqitE5D4gT1VnAHcAz4nI7bhG3WvVnU1WiMg0\nYCVQBNxcWcA3xph4sWfPHs444wyKiopQVZ599tm4CPg15WsLVPVtXANt4LR7A96vBIJeX6nqRCBK\nTTvGGBMdzZs3Z/HixbHORsTZMAzGGJNCLOgbY0wKsaBvjDEpxIK+McakEAv6xpi4MXjw4B/caPXY\nY49x0003Vbpc48aNAdi0aRPDhw8PmmbQoEFU1R38scceY1/AKHLnnXce33zzjZ+sJwwL+saYuDFi\nxAimTp1abtrUqVMZMWKEr+Xbtm3LG2+8Ue3vrxj03377bZo3b17t9cWjxO90aoyJijFjIMhIwjXS\nsyd4IxoHNXz4cMaNG8eBAweoX78+69atY9OmTQwYMIA9e/YwbNgwdu7cycGDB3nggQcYNqz82I/r\n1q3j/PPP55NPPmH//v1cd9115Ofn07Vr19KhDwBuuukmFi1axP79+xk+fDi//e1veeKJJ9i0aROD\nBw8mMzOT+fPn06lTJ/Ly8sjMzOTRRx8tHaXzhhtuYMyYMaxbt45zzz2X/v37s3DhQtq1a8dbb71V\nOqBaiZkzZ/LAAw9w4MABMjIyyMnJoXXr1uzZs4dbbrmFvLw8RITx48dzySWXMHv2bO655x4OHTpE\nZmYmubm5ETsGFvSNMXGjZcuW9O3bl1mzZjFs2DCmTp3KZZddhoiQlpbG9OnTadq0Kdu2bePkk0/m\nggsuCPnM2Keffpr09HRWrVrFsmXL6N27d+m8iRMn0rJlSw4dOsQZZ5zBsmXLuPXWW3n00UeZP38+\nmZmZ5da1ePFiXnjhBT766CNUlX79+jFw4EBatGjBmjVrmDJlCs899xyXXXYZb775JldeeWW55fv3\n78+HH36IiPDXv/6VP/7xjzzyyCPcf//9NGvWjOXLlwOwc+dOCgsL+dnPfsaCBQvo3LlzxMfnsaBv\njAmqshJ5NJVU8ZQE/eeffx5wY97fc889LFiwgDp16rBx40a2bt1KmzZtgq5nwYIF3HrrrQB0796d\n7t27l86bNm0akyZNoqioiM2bN7Ny5cpy8yt6//33ueiii0pH+rz44ot57733uOCCC+jcuTM9e/YE\nQg/fXFBQwOWXX87mzZs5cOAAnTt3BuCdd94pV53VokULZs6cyWmnnVaaJtLDLydNnX6kn9VpjImN\nYcOGkZuby//+9z/27dtHnz59ADeAWWFhIYsXL2bp0qW0bt26WsMYf/nllzz88MPk5uaybNkyfvKT\nn9RoOOSSYZkh9NDMt9xyC6NHj2b58uU8++yzMR1+OSmCfsmzOtevB9WyZ3Va4Dcm8TRu3JjBgwdz\n/fXXl2vA3bVrF4cffjj16tVj/vz5rF+/vtL1nHbaabz66qsAfPLJJyxbtgxwwzI3atSIZs2asXXr\nVmbNmlW6TJMmTdi9e/cP1jVgwAD+8Y9/sG/fPvbu3cv06dMZMGCA723atWsX7dq5R4m89NJLpdPP\nOussnnrqqdLPO3fu5OSTT2bBggV8+eWXQOSHX06KoF+bz+o0xkTfiBEjyM/PLxf0s7OzycvL48QT\nT+Tll1+ma9eula7jpptuYs+ePRx33HHce++9pVcMPXr0oFevXnTt2pWRI0eWG5Z51KhRDBkyhMGD\nB5dbV+/evbn22mvp27cv/fr144YbbqBXr16+t2fChAlceuml9OnTp1x7wbhx49i5cycnnHACPXr0\nYP78+bRq1YpJkyZx8cUX06NHDy6//HLf3+NHlUMr17bqDK1cp44r4VckAsXFEcqYMSnAhlZODDUZ\nWjkpSvq1/axOY4xJVEkR9Gv7WZ3GGJOokiLox/JZncYkm3ir8jXl1fT4JE0//Vg9q9OYZJKWlsb2\n7dvJyMgIedOTiR1VZfv27aSlpVV7HUkT9I0xNde+fXsKCgooLCyMdVZMCGlpabRv377ay1vQN8aU\nqlevXumdoCY5JUWdvjHGGH8s6BtjTAqxoG+MMSkk7u7IFZFCoPJBNSqXCWyLUHbiQbJtDyTfNiXb\n9kDybVOybQ/8cJs6qmqrqhaKu6BfUyKS5+dW5ESRbNsDybdNybY9kHzblGzbA9XfJqveMcaYFGJB\n3xhjUkgyBv1Jsc5AhCXb9kDybVOybQ8k3zYl2/ZANbcp6er0jTHGhJaMJX1jjDEhWNA3xpgUkjRB\nX0SGiMhqEVkrInfFOj+RICLrRGS5iCwVkfAeJxYHRGSyiHwtIp8ETGspInNFZI33t0Us8xiuENs0\nQUQ2esdpqYicF8s8hkNEjhSR+SKyUkRWiMht3vSEPE6VbE8iH6M0EflYRPK9bfqtN72ziHzkxbzX\nRKS+r/UlQ52+iNQFPgPOAgqARcAIVV0Z04zVkIisA7JUNSFvKhGR04A9wMuqeoI37Y/ADlX9vXdy\nbqGq/xfLfIYjxDZNAPao6sOxzFt1iMgRwBGq+j8RaQIsBi4EriUBj1Ml23MZiXuMBGikqntEpB7w\nPnAb8Evg76o6VUSeAfJV9emq1pcsJf2+wFpV/UJVDwBTgWExzlPKU9UFwI4Kk4cBL3nvX8L9QyaM\nENuUsFR1s6r+z3u/G1gFtCNBj1Ml25Ow1NnjfaznvRQ4HXjDm+77GCVL0G8HfBXwuYAEP9AeBf4t\nIotFZFSsMxMhrVV1s/d+C9A6lpmJoNEissyr/kmIqpCKRKQT0Av4iCQ4ThW2BxL4GIlIXRFZCnwN\nzAU+B75R1SIvie+YlyxBP1n1V9XewLnAzV7VQtJQV7eY+PWL8DRwNNAT2Aw8EtvshE9EGgNvAmNU\n9dvAeYl4nIJsT0IfI1U9pKo9gfa4mo2u1V1XsgT9jcCRAZ/be9MSmqpu9P5+DUzHHexEt9Wrdy2p\nf/06xvmpMVXd6v1TFgPPkWDHyasnfhPIUdW/e5MT9jgF255EP0YlVPUbYD5wCtBcREoehOU75iVL\n0F8EdPFas+sDVwAzYpynGhGRRl5DFCLSCDgb+KTypRLCDOAa7/01wFsxzEtElARHz0Uk0HHyGgmf\nB1ap6qMBsxLyOIXangQ/Rq1EpLn3viGuw8oqXPAf7iXzfYySovcOgNcF6zGgLjBZVSfGOEs1IiJH\n4Ur34B5r+WqibZOITAEG4YaA3QqMB/4BTAM64IbQvkxVE6ZhNMQ2DcJVGyiwDrgxoD48rolIf+A9\nYDlQ7E2+B1cPnnDHqZLtGUHiHqPuuIbauriC+jRVvc+LEVOBlsAS4EpV/b7K9SVL0DfGGFO1ZKne\nMcYY44MFfWOMSSEW9I0xJoVY0DfGmBRiQd8YY1KIBX1jjEkhFvSNMSaF/D8bKEGQJUzoLwAAAABJ\nRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fd55306e400>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAEICAYAAACktLTqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xl8FOX9wPHPFwg3clo8OD0qdzgiaBERQYpSpVi0HB5o\nbdRqsaW2UrAWsbRqqVIs9Se2Wi0BpGoUKkitogityCGCCAjFgAFEQEAwKCT5/v54dnOxSWaz9+T7\nfr3y2t3Z2Zlndjbfeeb7PPOMqCrGGGP8pUaiC2CMMSb6LLgbY4wPWXA3xhgfsuBujDE+ZMHdGGN8\nyIK7Mcb4kAV3E5KI1BSRoyLSJprzJpKInCMiUe/7KyKDRCSnxOstItLPy7xVWNdfRGRiVT9fwXJ/\nIyJ/i/ZyTeLUSnQBTHSIyNESL+sDXwMFgde3qmpWOMtT1QKgYbTnrQ5U9bxoLEdEbgGuU9VLSiz7\nlmgs2/ifBXefUNWi4BqoGd6iqv8ub34RqaWq+fEomzEm/iwtU00ETrufE5G5InIEuE5ELhSRd0Tk\nkIjsEZEZIpIWmL+WiKiItAu8nh14f7GIHBGR/4pI+3DnDbx/uYh8JCKHReQxEVkhImPLKbeXMt4q\nIttE5KCIzCjx2Zoi8qiIHBCR7cCQCr6fSSIyr8y0mSLySOD5LSKyKbA9/wvUqstbVq6IXBJ4Xl9E\n/h4o20agV5l57xWR7YHlbhSRqwLTuwJ/AvoFUl77S3y3k0t8/rbAth8QkZdE5HQv301lRGR4oDyH\nROQNETmvxHsTRWS3iHwhIptLbOsFIrI2MH2viPze6/pMDKiq/fnsD8gBBpWZ9hvgOHAl7qBeDzgf\n6IM7gzsL+Ai4MzB/LUCBdoHXs4H9QAaQBjwHzK7CvN8AjgDDAu+NB04AY8vZFi9lfBloDLQDPg9u\nO3AnsBFoBTQHlrmffMj1nAUcBRqUWPZnQEbg9ZWBeQS4FDgGdAu8NwjIKbGsXOCSwPNpwJtAU6At\n8GGZea8FTg/sk9GBMrQMvHcL8GaZcs4GJgeeDw6UsTtQF/gz8IaX7ybE9v8G+FvgecdAOS4N7KOJ\nwJbA887ADuC0wLztgbMCz1cBowLPGwF9Ev2/UJ3/rOZevSxX1YWqWqiqx1R1laquVNV8Vd0OzAL6\nV/D551V1taqeALJwQSXceb8DrFPVlwPvPYo7EITksYy/U9XDqpqDC6TBdV0LPKqquap6AHiwgvVs\nBz7AHXQALgMOqurqwPsLVXW7Om8ArwMhG03LuBb4jaoeVNUduNp4yfXOV9U9gX0yB3dgzvCwXIAx\nwF9UdZ2qfgVMAPqLSKsS85T33VRkJLBAVd8I7KMHcQeIPkA+7kDSOZDa+zjw3YE7SJ8rIs1V9Yiq\nrvS4HSYGLLhXL5+UfCEiHUTkFRH5VES+AKYALSr4/KclnudRcSNqefOeUbIcqqq4mm5IHsvoaV24\nGmdF5gCjAs9HB14Hy/EdEVkpIp+LyCFcrbmi7yro9IrKICJjReT9QPrjENDB43LBbV/R8lT1C+Ag\ncGaJecLZZ+UttxC3j85U1S3Az3D74bNAmu+0wKw3AZ2ALSLyrohc4XE7TAxYcK9eynYDfAJXWz1H\nVU8B7sOlHWJpDy5NAoCICKWDUVmRlHEP0LrE68q6as4HBonImbga/JxAGesBzwO/w6VMmgD/8liO\nT8srg4icBTwO3A40Dyx3c4nlVtZtczcu1RNcXiNc+meXh3KFs9wauH22C0BVZ6tqX1xKpibue0FV\nt6jqSFzq7Q/ACyJSN8KymCqy4F69NQIOA1+KSEfg1jis859ATxG5UkRqAXcBp8aojPOBn4jImSLS\nHLinoplV9VNgOfA3YIuqbg28VQeoDewDCkTkO8DAMMowUUSaiLsO4M4S7zXEBfB9uOPcD3E196C9\nQKtgA3IIc4EfiEg3EamDC7Jvq2q5Z0JhlPkqEbkksO6f49pJVopIRxEZEFjfscBfIW4DrheRFoGa\n/uHAthVGWBZTRRbcq7efATfi/nGfwDV8xpSq7gW+DzwCHADOBt7D9cuPdhkfx+XGN+Aa+5738Jk5\nuAbSopSMqh4Cfgpk4xolR+AOUl78GncGkQMsBp4tsdz1wGPAu4F5zgNK5qlfA7YCe0WkZHol+PlX\ncemR7MDn2+Dy8BFR1Y247/xx3IFnCHBVIP9eB3gY107yKe5MYVLgo1cAm8T1xpoGfF9Vj0daHlM1\n4lKexiSGiNTEpQFGqOrbiS6PMX5hNXcTdyIyJJCmqAP8CtfL4t0EF8sYX7HgbhLhImA77pT/28Bw\nVS0vLWOMqQJLyxhjjA9Zzd0YY3woYQOHtWjRQtu1a5eo1RtjTEpas2bNflWtqPswkMDg3q5dO1av\nXp2o1RtjTEoSkcqutAYsLWOMMb5kwd0YY3zIgrsxxviQ3YnJmGrixIkT5Obm8tVXXyW6KMaDunXr\n0qpVK9LSyhtaqGIW3I2pJnJzc2nUqBHt2rXDDcZpkpWqcuDAAXJzc2nfvn3lHwghpdIyWVnQrh3U\nqOEes8K65bMx1dtXX31F8+bNLbCnABGhefPmEZ1lpUzNPSsLMjMhL8+93rHDvQYYE/E4eMZUDxbY\nU0ek+yplau6TJhUH9qC8PDfdGGNMaSkT3HfuDG+6MSa5HDhwgO7du9O9e3dOO+00zjzzzKLXx497\nG/b9pptuYsuWLRXOM3PmTLKilLO96KKLWLduXVSWFW8pk5Zp08alYkJNN8ZEX1aWOzPeudP9n02d\nGlkKtHnz5kWBcvLkyTRs2JC777671DyqiqpSo0boeufTTz9d6XruuOOOqhfSR1Km5j51KtSvX3pa\n/fpuujEmuoJtXDt2gGpxG1csOjFs27aNTp06MWbMGDp37syePXvIzMwkIyODzp07M2XKlKJ5gzXp\n/Px8mjRpwoQJE0hPT+fCCy/ks88+A+Dee+9l+vTpRfNPmDCB3r17c9555/Gf//wHgC+//JLvfe97\ndOrUiREjRpCRkVFpDX327Nl07dqVLl26MHHiRADy8/O5/vrri6bPmDEDgEcffZROnTrRrVs3rrvu\nuqh/Z16kTM09WGOIZk3CGBNaRW1csfif27x5M88++ywZGRkAPPjggzRr1oz8/HwGDBjAiBEj6NSp\nU6nPHD58mP79+/Pggw8yfvx4nnrqKSZMmHDSslWVd999lwULFjBlyhReffVVHnvsMU477TReeOEF\n3n//fXr27Flh+XJzc7n33ntZvXo1jRs3ZtCgQfzzn//k1FNPZf/+/WzYsAGAQ4cOAfDwww+zY8cO\nateuXTQt3lKm5g7uR5WTA4WF7tECuzGxEe82rrPPPrsosAPMnTuXnj170rNnTzZt2sSHH3540mfq\n1avH5ZdfDkCvXr3IyckJueyrr776pHmWL1/OyJEjAUhPT6dz584Vlm/lypVceumltGjRgrS0NEaP\nHs2yZcs455xz2LJlC+PGjWPJkiU0btwYgM6dO3PdddeRlZVV5YuQIpVSwd0YEx/ltWXFqo2rQYMG\nRc+3bt3KH//4R9544w3Wr1/PkCFDQvb3rl27dtHzmjVrkp+fH3LZderUqXSeqmrevDnr16+nX79+\nzJw5k1tvvRWAJUuWcNttt7Fq1Sp69+5NQUFBVNfrhQV3Y8xJEtnG9cUXX9CoUSNOOeUU9uzZw5Il\nS6K+jr59+zJ//nwANmzYEPLMoKQ+ffqwdOlSDhw4QH5+PvPmzaN///7s27cPVeWaa65hypQprF27\nloKCAnJzc7n00kt5+OGH2b9/P3llc1xxkDI5d2NM/CSyjatnz5506tSJDh060LZtW/r27Rv1dfz4\nxz/mhhtuoFOnTkV/wZRKKK1ateKBBx7gkksuQVW58sorGTp0KGvXruUHP/gBqoqI8NBDD5Gfn8/o\n0aM5cuQIhYWF3H333TRq1Cjq21CZhN1DNSMjQ+1mHcbEz6ZNm+jYsWOii5EU8vPzyc/Pp27dumzd\nupXBgwezdetWatVKrvpuqH0mImtUNaOcjxRJri0xxpg4OHr0KAMHDiQ/Px9V5Yknnki6wB4pf22N\nMcZ40KRJE9asWZPoYsSUNagaY4wPWXA3xhgfsuBujDE+ZMHdGGN8yIK7MSYuBgwYcNIFSdOnT+f2\n22+v8HMNGzYEYPfu3YwYMSLkPJdccgmVda2ePn16qYuJrrjiiqiM+zJ58mSmTZsW8XKizYK7MSYu\nRo0axbx580pNmzdvHqNGjfL0+TPOOIPnn3++yusvG9wXLVpEkyZNqry8ZGfB3RgTFyNGjOCVV14p\nujFHTk4Ou3fvpl+/fkX9znv27EnXrl15+eWXT/p8Tk4OXbp0AeDYsWOMHDmSjh07Mnz4cI4dO1Y0\n3+233140XPCvf/1rAGbMmMHu3bsZMGAAAwYMAKBdu3bs378fgEceeYQuXbrQpUuXouGCc3Jy6Nix\nIz/84Q/p3LkzgwcPLrWeUNatW8cFF1xAt27dGD58OAcPHixaf3AI4OCAZW+99VbRzUp69OjBkSNH\nqvzdhmL93I2phn7yE4j2DYa6d4dAXAypWbNm9O7dm8WLFzNs2DDmzZvHtddei4hQt25dsrOzOeWU\nU9i/fz8XXHABV111Vbn3EX388cepX78+mzZtYv369aWG7J06dSrNmjWjoKCAgQMHsn79esaNG8cj\njzzC0qVLadGiRallrVmzhqeffpqVK1eiqvTp04f+/fvTtGlTtm7dyty5c3nyySe59tpreeGFFyoc\nn/2GG27gscceo3///tx3333cf//9TJ8+nQcffJCPP/6YOnXqFKWCpk2bxsyZM+nbty9Hjx6lbt26\nYXzblfNUcxeRISKyRUS2icjJAyYXz/c9EVERqfTSWGNM9VMyNVMyJaOqTJw4kW7dujFo0CB27drF\n3r17y13OsmXLioJst27d6NatW9F78+fPp2fPnvTo0YONGzdWOijY8uXLGT58OA0aNKBhw4ZcffXV\nvP322wC0b9+e7t27AxUPKwxufPlDhw7Rv39/AG688UaWLVtWVMYxY8Ywe/bsoith+/bty/jx45kx\nYwaHDh2K+hWylS5NRGoCM4HLgFxglYgsUNUPy8zXCLgLWBnVEhpjoq6iGnYsDRs2jJ/+9KesXbuW\nvLw8evXqBUBWVhb79u1jzZo1pKWl0a5du5DD/Fbm448/Ztq0aaxatYqmTZsyduzYKi0nKDhcMLgh\ngytLy5TnlVdeYdmyZSxcuJCpU6eyYcMGJkyYwNChQ1m0aBF9+/ZlyZIldOjQocplLctLzb03sE1V\nt6vqcWAeMCzEfA8ADwFV/yaNMb7WsGFDBgwYwM0331yqIfXw4cN84xvfIC0tjaVLl7Ij1A2TS7j4\n4ouZM2cOAB988AHr168H3HDBDRo0oHHjxuzdu5fFixcXfaZRo0Yh89r9+vXjpZdeIi8vjy+//JLs\n7Gz69esX9rY1btyYpk2bFtX6//73v9O/f38KCwv55JNPGDBgAA899BCHDx/m6NGj/O9//6Nr167c\nc889nH/++WzevDnsdVbEy3nAmcAnJV7nAn1KziAiPYHWqvqKiPy8vAWJSCaQCdDG7mxtTLU0atQo\nhg8fXqrnzJgxY7jyyivp2rUrGRkZldZgb7/9dm666SY6duxIx44di84A0tPT6dGjBx06dKB169al\nhgvOzMxkyJAhnHHGGSxdurRoes+ePRk7diy9e/cG4JZbbqFHjx4VpmDK88wzz3DbbbeRl5fHWWed\nxdNPP01BQQHXXXcdhw8fRlUZN24cTZo04Ve/+hVLly6lRo0adO7cueiuUtFS6ZC/IjICGKKqtwRe\nXw/0UdU7A69rAG8AY1U1R0TeBO5W1Qo7ndqQv8bElw35m3oiGfLXS1pmF9C6xOtWgWlBjYAuwJsi\nkgNcACywRlVjjEkcL8F9FXCuiLQXkdrASGBB8E1VPayqLVS1naq2A94Brqqs5m6MMSZ2Kg3uqpoP\n3AksATYB81V1o4hMEZGrYl1AY0z0JOrOayZ8ke4rTx0rVXURsKjMtPvKmfeSiEpkjImJunXrcuDA\nAZo3b17uxUEmOagqBw4ciOjCJrtC1ZhqolWrVuTm5rJv375EF8V4ULduXVq1alXlz1twN6aaSEtL\no3379okuhokTGzjMGGN8yIK7Mcb4kAV3Y4zxIQvuxhjjQxbcjTHGhyy4G2OMD1lwN8YYH7Lgbowx\nPmTB3RhjfMiCuzHG+JAFd2OM8SEL7sYY40MW3I0xxocsuBtjjA9ZcDfGGB+y4G6MMT5kwd0YY3zI\ngrsxxviQBXdjjPEhC+7GGONDFtyNMcaHLLgbY4wPWXA3xhgfsuBujDE+ZMHdGGN8yIK7Mcb4kAV3\nY4zxIQvuxhjjQxbcjTHGhyy4G2OMD1lwN8YYH7LgbowxPmTB3RhjfMiCuzHG+JAFd2OM8SEL7sYY\n40MW3I0xxoc8BXcRGSIiW0Rkm4hMCPH+bSKyQUTWichyEekU/aIaY4zxqtLgLiI1gZnA5UAnYFSI\n4D1HVbuqanfgYeCRqJfUGGOMZ15q7r2Bbaq6XVWPA/OAYSVnUNUvSrxsAGj0imiMMSZctTzMcybw\nSYnXuUCfsjOJyB3AeKA2cGmoBYlIJpAJ0KZNm3DLaowxxqOoNaiq6kxVPRu4B7i3nHlmqWqGqmac\neuqp0Vq1McaYMrwE911A6xKvWwWmlWce8N1ICmWMMSYyXoL7KuBcEWkvIrWBkcCCkjOIyLklXg4F\ntkaviMYYY8JVac5dVfNF5E5gCVATeEpVN4rIFGC1qi4A7hSRQcAJ4CBwYywLbYwxpmJeGlRR1UXA\nojLT7ivx/K4ol8sYY0wE7ApVY4zxIQvuxhjjQxbcjTHGh3wb3LOyoF07qFHDPWZlJbpExhgTP54a\nVFNNVhZkZkJennu9Y4d7DTBmTOLKZYwx8eLLmvukScWBPSgvz003xpjqwJfBfefO8KYbY4zf+DK4\nlzcmmY1VZoypLnwZ3KdOhfr1S0+rX99NN8aY6sCXwX3MGJg1C9q2BRH3OGuWNaYaY6oPX/aWARfI\nLZgbY6orX9bcjTGmurPgbowxPmTB3RhjfMiCuzHG+JAFd2OM8SEL7sYY40MW3I0xxocsuBtjjA9Z\ncDfGGB+y4G6MMT5kwd0YY3zIgrsxxviQBXdjjPEhC+7GGONDFtyNMcaHLLgbY4wPWXA3xhgfsuBu\njDE+ZMHdGGN8yIK7Mcb4kAV3Y4zxIQvuxhjjQxbcjTHGh6p9cM/KgnbtoEYN95iVlegSGWNM5Kp1\ncM/KgsxM2LEDVN1jZmb1CvD/+x88+WSiS2GMibZqHdwnTYK8vNLT8vLc9Opi8mR3QNu/P9ElMcZE\nU8oF9wMH4NVXo7OsnTvDm+43x4/DP//pnr//fmLLYoyJLk/BXUSGiMgWEdkmIhNCvD9eRD4UkfUi\n8rqItI1+UZ3HH4crrohOTbNNm/Cm+82bb8KhQ+65BXdj/KXS4C4iNYGZwOVAJ2CUiHQqM9t7QIaq\ndgOeBx6OdkGDLrvM5cdffz3yZU2dCvXrl55Wv76bXh28+CI0aACnnmrBvbrJz4cvvkh0KUwseam5\n9wa2qep2VT0OzAOGlZxBVZeqajB7/Q7QKrrFLJaRAU2awL/+FfmyxoyBWbOgbVsQcY+zZrnpfldQ\nAC+9BEOHQs+eFtyrmylT4Lzz4OuvE10SEytegvuZwCclXucGppXnB8DiUG+ISKaIrBaR1fv27fNe\nyhJq1oSBA+G111wNPlJjxkBODhQWusfqENgB3nkH9u6F4cMhPR0+/NDl4E31sGQJfPopLFqU6JKY\nWIlqg6qIXAdkAL8P9b6qzlLVDFXNOPXUU6u8nsGD4ZNP4KOPqryIau/FF6F2bdd+0b07nDgBmzcn\nulQmHvLyYO1a93zOnMSWxcSOl+C+C2hd4nWrwLRSRGQQMAm4SlVjerJ32WXuMRqpmepIFbKz3fd4\nyimu5g6Wmqku3n3X5dzPOw8WLrTcu195Ce6rgHNFpL2I1AZGAgtKziAiPYAncIH9s+gXs7T27eHs\ns11qxoTv/ffh449dSgbgm9+EOnUsuFcXK1a4x0cfdTn37OzElsfERqXBXVXzgTuBJcAmYL6qbhSR\nKSJyVWC23wMNgX+IyDoRWVDO4qJm8GBYutSlE0x4XnzRDbdwVWDv1aoFXbpYcK8uli+Hzp1hyBA4\n66zqdUV2deIp566qi1T1m6p6tqpODUy7T1UXBJ4PUtWWqto98HdVxUuM3GWXwdGjrmEwXvwyDk12\nNvTr57pABqWnu+AejUZqk7wKC+G//4WLLnI9xEaPdt2KP/000SUz0ZZyV6gGDRjggmy8UjN+GYfm\no4/ggw/g6qtLT09Ph3377J/c7zZuhMOHoW9f93r0aBfw589PbLlM9KVscG/SBPr0iV9w98s4NMH8\najDfHmSNqtXD8uXuMRjcO3Z0vaWs10x8fP01fO97sHJl7NeVssEdXGrm3Xfh4MHYr8sv49BkZ7sL\nwVq3Lj29Wzf3aMHd31asgNNPd50SgkaPdsFm27bElau6+MUvXJtXPM6QUzq4Dx7sTimXLo39uvww\nDk1urvsnLpuSAWja1G2LBXd/W77c1dpFiqeNGuVez52buHJVB9nZMGMG/OQnMGxY5fNHKqWDe+/e\n0KhRfPq7+2Ecmpdeco9lUzJBwUZV40+7drm2omBKJqhVK7j4Ytd+ZA3qsZGTAzff7M6aH3ooPutM\n6eCeluYaVuORd/fDODTZ2S7H2qFD6PfT02HLFvjqq/iWy8RHsH/7RRed/N7o0W7fr1sX3zJVB8eP\nw/e/77IMzz3nrgyPh5QO7uBSM9u3uzsKxVoqj0Ozfz+89VbolExQerobUGzjxviVy8TP8uXubDPY\neF7SiBGuspRqvb9SwS9/6doGn3rKXVcQLykf3INDEdjVqhVbuNAF7vJSMmA9ZvxuxQq44AIXxMtq\n1gwuv9zl3QsK4l82v1q4EB55BO64w/WSiaeUD+7nnusaApMpuCfjxU7Z2e576tmz/HnOPtuN727B\n3X+OHHEpl7L59pJGj4bdu+Htt+NXLj/buRNuvBF69IBp0+K//pQP7iIuNfPGG24wpERLxoudjhxx\njc5XX126l0RZNWpA164W3P1o5UqXTqwouF95pTu4W5/3yJ044Xoh5ee7C8Tq1o1/GVI+uINLzRw6\nBKtXJ7okyXmx0+LF7uKJilIyQTYMgT+tWOEO3hdeWP489eu738g//mE38YjUr34F//mP63RxzjmJ\nKYMvgvvAga5GmgypmWS82Ck7240jU1GtLSg93R0oP/mk8nlN6li+3J2VnXJKxfONGeP2f7RuQl8d\nLV7sujtmZsLIkYkrhy+Ce/Pm0KtXcozvnmwXO331Ffzzn+6iiZo1K58/2KhqXeL8Iz/fDbAXqgtk\nWQMHuoqApWaqZtcuuOEGd8X39OmJLYsvgju41Mw777j8ciIl28VOr7/uRs+sqAtkSV27ukfLu/vH\nhg3uN+DlzC0tDa69FhYsSPz/UqrJz3d59mPHXJ69Xr3Elsc3wX3wYPflvvlmYsuRbBc7ZWe7U/FL\nL/U2f6NGrteMBXf/KDtYWGXGjHFnfHYTj/Dcf7/rafR//+fucpVovgnuF17oasjJkJpJloud8vPh\n5Zdh6FB3pyWvbBgCf1mxwg0U5zU1eMEFrguvpWa8mz/fnZ3ffDNcd12iS+P4JrjXqQP9+ydHo2qy\nWL7cXZnqNSUTlJ7urvg9ejQ25TLxo1o8WJhXwZt4/PvfsHdv7MrmB5s3u5vMf//7rj/7Y48lukTF\nfBPcwaVmtmxJvWF4YyU72/WvHTIkvM+lp7ugsGFDbMpl4mfnTtfI56UxtaTRo92VqnYTj9AOHYKf\n/cy1Ua1YAX/4g7vDVdn2tkTyVXD381AE4V71qurGjR48GBo2DG9dNgyBfwQHCwun5g7uHqvp6Zaa\nKaugAJ580t1U/tFHYexYd3ez8ePjNyCYV74K7p06wRln+C+4V+Wq19Wr3fjt4aZkwDUCN25swd0P\nli93jeTBXlDhGD3a9UDbvj365UpFb78N55/v/ve++U1YtcoF+pYtE12y0HwV3EVc7f3f/3aNmX5R\nlates7Ndv/Yrrwx/fSKun64F99S3YoXrbODlGoeyghfgVPebeOzc6b6Liy929xmeO9cF+l69El2y\nivkquIML7gcOwHvvJbok3nhJt4R71asqvPACXHKJG+2vKtLTYf16fx0kq5vDh127SbgpmaA2baBf\nv+p7Ew9V+N3v3P0PXn4Z7rvPNaCOHFnxGE3JwnfBfdAg95gKqRmv6ZZwr3pdvNjlASPpgpmeDl9+\naafkqey//3W/q6oGd3C/oU2bXHqmulm2DCZOdO1Wmze7fuwNGiS6VN75Lri3bOkCUzL0d6+M13RL\nOFe9qsLkye4sIJL+tt27u0dLzaSuFStcOqZPn6ovY8QI1/7Srx9cf3316kE1Y4Y7850717VDpRrf\nBXdwqZkVK04OnMnGa7olnKteFy92DT333hv6pgxede7sUkUW3FPX8uXuIB1ub6mSmjd3AX3cONeO\n062buyhu2TJ/p2p27nT3HP7hDxM/jEBV+TK4Dx7s7lu4bFmiS1KxcNItXq56LVlrv+GGyMpWr567\nhNqCe2o6ccKN4R5u//ZQWrd2dxPauRMeeMBVHvr3h299ywVAP7bL/PnP7vFHP0psOSLhy+B+0UXu\nitVkT81Ee5CxYK190qTIau1BNgxB6lq3zg1gFUm+vaxmzdwZ4Y4dMHOmu3p1+HDXBfmvf/XPGPDH\njrkujt/9buJGc40GXwb3evVcjjDZG1WjOchYNGvtQenp7h/50KHoLM/ET7iDhYWjXj1Xo/3oI5g3\nz1VIbrnF3fz5ySejv754mzMHPv/cpaJSmS+DO7jUzAcfwJ49iS5JxaI1yNirrxbX2qN1pVzwStX1\n66OzvEj8/veuX/FnnyW6JKlhxQpo395d1BcrtWq5MVXWrHEVqfbtXW+vZPi9VJWqGx+ma1fXrz2V\n+Ta4B4cieOgh1+f9xInElieWYlFrh+QZhuCll+AXv4C1a+Gaa/y9L6NB1QX3WNTaQxFxXZAXLnTD\nS0+eHJ+UTh41AAASr0lEQVT1xsLbb7vf+7hxqdGXvSK+De7durmj7x//CD17ukuwL7gA7rwTnnkG\nNm5040T4wauvwrvvhl9rr+wCqtNPhxYtEhvcP/zQdcHr3Rueeso1ko8fn7jypILt2+HTT6PTmBqO\npk3dvsnOdgfiVDRjhtuO0aMTXZIoUNWE/PXq1UtjrbBQdds21XnzVO++W7V/f9WGDVVd3Ua1QQPV\niy5S/elPVRcscPOnmsJC1d69Vdu1U/36a++fmz1btX794u8C3OvZs0vPN3CgakZGeGWaN0916FDV\nPXvC+1xZBw+qnnOOasuWqrm5btrPfubK+tRTkS3bz555xn1HGzbEf92HDqk2bar6ne/Ef92R2rFD\ntWZN1V/8ItElqRiwWj3EWF8H91AKClQ3bVJ99lnVceNUv/Ut1Xr13DcxapTqkSMJKVaVLVrkyj5r\nVnifa9u2dGAP/rVtW3q+8eNV69ZVPXHC23I3biz+Ps85RzUnJ7xyBeXnq15+uWpamury5cXTT5xQ\nHTRItXZt1Xfeqdqy/e6HP1Rt0sT91hNh6lS3/1euTMz6q2rCBNUaNar+m40XC+5hOHHC/SBr1FDt\n0CExNZ6qCNba27YNr9auqioSOriLlJ4vWAv88MPKl5mXp9q1q+qpp6q++KILMK1bq27ZEl7ZVN0/\nGqg+8cTJ7+3fr9q+veoZZ0R+dpAK5s9XHTZM9dVXvZ1dduqkesUVsS9Xeb74QrV5c9UhQxJXhnDl\n5bkyDx+e6JJUzoJ7FbzxhksB1Kvnglqyq2qtXdV7zX3dOjd97tzKl3nHHW7eRYvc6/fec4H+G99w\ny/Hquefccm67rfx51q1zaaRvfSv8A1sq+fhjl0qsUcN9J927q86ZU/6Z1IEDbr6pU+NazJM89JAr\nx4oViS2HV3/9qyvv0qWJLknlLLhX0e7dLjcPqrfc4o7oySiSWruq95z711+71MiECRUvLzvbLWP8\n+NLTN29WbdXK1eL/+9/KyxUM2n37Vr5dwYPArbdWvtxUVFCgOmCAC+5btrgA1KGD2+b27VX/9CfV\nL78s/ZmFC937b72VmDIHHT3qDuoDBya2HF4UFrqDZpcuqdHuZsE9AidOqE6c6L6d9HTVjz5KdIlO\nFkmtPWj2bHdwEHGPZQN7ULduLv9dnp07XSNar16hA3JOjsu/N2ig+vrr5S9n/37XMHzmmd7TLRWl\nb1Ldn/508j4uKFB96SXVCy9077VooXr//e67U3XfR1paclRKHnnElfHNNxNdkootWxb5/1I8WXCP\ngldeUW3WTLVRI9Xnn090aYpFWmsP1/XXu/x2KCdOqPbr52qXFR0E9+xxNaM6dVzPpFDLGTjQvf/u\nu97Llp/vcrtpaamTAvBi61Z3BjNkSOjaZGGhC0pDhxafdd11l2qPHqp9+sS/vKHk5amefrrqxRcn\nd434mmtc5aTsWVCyimpwB4YAW4BtwIQQ718MrAXygRFelpkKwV3V1Tp793bf1F13JUd+d/Hiimsa\nXmvkXk2b5tb32Wcnvzd5snvv2WcrX86BA6rnn++6m82ZU/q9n/7ULacqbR2ff6569tmqp51W3GUy\nleXnuy66jRurfvJJ5fNv2OAOwLVqacjUWCI99pgr07//neiShLZzp/s9/vzniS6Jd1EL7kBN4H/A\nWUBt4H2gU5l52gHdgGf9FtxVXUAfN859W336uL7ziVJY6MrQpk3oA43XXHo4XnvNLee110pPf+st\n19B3/fXel/XFF65NQ6Q4lfLss8UHz6r64AOX9undW/XYsaovJxkE0xl/+1t4n9uxQ/W3v3WPyeLY\nMdfm8q1vJWftfeJE9xv++ONEl8S7aAb3C4ElJV7/EvhlOfP+zY/BPegf/3ApmrQ0V9M8cCD+ZQjW\n2svLMXvtBROOzz5zy5g2rXja/v3un/acc1zADkdenuuqB6o//rHrRz9ggOrx41Uvo6rqCy+4Zd58\nc3IGEi82b3bfx5VXpu42lPX4426/vPpq+J89eNBdlxILwe6P3/1ubJYfK9EM7iOAv5R4fT3wp3Lm\n9XVwV3Wn/Tff7GqeTZq4gPfVV/FZd2W1dlXv/dfD1aSJqxmLuPX36uUOcqtXV215X3/tcp3BA0+o\nlE9V3Htv1Wq9yeDECbd/mzXzV//9r792+/j888M7YL36qsvZi7hKQLQvMHzqKfdbeeON6C431pIy\nuAOZwGpgdZs2beLxPcTM+vWusQtcD485c2J/ReDcuRXX2lVjU3OfPbu4n3XJv9Gjq75MVZdbfuwx\nV1uNloICl5pp0yZ+B92gDz5Qvekm1csuq9rVsw8+6L7Xsu0RfvCXv7htW7iw8nnz8orToJ07q2Zm\nFrcfLVkSnfIUFrrG51Tp/liSpWXi5F//ct0lwdVMYtHtq6BA9YEH3A+8R4+KG3VjkXMv74AR6vgc\n7cbcqliyxJXvz3+O/bqCvVa+853i77ply+LapteU1YYNbkiF730v9YKNF8ePq551lvv9VrR9a9e6\nK2xB9Sc/KW4/Wb68uI//2LGRp0TffrvyilKyimZwrwVsB9qXaFDtXM681S64q7oa6N/+5nLQoHrV\nVdHLEx48WBw4Ro92F4dUJtoB1muqJxYHlqooLHQXQZ15ZuwaV/PzXY7/ggu0qL/5lCmuLeLwYdU7\n73TfT6tWqi+/XPGyjh9X7dnTXc0brfRUMgoOZfHiiye/l5/vrmpNS3Pdbv/1r5PnOXZMddIk17ul\nZcvIuidfe61LNXr5f0o20e4KeQXwUaDXzKTAtCnAVYHn5wO5wJfAAWBjZcv0U3APystzvRUaNXI/\nwFtvjaznwvvvuy5+tWq59EWianReUz2xSAlV1euvu3XPmBHd5R475mp7557rln/WWe4MIVQf6Xfe\ncWPtgOrVV5ffTfP++908yXQtRSycOKH6zW+676RkCjMnp/iq8BEjii/IKs9777kzgOD3unt3eOX4\n5BP3/3n33WFvQlKwi5gSaO9eN85KWpr7y8wMv6vV7NlujJvTT0/8xTnBslRWI49VY25VFBa6gHHa\nadG5WvPzz914LS1bum3KyHADeuXnV/y548dVf/c71wPmlFNUZ84sHdjee88dvEeNiryMqSAry31/\n8+e717Nnu++lUSNXs/dagTlxwrVR1KnjauBPP33yZ7/6yp1BL1yo+uij7n/y2992Z3Qiqtu3R3XT\n4saCexLYsUP1Rz9yudRatVwvm61bK/7M11+7XC24K/uSpdfE7Nkux15RqieZau6qrv0DXL/xSOzY\n4dIu4IZhWLo0/LOobdvcUMXghg7YsMHt627d3AEoEd1qEyE/3+XUO3ZUHTnSfR99+1Y90G7e7K6Q\nBneF8623usdgWrLk77BxY3dQHjkytRutLbgnkdxc1/pft27xRT+heojs2uUu9gheZRhpv+94S5ac\ne0kDB7oBrKqaW83PdwfZhg1VV62KrCyFhe6CrebN3cH+oovUcw8SP5k/3213rVqqv/mN93sFlKeg\nwKXGGjd2322fPqpjxqj++tfut/fOOy7V45eGagvuSWjPHncnofr1Xa1i5EjXfU7VXe3ZsqXrS/7c\nc4ktZyTCacyNR8+a5cvdr/zhh6v2+d/+VqPeb37fPtUbbtCinh/VTUGB6h/+EPnBsiy/BO/KWHBP\nYnv3qt5zjwvkwdPJmjVVzzvP3cmoOohnLf/b33Y1unCvpF21ytUur7kmNoFjy5bUOzsziec1uIub\nN/4yMjJ09erVCVl3sjhwAKZPdzflvewydwPoU05JdKnio1072LHj5Olt20JOTnTXtXKluzn6b38L\nv/ylt898+aW7sXpenrtBeLNm0S2TMVUlImtUNaOy+WrEozAmtObN4YEH4PPP4fnnq09gB9i50/v0\nrCx3MKhRwz1mZYW3rj59YOhQ+P3v4YsvvC1z/HjYuhWefdYCu0lNFtyTQM2aiS5B/LVp4216VhZk\nZrpavqp7zMwMP8Dffz8cPAh//GPly3z5ZZg1C+6+GwYM8L6OSA9CxkSVl9xNLP6qc87deM+5h9u9\nsqJG2mHDim/aXd4yd+923R579AhvbJpk7Clk/AlrUDXJzktvmXAujKoswL73Xuhllfz79rddl9UP\nPwxvW5Ktj7/xL6/B3RpUTVILp+HVy7wjRsCLL7rQW1bTpi51M3Mm/OhH4ZWzRo3QyxSBwsLwlmVM\nRaxB1fjC1KlQv37pafXru+lleWmknTzZPdaqVXqeunXh6FHX8Hr77aXf85JL99qGYGLD2jtC8FK9\nj8WfpWWMV14vdvKaGvn+913qpVWr4puPtG7trmTdu/fkdXvJpYebc0+GoZH9orq1d2A5d1PdeP0n\n37jRBdV77nGvgzfnfuWVk5cZTi7da8BOdDDy24ElFvsomVlwN9WS13/e0aOLAyq4EQNDicVIl7Fq\nfPWy7Yk+sMRCqt1vIFIW3I2pwObNxbcO7Nix/GGBYxGIY3HAiFXX0liIdu05Fe83EAkL7sZUYuxY\nNxzze++VP088b1sYSRrB6zITPeZ+LL5Pr8tM9LZHiwV3Yypx/Li3O2VFu6YZi0Zar4ErmheFVUUi\nU1JWc7fgbkzMRTsYeZ03nANGLGrZiaw9W87dgrsxSSGaV+eWnTeaqZ5wJLr2HIveMvHugWPB3ZgU\nl+gUSiIbfoPzJnu3xUScDVhwNybFJTqNYF02K5eIMxGvwd3GljEmiWVlwaRJbgiFNm3csAtjxsRv\n3ZmZ7oYlQfXru+GQY12GeN7MJRKJGFPIxpYxxgfGjHHBrLDQPcYrsAfXPWuWC6gi7jEegR3Cu5lL\nIoU7plA8x8Cx4G6MKZfXg0u0g1aqDMQWzsB20brxjFcW3I0xEYlF0AonaCZSOGc3kyaVTnGBez1p\nUmzKZsHdGBORWAStRKaEgryejXg9u4l3qskaVI0xEfHjjUpi0ZgcrUZia1A1xsRFquTHwxGLs5F4\np5osuBtjIpIq+fFwxCKFEu9UkwV3Y0xEkiE/Hm2xOhuJZ9dWC+7GmIglsj9+LPjhbMSCuzHGlOGH\ns5Falc9ijDHVz5gxqRXMy7KauzHG+JAFd2OM8SEL7sYY40MW3I0xxocsuBtjjA8lbGwZEdkHhBhp\nwZMWwP4oFicZ+G2b/LY94L9t8tv2gP+2KdT2tFXVUyv7YMKCeyREZLWXgXNSid+2yW/bA/7bJr9t\nD/hvmyLZHkvLGGOMD1lwN8YYH0rV4D4r0QWIAb9tk9+2B/y3TX7bHvDfNlV5e1Iy526MMaZiqVpz\nN8YYUwEL7sYY40MpF9xFZIiIbBGRbSIyIdHliZSI5IjIBhFZJyIpeVNZEXlKRD4TkQ9KTGsmIq+J\nyNbAY9NEljEc5WzPZBHZFdhP60TkikSWMVwi0lpElorIhyKyUUTuCkxPyf1Uwfak7H4Skboi8q6I\nvB/YpvsD09uLyMpAzHtORGp7Wl4q5dxFpCbwEXAZkAusAkap6ocJLVgERCQHyFDVlL3wQkQuBo4C\nz6pql8C0h4HPVfXBwEG4qarek8hyelXO9kwGjqrqtESWrapE5HTgdFVdKyKNgDXAd4GxpOB+qmB7\nriVF95OICNBAVY+KSBqwHLgLGA+8qKrzROT/gPdV9fHKlpdqNffewDZV3a6qx4F5wLAEl6naU9Vl\nwOdlJg8Dngk8fwb3j5cSytmelKaqe1R1beD5EWATcCYpup8q2J6Upc7RwMu0wJ8ClwLPB6Z73kep\nFtzPBD4p8TqXFN+huJ33LxFZIyKZiS5MFLVU1T2B558CLRNZmCi5U0TWB9I2KZG+CEVE2gE9gJX4\nYD+V2R5I4f0kIjVFZB3wGfAa8D/gkKrmB2bxHPNSLbj70UWq2hO4HLgjkBLwFXW5v9TJ/4X2OHA2\n0B3YA/whscWpGhFpCLwA/ERVvyj5XirupxDbk9L7SVULVLU70AqXqehQ1WWlWnDfBbQu8bpVYFrK\nUtVdgcfPgGzcDvWDvYG8aDA/+lmCyxMRVd0b+McrBJ4kBfdTII/7ApClqi8GJqfsfgq1PX7YTwCq\neghYClwINBGR4C1RPce8VAvuq4BzA63HtYGRwIIEl6nKRKRBoDEIEWkADAY+qPhTKWMBcGPg+Y3A\nywksS8SCATBgOCm2nwKNdX8FNqnqIyXeSsn9VN72pPJ+EpFTRaRJ4Hk9XMeRTbggPyIwm+d9lFK9\nZQACXZumAzWBp1R1aoKLVGUichautg7uZuVzUnF7RGQucAlueNK9wK+Bl4D5QBvc0M7XqmpKNFKW\nsz2X4E71FcgBbi2Rq056InIR8DawASgMTJ6Iy1On3H6qYHtGkaL7SUS64RpMa+Iq3vNVdUogTswD\nmgHvAdep6teVLi/VgrsxxpjKpVpaxhhjjAcW3I0xxocsuBtjjA9ZcDfGGB+y4G6MMT5kwd0YY3zI\ngrsxxvjQ/wPcz2v1dLH/YgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fd544300160>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "acc = history.history['acc']\n",
    "val_acc = history.history['val_acc']\n",
    "loss = history.history['loss']\n",
    "val_loss = history.history['val_loss']\n",
    "\n",
    "epochs = range(len(acc))\n",
    "\n",
    "plt.plot(epochs, acc, 'bo', label='Training acc')\n",
    "plt.plot(epochs, val_acc, 'b', label='Validation acc')\n",
    "plt.title('Training and validation accuracy')\n",
    "plt.legend()\n",
    "\n",
    "plt.figure()\n",
    "\n",
    "plt.plot(epochs, loss, 'bo', label='Training loss')\n",
    "plt.plot(epochs, val_loss, 'b', label='Validation loss')\n",
    "plt.title('Training and validation loss')\n",
    "plt.legend()\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see, we reach a validation accuracy of about 96%. This is much better than our small convnet trained from scratch."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-block alert-info\">\n",
    "<div style=\"direction:rtl;text-align:right;font-family:B Lotus, B Nazanin, Tahoma\"> دانشگاه تربیت دبیر شهید رجایی<br>مباحث ویژه - آشنایی با بینایی کامپیوتر<br>علیرضا اخوان پور<br>96-97<br>\n",
    "</div>\n",
    "<a href=\"https://www.srttu.edu/\">SRTTU.edu</a> - <a href=\"http://class.vision\">Class.Vision</a> - <a href=\"http://AkhavanPour.ir\">AkhavanPour.ir</a>\n",
    "</div>"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python [conda env:keras]",
   "language": "python",
   "name": "conda-env-keras-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}