Train_CNN_Digital-Readout-Small-v3.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# CNN Training\n",
    "\n",
    "Target of this code is to train a CNN network to classify images of a digital readout to the digits 0 to 9. Additionally a category \"NaN\" is introduced, to mark images that are not amibiguous.\n",
    "\n",
    "### Preparing the training\n",
    "* First all libraries are loaded\n",
    "    * It is assumed, that they are installed during the Python setup\n",
    "* matplotlib is set to print the output inline in the jupyter notebook"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "########### Basic Parameters for Running: ################################\n",
    "    \n",
    "TFliteNamingAndVersion = \"dig1320s3\"   # Used for tflite Filename\n",
    "Training_Percentage = 0.0              # 0.0 = Use all Images for Training\n",
    "Epoch_Anz = 500\n",
    "\n",
    "##########################################################################\n",
    "\n",
    "\n",
    "import tensorflow as tf\n",
    "import matplotlib.pyplot as plt\n",
    "import glob\n",
    "import os\n",
    "import numpy as np\n",
    "from sklearn.utils import shuffle\n",
    "from tensorflow.python import keras\n",
    "from tensorflow.python.keras import Sequential\n",
    "from tensorflow.python.keras.layers import Dense, InputLayer, Conv2D, MaxPool2D, Flatten, BatchNormalization\n",
    "from tensorflow.keras.preprocessing.image import ImageDataGenerator\n",
    "from sklearn.model_selection import train_test_split\n",
    "from tensorflow.keras.callbacks import History \n",
    "from tensorflow.keras.utils import to_categorical\n",
    "from PIL import Image \n",
    "from pathlib import Path\n",
    "\n",
    "loss_ges = np.array([])\n",
    "val_loss_ges = np.array([])\n",
    "\n",
    "%matplotlib inline\n",
    "np.set_printoptions(precision=4)\n",
    "np.set_printoptions(suppress=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Load training data\n",
    "* The data is expected in the \"Input_dir\"\n",
    "* Inside subdirectories are expected from -1, 0, 1, ... 9 in which the pictures are sorted according to their values (=category)\n",
    "* Picture size must be 20x32 with 3 color channels (RGB)\n",
    "* The filename can be arbitrary\n",
    "\n",
    "* The images are stored in the x_data[]\n",
    "* The expected category for each image in the corresponding y_data[]\n",
    "\n",
    "* The last step is a shuffle (from sklearn.utils) and split the data into training and validation data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(1582, 32, 20, 3)\n",
      "(1582, 11)\n"
     ]
    }
   ],
   "source": [
    "Input_dir='ziffer_sortiert_resize'\n",
    "\n",
    "files = glob.glob(Input_dir + '/*.jpg')\n",
    "x_data = []\n",
    "y_data = []\n",
    "\n",
    "for aktfile in files:\n",
    "    base = os.path.basename(aktfile)\n",
    "    target = base[0:1]\n",
    "    if target == \"N\":\n",
    "        category = 10                # NaN does not work --> convert to 10\n",
    "    else:\n",
    "        category = int(target)\n",
    "    test_image = Image.open(aktfile)\n",
    "    test_image = np.array(test_image, dtype=\"float32\")\n",
    "    x_data.append(test_image)\n",
    "    y_data.append(np.array([category]))\n",
    "\n",
    "x_data = np.array(x_data)\n",
    "y_data = np.array(y_data)\n",
    "y_data = to_categorical(y_data, 11)\n",
    "print(x_data.shape)\n",
    "print(y_data.shape)\n",
    "\n",
    "x_data, y_data = shuffle(x_data, y_data)\n",
    "\n",
    "if (Training_Percentage > 0):\n",
    "    X_train, X_test, y_train, y_test = train_test_split(x_data, y_data, test_size=Training_Percentage)\n",
    "else:\n",
    "    X_train = x_data\n",
    "    y_train = y_data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Define the model\n",
    "\n",
    "The layout of the network ist a typcial CNN network with alternating **Conv2D** and **MaxPool2D** layers. Finished after **flattening** with additional **Dense** layer.\n",
    "\n",
    "#### Important\n",
    "* Shape of the input layer: (32, 20, 3)\n",
    "* Number of output layers: 11\n",
    "* As loss function \"categorical_crossentropy\" is choosen, as it is a categories task"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "batch_normalization (BatchNo (None, 32, 20, 3)         12        \n",
      "_________________________________________________________________\n",
      "conv2d (Conv2D)              (None, 32, 20, 32)        896       \n",
      "_________________________________________________________________\n",
      "max_pooling2d (MaxPooling2D) (None, 16, 10, 32)        0         \n",
      "_________________________________________________________________\n",
      "conv2d_1 (Conv2D)            (None, 16, 10, 32)        9248      \n",
      "_________________________________________________________________\n",
      "max_pooling2d_1 (MaxPooling2 (None, 8, 5, 32)          0         \n",
      "_________________________________________________________________\n",
      "conv2d_2 (Conv2D)            (None, 8, 5, 32)          9248      \n",
      "_________________________________________________________________\n",
      "max_pooling2d_2 (MaxPooling2 (None, 4, 2, 32)          0         \n",
      "_________________________________________________________________\n",
      "flatten (Flatten)            (None, 256)               0         \n",
      "_________________________________________________________________\n",
      "dense (Dense)                (None, 128)               32896     \n",
      "_________________________________________________________________\n",
      "dense_1 (Dense)              (None, 11)                1419      \n",
      "=================================================================\n",
      "Total params: 53,719\n",
      "Trainable params: 53,713\n",
      "Non-trainable params: 6\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "model = Sequential()\n",
    "model.add(BatchNormalization(input_shape=(32,20,3)))\n",
    "model.add(Conv2D(32, (3, 3), padding='same', activation=\"relu\"))\n",
    "model.add(MaxPool2D(pool_size=(2,2)))\n",
    "model.add(Conv2D(32, (3, 3), padding='same', activation=\"relu\"))\n",
    "model.add(MaxPool2D(pool_size=(2,2)))\n",
    "model.add(Conv2D(32, (3, 3), padding='same', activation=\"relu\"))\n",
    "model.add(MaxPool2D(pool_size=(2,2)))\n",
    "model.add(Flatten())\n",
    "model.add(Dense(128,activation=\"relu\"))\n",
    "model.add(Dense(11, activation = \"softmax\"))\n",
    "model.summary()\n",
    "\n",
    "model.compile(loss=keras.losses.categorical_crossentropy, optimizer=tf.keras.optimizers.Adadelta(learning_rate=1.0, rho=0.95), metrics = [\"accuracy\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Training\n",
    "The input pictures are randomly scattered for brightness, pixel shift variations and rotation angle. This is implemented with a ImageDataGenerator.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 2.2329 - accuracy: 0.3049\n",
      "Epoch 2/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 1.5872 - accuracy: 0.4925\n",
      "Epoch 3/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 1.1866 - accuracy: 0.6364\n",
      "Epoch 4/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.9064 - accuracy: 0.7150 0s - loss: 0.9015 - accu - ETA: 0s - loss: 0.9\n",
      "Epoch 5/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.8126 - accuracy: 0.7514\n",
      "Epoch 6/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.7128 - accuracy: 0.7697 1s - l\n",
      "Epoch 7/500\n",
      "396/396 [==============================] - 4s 11ms/step - loss: 0.6657 - accuracy: 0.8073\n",
      "Epoch 8/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.6313 - accuracy: 0.8061\n",
      "Epoch 9/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.6305 - accuracy: 0.8042\n",
      "Epoch 10/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.5824 - accuracy: 0.8051\n",
      "Epoch 11/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.5110 - accuracy: 0.8454\n",
      "Epoch 12/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.5099 - accuracy: 0.8482\n",
      "Epoch 13/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.5142 - accuracy: 0.8547\n",
      "Epoch 14/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.4351 - accuracy: 0.8675\n",
      "Epoch 15/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.5058 - accuracy: 0.8393\n",
      "Epoch 16/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.4683 - accuracy: 0.8705\n",
      "Epoch 17/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.3843 - accuracy: 0.8885\n",
      "Epoch 18/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.4022 - accuracy: 0.8791\n",
      "Epoch 19/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.3691 - accuracy: 0.8818: 0s - loss: 0.3678 \n",
      "Epoch 20/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.4496 - accuracy: 0.8687\n",
      "Epoch 21/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.3585 - accuracy: 0.9063\n",
      "Epoch 22/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.3357 - accuracy: 0.8953: 0s - loss: 0.3343 - accuracy: 0.89 - ETA: 0s - loss: 0.3347 - accura\n",
      "Epoch 23/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.3157 - accuracy: 0.8932\n",
      "Epoch 24/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.3455 - accuracy: 0.8920\n",
      "Epoch 25/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.3010 - accuracy: 0.9097\n",
      "Epoch 26/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.3287 - accuracy: 0.9031\n",
      "Epoch 27/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.2761 - accuracy: 0.9211\n",
      "Epoch 28/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.2840 - accuracy: 0.9138\n",
      "Epoch 29/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.2891 - accuracy: 0.9139\n",
      "Epoch 30/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.2666 - accuracy: 0.9175\n",
      "Epoch 31/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.2579 - accuracy: 0.9137\n",
      "Epoch 32/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.2977 - accuracy: 0.9080\n",
      "Epoch 33/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.2533 - accuracy: 0.9182 0s - loss: 0.2523 - accuracy\n",
      "Epoch 34/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.2584 - accuracy: 0.9349\n",
      "Epoch 35/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.2475 - accuracy: 0.9294\n",
      "Epoch 36/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.3411 - accuracy: 0.9094\n",
      "Epoch 37/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.2582 - accuracy: 0.9287: 1s - loss: 0.2615 -  - ETA: 1s - loss: 0.2595 - accuracy: 0. - ETA: 1s\n",
      "Epoch 38/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.2961 - accuracy: 0.9213\n",
      "Epoch 39/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.2628 - accuracy: 0.9263\n",
      "Epoch 40/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.2460 - accuracy: 0.9257\n",
      "Epoch 41/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.2151 - accuracy: 0.9503\n",
      "Epoch 42/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.2220 - accuracy: 0.9223\n",
      "Epoch 43/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.1888 - accuracy: 0.9467\n",
      "Epoch 44/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.1940 - accuracy: 0.9453\n",
      "Epoch 45/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.2405 - accuracy: 0.9352\n",
      "Epoch 46/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.1905 - accuracy: 0.9407\n",
      "Epoch 47/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.2039 - accuracy: 0.9295\n",
      "Epoch 48/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.1995 - accuracy: 0.9498\n",
      "Epoch 49/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.2082 - accuracy: 0.9368\n",
      "Epoch 50/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.1869 - accuracy: 0.9471\n",
      "Epoch 51/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.1986 - accuracy: 0.9402\n",
      "Epoch 52/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.2258 - accuracy: 0.9302\n",
      "Epoch 53/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.2321 - accuracy: 0.9289\n",
      "Epoch 54/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.1759 - accuracy: 0.9427\n",
      "Epoch 55/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.1816 - accuracy: 0.9505\n",
      "Epoch 56/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.2069 - accuracy: 0.9471\n",
      "Epoch 57/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.1884 - accuracy: 0.9375\n",
      "Epoch 58/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.1520 - accuracy: 0.9569\n",
      "Epoch 59/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.2414 - accuracy: 0.9250\n",
      "Epoch 60/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.1539 - accuracy: 0.9398\n",
      "Epoch 61/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.1743 - accuracy: 0.9450\n",
      "Epoch 62/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.1843 - accuracy: 0.9424\n",
      "Epoch 63/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.1595 - accuracy: 0.9438\n",
      "Epoch 64/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.1671 - accuracy: 0.9568\n",
      "Epoch 65/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.2543 - accuracy: 0.9358\n",
      "Epoch 66/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.2097 - accuracy: 0.9354\n",
      "Epoch 67/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.1954 - accuracy: 0.9403\n",
      "Epoch 68/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.1606 - accuracy: 0.9480\n",
      "Epoch 69/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.1789 - accuracy: 0.9451\n",
      "Epoch 70/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.2137 - accuracy: 0.9513\n",
      "Epoch 71/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.1346 - accuracy: 0.9571: 0s - loss: 0.1344 - accuracy: 0.95\n",
      "Epoch 72/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.1743 - accuracy: 0.9512\n",
      "Epoch 73/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.1851 - accuracy: 0.9468\n",
      "Epoch 74/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.1787 - accuracy: 0.9436\n",
      "Epoch 75/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.1379 - accuracy: 0.9586\n",
      "Epoch 76/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.1321 - accuracy: 0.9615\n",
      "Epoch 77/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.1115 - accuracy: 0.9664\n",
      "Epoch 78/500\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "396/396 [==============================] - 4s 9ms/step - loss: 0.1872 - accuracy: 0.9440\n",
      "Epoch 79/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.1380 - accuracy: 0.9566\n",
      "Epoch 80/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.2197 - accuracy: 0.9463\n",
      "Epoch 81/500\n",
      "396/396 [==============================] - 3s 9ms/step - loss: 0.1918 - accuracy: 0.9487\n",
      "Epoch 82/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.1553 - accuracy: 0.9462\n",
      "Epoch 83/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.2009 - accuracy: 0.9465\n",
      "Epoch 84/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.1432 - accuracy: 0.9600\n",
      "Epoch 85/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.1475 - accuracy: 0.9488\n",
      "Epoch 86/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.1692 - accuracy: 0.9413\n",
      "Epoch 87/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.1050 - accuracy: 0.9650\n",
      "Epoch 88/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.1518 - accuracy: 0.9539\n",
      "Epoch 89/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.1576 - accuracy: 0.9487\n",
      "Epoch 90/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.1428 - accuracy: 0.9557\n",
      "Epoch 91/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.1430 - accuracy: 0.9547\n",
      "Epoch 92/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.1272 - accuracy: 0.9661\n",
      "Epoch 93/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.2195 - accuracy: 0.9419\n",
      "Epoch 94/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.1401 - accuracy: 0.9604\n",
      "Epoch 95/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.1257 - accuracy: 0.9646: 0s - loss: 0\n",
      "Epoch 96/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.1160 - accuracy: 0.9660\n",
      "Epoch 97/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.1181 - accuracy: 0.9610\n",
      "Epoch 98/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.1070 - accuracy: 0.9725\n",
      "Epoch 99/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.2028 - accuracy: 0.9559\n",
      "Epoch 100/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.1186 - accuracy: 0.9635\n",
      "Epoch 101/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.1462 - accuracy: 0.9509\n",
      "Epoch 102/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.1487 - accuracy: 0.9512\n",
      "Epoch 103/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.1620 - accuracy: 0.9587: 0s - loss: 0.1633 - accu\n",
      "Epoch 104/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.1160 - accuracy: 0.9588\n",
      "Epoch 105/500\n",
      "396/396 [==============================] - 5s 12ms/step - loss: 0.1317 - accuracy: 0.9558\n",
      "Epoch 106/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.1372 - accuracy: 0.9506\n",
      "Epoch 107/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0954 - accuracy: 0.9706\n",
      "Epoch 108/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.1099 - accuracy: 0.9641\n",
      "Epoch 109/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.1095 - accuracy: 0.9651\n",
      "Epoch 110/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.1142 - accuracy: 0.9703\n",
      "Epoch 111/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.1284 - accuracy: 0.9629\n",
      "Epoch 112/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.1035 - accuracy: 0.9722\n",
      "Epoch 113/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.1231 - accuracy: 0.9664\n",
      "Epoch 114/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.1230 - accuracy: 0.9599\n",
      "Epoch 115/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.1567 - accuracy: 0.9497\n",
      "Epoch 116/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.1339 - accuracy: 0.9591\n",
      "Epoch 117/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.1057 - accuracy: 0.9667\n",
      "Epoch 118/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.1067 - accuracy: 0.9679\n",
      "Epoch 119/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.1489 - accuracy: 0.9624\n",
      "Epoch 120/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.1099 - accuracy: 0.9617\n",
      "Epoch 121/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.1063 - accuracy: 0.9612\n",
      "Epoch 122/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.1037 - accuracy: 0.9694\n",
      "Epoch 123/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.1330 - accuracy: 0.9539\n",
      "Epoch 124/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.1592 - accuracy: 0.9585\n",
      "Epoch 125/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.1331 - accuracy: 0.9572\n",
      "Epoch 126/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0883 - accuracy: 0.9775\n",
      "Epoch 127/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.1181 - accuracy: 0.9625\n",
      "Epoch 128/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.1184 - accuracy: 0.9704\n",
      "Epoch 129/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.1479 - accuracy: 0.9559\n",
      "Epoch 130/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.1176 - accuracy: 0.9620\n",
      "Epoch 131/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.1075 - accuracy: 0.9786\n",
      "Epoch 132/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.1160 - accuracy: 0.9634\n",
      "Epoch 133/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.1112 - accuracy: 0.9682\n",
      "Epoch 134/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.1403 - accuracy: 0.9653\n",
      "Epoch 135/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0873 - accuracy: 0.9744\n",
      "Epoch 136/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.1132 - accuracy: 0.9592\n",
      "Epoch 137/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.1431 - accuracy: 0.9632\n",
      "Epoch 138/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.1089 - accuracy: 0.9671\n",
      "Epoch 139/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.1155 - accuracy: 0.9638\n",
      "Epoch 140/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.1413 - accuracy: 0.9669\n",
      "Epoch 141/500\n",
      "396/396 [==============================] - 4s 11ms/step - loss: 0.1165 - accuracy: 0.9598\n",
      "Epoch 142/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0964 - accuracy: 0.9688\n",
      "Epoch 143/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.1287 - accuracy: 0.9683\n",
      "Epoch 144/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0740 - accuracy: 0.9768\n",
      "Epoch 145/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.1690 - accuracy: 0.9539\n",
      "Epoch 146/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.1039 - accuracy: 0.9714 1s - l - ETA\n",
      "Epoch 147/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.1032 - accuracy: 0.9743\n",
      "Epoch 148/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.1069 - accuracy: 0.9711\n",
      "Epoch 149/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0652 - accuracy: 0.9767\n",
      "Epoch 150/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.1405 - accuracy: 0.9635\n",
      "Epoch 151/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.1042 - accuracy: 0.9674\n",
      "Epoch 152/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.1004 - accuracy: 0.9707\n",
      "Epoch 153/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0948 - accuracy: 0.9697 0s - loss: 0.0944 - accuracy: \n",
      "Epoch 154/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0981 - accuracy: 0.9658\n",
      "Epoch 155/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.1273 - accuracy: 0.9665\n",
      "Epoch 156/500\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0819 - accuracy: 0.9716\n",
      "Epoch 157/500\n",
      "396/396 [==============================] - 4s 11ms/step - loss: 0.0833 - accuracy: 0.9717\n",
      "Epoch 158/500\n",
      "396/396 [==============================] - 4s 11ms/step - loss: 0.1273 - accuracy: 0.9624\n",
      "Epoch 159/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.1084 - accuracy: 0.9749\n",
      "Epoch 160/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.1275 - accuracy: 0.9675\n",
      "Epoch 161/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0655 - accuracy: 0.9744\n",
      "Epoch 162/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.1273 - accuracy: 0.9639\n",
      "Epoch 163/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0974 - accuracy: 0.9660\n",
      "Epoch 164/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.1180 - accuracy: 0.9722\n",
      "Epoch 165/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.1061 - accuracy: 0.9743\n",
      "Epoch 166/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0885 - accuracy: 0.9735\n",
      "Epoch 167/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.1092 - accuracy: 0.9728\n",
      "Epoch 168/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0833 - accuracy: 0.9736\n",
      "Epoch 169/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0582 - accuracy: 0.9830\n",
      "Epoch 170/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.1164 - accuracy: 0.9666\n",
      "Epoch 171/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0755 - accuracy: 0.9788\n",
      "Epoch 172/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0929 - accuracy: 0.9730 0s - los\n",
      "Epoch 173/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0904 - accuracy: 0.9713\n",
      "Epoch 174/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0854 - accuracy: 0.9759\n",
      "Epoch 175/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.1101 - accuracy: 0.9599: 0s - loss: 0.1102 - accuracy: 0.95\n",
      "Epoch 176/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0880 - accuracy: 0.9766\n",
      "Epoch 177/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0962 - accuracy: 0.9661\n",
      "Epoch 178/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0886 - accuracy: 0.9726\n",
      "Epoch 179/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0898 - accuracy: 0.9733\n",
      "Epoch 180/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0923 - accuracy: 0.9819 0s - loss: 0\n",
      "Epoch 181/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.1064 - accuracy: 0.9650\n",
      "Epoch 182/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.1262 - accuracy: 0.9633: 0s - loss: 0.1269 - accuracy: 0.\n",
      "Epoch 183/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.1296 - accuracy: 0.9658\n",
      "Epoch 184/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0924 - accuracy: 0.9694\n",
      "Epoch 185/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0729 - accuracy: 0.9800: 0s - loss: 0.0720 - accuracy: \n",
      "Epoch 186/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0982 - accuracy: 0.9791\n",
      "Epoch 187/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0833 - accuracy: 0.9783\n",
      "Epoch 188/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.1345 - accuracy: 0.9657\n",
      "Epoch 189/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0810 - accuracy: 0.9711\n",
      "Epoch 190/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.1339 - accuracy: 0.9591 1s - loss: 0.1407 - ac - ETA: 0s - l\n",
      "Epoch 191/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0944 - accuracy: 0.9677\n",
      "Epoch 192/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0797 - accuracy: 0.9748\n",
      "Epoch 193/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.1160 - accuracy: 0.9637\n",
      "Epoch 194/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0974 - accuracy: 0.9825\n",
      "Epoch 195/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0892 - accuracy: 0.9705\n",
      "Epoch 196/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0889 - accuracy: 0.9761\n",
      "Epoch 197/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0691 - accuracy: 0.9721: 0s - loss: 0.065\n",
      "Epoch 198/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.1010 - accuracy: 0.9631\n",
      "Epoch 199/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0720 - accuracy: 0.9731\n",
      "Epoch 200/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.1266 - accuracy: 0.9642\n",
      "Epoch 201/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.1462 - accuracy: 0.9606\n",
      "Epoch 202/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0679 - accuracy: 0.9740\n",
      "Epoch 203/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0720 - accuracy: 0.9781\n",
      "Epoch 204/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0783 - accuracy: 0.9796: 2s\n",
      "Epoch 205/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0917 - accuracy: 0.9646\n",
      "Epoch 206/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0668 - accuracy: 0.9812\n",
      "Epoch 207/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0855 - accuracy: 0.9749\n",
      "Epoch 208/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0635 - accuracy: 0.9826\n",
      "Epoch 209/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0824 - accuracy: 0.9737\n",
      "Epoch 210/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.1034 - accuracy: 0.9707\n",
      "Epoch 211/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.1024 - accuracy: 0.9687: 0s - loss:\n",
      "Epoch 212/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0923 - accuracy: 0.9759: 2s - los - ETA\n",
      "Epoch 213/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0902 - accuracy: 0.9750: 2s - l\n",
      "Epoch 214/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0756 - accuracy: 0.9718: 0s - loss: 0.0724 - accuracy: 0.97 - ETA: 0s - loss: 0.072\n",
      "Epoch 215/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0986 - accuracy: 0.9678\n",
      "Epoch 216/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0463 - accuracy: 0.9799\n",
      "Epoch 217/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0941 - accuracy: 0.9721\n",
      "Epoch 218/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0540 - accuracy: 0.9807: 0s - loss: 0.0522 - accu\n",
      "Epoch 219/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0827 - accuracy: 0.9800\n",
      "Epoch 220/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.1358 - accuracy: 0.9616\n",
      "Epoch 221/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.1105 - accuracy: 0.9629\n",
      "Epoch 222/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0749 - accuracy: 0.9701\n",
      "Epoch 223/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.1058 - accuracy: 0.9725\n",
      "Epoch 247/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0721 - accuracy: 0.9745\n",
      "Epoch 248/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0750 - accuracy: 0.9807\n",
      "Epoch 249/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0804 - accuracy: 0.9766\n",
      "Epoch 250/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0755 - accuracy: 0.9705\n",
      "Epoch 251/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0681 - accuracy: 0.9788\n",
      "Epoch 252/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0940 - accuracy: 0.9723\n",
      "Epoch 253/500\n",
      "396/396 [==============================] - 6s 14ms/step - loss: 0.0710 - accuracy: 0.9797\n",
      "Epoch 254/500\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "396/396 [==============================] - 5s 12ms/step - loss: 0.0751 - accuracy: 0.9828\n",
      "Epoch 255/500\n",
      "396/396 [==============================] - 5s 13ms/step - loss: 0.0821 - accuracy: 0.9735 0s - loss: 0.0809 - accuracy: 0. - ETA: 0s - loss:\n",
      "Epoch 256/500\n",
      "396/396 [==============================] - 4s 11ms/step - loss: 0.0577 - accuracy: 0.9840 0s - loss:\n",
      "Epoch 257/500\n",
      "396/396 [==============================] - 4s 11ms/step - loss: 0.0777 - accuracy: 0.9788\n",
      "Epoch 258/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.1004 - accuracy: 0.9737\n",
      "Epoch 259/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0855 - accuracy: 0.9726\n",
      "Epoch 260/500\n",
      "396/396 [==============================] - 4s 11ms/step - loss: 0.0619 - accuracy: 0.9768\n",
      "Epoch 261/500\n",
      "396/396 [==============================] - 4s 11ms/step - loss: 0.0859 - accuracy: 0.9764\n",
      "Epoch 262/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0646 - accuracy: 0.9850\n",
      "Epoch 263/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0681 - accuracy: 0.9812\n",
      "Epoch 264/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.1160 - accuracy: 0.9681\n",
      "Epoch 265/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.1084 - accuracy: 0.9719\n",
      "Epoch 266/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.1019 - accuracy: 0.9709\n",
      "Epoch 267/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0775 - accuracy: 0.9810\n",
      "Epoch 268/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.1018 - accuracy: 0.9718\n",
      "Epoch 269/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0584 - accuracy: 0.9824\n",
      "Epoch 270/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.1003 - accuracy: 0.9689\n",
      "Epoch 271/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0878 - accuracy: 0.9732\n",
      "Epoch 272/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0891 - accuracy: 0.9733\n",
      "Epoch 273/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.1050 - accuracy: 0.9659\n",
      "Epoch 274/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0741 - accuracy: 0.9728\n",
      "Epoch 275/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0568 - accuracy: 0.9823\n",
      "Epoch 276/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.1365 - accuracy: 0.9647\n",
      "Epoch 277/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0820 - accuracy: 0.9785\n",
      "Epoch 278/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0752 - accuracy: 0.9767\n",
      "Epoch 279/500\n",
      "396/396 [==============================] - 4s 11ms/step - loss: 0.0580 - accuracy: 0.9837\n",
      "Epoch 280/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0810 - accuracy: 0.9760\n",
      "Epoch 281/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0686 - accuracy: 0.9779\n",
      "Epoch 282/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0732 - accuracy: 0.9796\n",
      "Epoch 283/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0948 - accuracy: 0.9767\n",
      "Epoch 284/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0848 - accuracy: 0.9673 0s - los\n",
      "Epoch 285/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0648 - accuracy: 0.9780\n",
      "Epoch 286/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0944 - accuracy: 0.9727\n",
      "Epoch 305/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0572 - accuracy: 0.9819\n",
      "Epoch 306/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0801 - accuracy: 0.9748\n",
      "Epoch 307/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0550 - accuracy: 0.9840\n",
      "Epoch 308/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0638 - accuracy: 0.9895\n",
      "Epoch 309/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0696 - accuracy: 0.9834: 0s - loss: 0.0695 - accuracy\n",
      "Epoch 310/500\n",
      "396/396 [==============================] - 5s 12ms/step - loss: 0.0779 - accuracy: 0.9739\n",
      "Epoch 311/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.1207 - accuracy: 0.9685\n",
      "Epoch 312/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0621 - accuracy: 0.9830\n",
      "Epoch 313/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0993 - accuracy: 0.9904\n",
      "Epoch 314/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0778 - accuracy: 0.9802\n",
      "Epoch 315/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0838 - accuracy: 0.9758  - ETA: 0s - loss: 0.0851 - accuracy:  - ETA: 0s - loss: 0\n",
      "Epoch 316/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0675 - accuracy: 0.9758\n",
      "Epoch 317/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0621 - accuracy: 0.9787\n",
      "Epoch 318/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0824 - accuracy: 0.9737\n",
      "Epoch 319/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0719 - accuracy: 0.9776\n",
      "Epoch 320/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0683 - accuracy: 0.9808\n",
      "Epoch 321/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0550 - accuracy: 0.9825\n",
      "Epoch 322/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0783 - accuracy: 0.9828\n",
      "Epoch 323/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0654 - accuracy: 0.9773: 0s - l\n",
      "Epoch 324/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0544 - accuracy: 0.9806\n",
      "Epoch 325/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0933 - accuracy: 0.9732\n",
      "Epoch 326/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0832 - accuracy: 0.9760\n",
      "Epoch 327/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.1329 - accuracy: 0.9693\n",
      "Epoch 328/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0648 - accuracy: 0.9842: 0s - loss: 0.0643 - accu\n",
      "Epoch 329/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.1501 - accuracy: 0.9624\n",
      "Epoch 330/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0792 - accuracy: 0.9736: 0s - loss:\n",
      "Epoch 331/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0496 - accuracy: 0.9845\n",
      "Epoch 332/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0578 - accuracy: 0.9830\n",
      "Epoch 333/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0588 - accuracy: 0.9816: 1s - loss: 0.0569 -  - ETA: 0s - loss: 0.0\n",
      "Epoch 334/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0879 - accuracy: 0.9731\n",
      "Epoch 335/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0934 - accuracy: 0.9769\n",
      "Epoch 336/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0900 - accuracy: 0.9789 0s -\n",
      "Epoch 337/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.1038 - accuracy: 0.9729\n",
      "Epoch 338/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0889 - accuracy: 0.9809\n",
      "Epoch 339/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0737 - accuracy: 0.9765\n",
      "Epoch 340/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.1337 - accuracy: 0.9781 ETA:  - E - ETA: \n",
      "Epoch 341/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0857 - accuracy: 0.9721\n",
      "Epoch 342/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0785 - accuracy: 0.9822\n",
      "Epoch 343/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0457 - accuracy: 0.9859 3s - loss: 0.0143 - accura - ETA: 3s - loss: 0.0246  - E\n",
      "Epoch 344/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0921 - accuracy: 0.9722\n",
      "Epoch 345/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0632 - accuracy: 0.9786\n",
      "Epoch 346/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0903 - accuracy: 0.9830\n",
      "Epoch 347/500\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0505 - accuracy: 0.9827 0s - loss: 0.0474 - accuracy: 0.98 - ETA: 0s - loss: 0.0476 \n",
      "Epoch 348/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0399 - accuracy: 0.9810\n",
      "Epoch 349/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0864 - accuracy: 0.9778\n",
      "Epoch 350/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0980 - accuracy: 0.9703\n",
      "Epoch 351/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0849 - accuracy: 0.9698: 0s - loss: 0.0852 - ac\n",
      "Epoch 352/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0556 - accuracy: 0.9859\n",
      "Epoch 353/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0697 - accuracy: 0.9790\n",
      "Epoch 354/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0663 - accuracy: 0.9815\n",
      "Epoch 355/500\n",
      "396/396 [==============================] - 3s 9ms/step - loss: 0.0603 - accuracy: 0.9790 E\n",
      "Epoch 356/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0618 - accuracy: 0.9807\n",
      "Epoch 357/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.1295 - accuracy: 0.9716\n",
      "Epoch 358/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0868 - accuracy: 0.9787\n",
      "Epoch 359/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0565 - accuracy: 0.9870\n",
      "Epoch 377/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0498 - accuracy: 0.9809\n",
      "Epoch 378/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.1040 - accuracy: 0.9650\n",
      "Epoch 379/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0618 - accuracy: 0.9758\n",
      "Epoch 380/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0563 - accuracy: 0.9802: 0s - los\n",
      "Epoch 381/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0873 - accuracy: 0.9800\n",
      "Epoch 382/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.1155 - accuracy: 0.9723 1s - - ETA: 0s - loss: 0.121\n",
      "Epoch 383/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0800 - accuracy: 0.9775\n",
      "Epoch 384/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0891 - accuracy: 0.9725\n",
      "Epoch 385/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0523 - accuracy: 0.9835\n",
      "Epoch 386/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0701 - accuracy: 0.9750\n",
      "Epoch 387/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0543 - accuracy: 0.9847\n",
      "Epoch 388/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0440 - accuracy: 0.9875\n",
      "Epoch 389/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0952 - accuracy: 0.9826\n",
      "Epoch 390/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0987 - accuracy: 0.9715\n",
      "Epoch 391/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0701 - accuracy: 0.9782: 1s - loss: 0.0\n",
      "Epoch 392/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0804 - accuracy: 0.9726\n",
      "Epoch 393/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0624 - accuracy: 0.9792\n",
      "Epoch 394/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0661 - accuracy: 0.9780\n",
      "Epoch 395/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0644 - accuracy: 0.9794\n",
      "Epoch 396/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0756 - accuracy: 0.9717\n",
      "Epoch 397/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0697 - accuracy: 0.9773\n",
      "Epoch 398/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0574 - accuracy: 0.9835 1s - loss: 0.0524 - accuracy: \n",
      "Epoch 399/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0613 - accuracy: 0.9848\n",
      "Epoch 400/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0771 - accuracy: 0.9781\n",
      "Epoch 401/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0663 - accuracy: 0.9807\n",
      "Epoch 402/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0898 - accuracy: 0.9747 0s -\n",
      "Epoch 403/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0524 - accuracy: 0.9877\n",
      "Epoch 404/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0440 - accuracy: 0.9833\n",
      "Epoch 405/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0715 - accuracy: 0.9758\n",
      "Epoch 406/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0791 - accuracy: 0.9760\n",
      "Epoch 407/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0586 - accuracy: 0.9779\n",
      "Epoch 408/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0527 - accuracy: 0.9869\n",
      "Epoch 409/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0707 - accuracy: 0.9773\n",
      "Epoch 410/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0775 - accuracy: 0.9829 0s - loss: 0.0757 \n",
      "Epoch 411/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0795 - accuracy: 0.9808\n",
      "Epoch 412/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0701 - accuracy: 0.9730\n",
      "Epoch 413/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0973 - accuracy: 0.9741\n",
      "Epoch 414/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0630 - accuracy: 0.9820 0s - loss: 0.0610 - ac\n",
      "Epoch 415/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0705 - accuracy: 0.9774\n",
      "Epoch 416/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0736 - accuracy: 0.9738\n",
      "Epoch 417/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0630 - accuracy: 0.9806\n",
      "Epoch 418/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0593 - accuracy: 0.9829\n",
      "Epoch 419/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.1128 - accuracy: 0.9706 0s - loss: 0.1137 - accuracy: 0.\n",
      "Epoch 420/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0620 - accuracy: 0.9798\n",
      "Epoch 421/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0444 - accuracy: 0.9881\n",
      "Epoch 422/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0730 - accuracy: 0.9810\n",
      "Epoch 423/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0428 - accuracy: 0.9879\n",
      "Epoch 424/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0458 - accuracy: 0.9809\n",
      "Epoch 425/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0583 - accuracy: 0.9813 1s - - ETA: 0s - loss: 0.0550 - \n",
      "Epoch 426/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0781 - accuracy: 0.9788\n",
      "Epoch 427/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0597 - accuracy: 0.9886\n",
      "Epoch 428/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0690 - accuracy: 0.9769\n",
      "Epoch 429/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0802 - accuracy: 0.9821\n",
      "Epoch 438/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0558 - accuracy: 0.9835\n",
      "Epoch 439/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0936 - accuracy: 0.9755\n",
      "Epoch 440/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0806 - accuracy: 0.9746\n",
      "Epoch 441/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.1058 - accuracy: 0.9676\n",
      "Epoch 442/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0472 - accuracy: 0.9880 0s - loss: 0.0448 - accura\n",
      "Epoch 443/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0633 - accuracy: 0.9831\n",
      "Epoch 444/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0484 - accuracy: 0.9850\n",
      "Epoch 445/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0705 - accuracy: 0.9789 1s - loss: 0.0665 - accuracy:  - ETA: \n",
      "Epoch 446/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0555 - accuracy: 0.9887\n",
      "Epoch 447/500\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0633 - accuracy: 0.9809\n",
      "Epoch 448/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0587 - accuracy: 0.9807\n",
      "Epoch 449/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0513 - accuracy: 0.9868\n",
      "Epoch 450/500\n",
      "396/396 [==============================] - 4s 9ms/step - loss: 0.0654 - accuracy: 0.9756\n",
      "Epoch 451/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0558 - accuracy: 0.9846 0s - loss: 0.0554 - accuracy: 0.\n",
      "Epoch 452/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0533 - accuracy: 0.9861\n",
      "Epoch 453/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0523 - accuracy: 0.9870\n",
      "Epoch 454/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0679 - accuracy: 0.9815\n",
      "Epoch 455/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0570 - accuracy: 0.9839\n",
      "Epoch 456/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0753 - accuracy: 0.9752\n",
      "Epoch 457/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0883 - accuracy: 0.9767\n",
      "Epoch 458/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0977 - accuracy: 0.9804 0s - loss: 0.0981 - accuracy: 0.\n",
      "Epoch 459/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0499 - accuracy: 0.9842\n",
      "Epoch 460/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0674 - accuracy: 0.9796 0s - loss: 0.0\n",
      "Epoch 461/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0617 - accuracy: 0.9810\n",
      "Epoch 462/500\n",
      "396/396 [==============================] - 4s 11ms/step - loss: 0.1069 - accuracy: 0.9802\n",
      "Epoch 463/500\n",
      "396/396 [==============================] - 4s 11ms/step - loss: 0.0447 - accuracy: 0.9878\n",
      "Epoch 464/500\n",
      "396/396 [==============================] - 4s 11ms/step - loss: 0.0684 - accuracy: 0.9797\n",
      "Epoch 465/500\n",
      "396/396 [==============================] - 4s 11ms/step - loss: 0.0684 - accuracy: 0.9809\n",
      "Epoch 466/500\n",
      "396/396 [==============================] - 5s 11ms/step - loss: 0.0525 - accuracy: 0.9783\n",
      "Epoch 467/500\n",
      "396/396 [==============================] - 4s 11ms/step - loss: 0.0474 - accuracy: 0.9882\n",
      "Epoch 468/500\n",
      "396/396 [==============================] - 4s 11ms/step - loss: 0.0627 - accuracy: 0.9832\n",
      "Epoch 469/500\n",
      "396/396 [==============================] - 4s 10ms/step - loss: 0.0840 - accuracy: 0.9780\n",
      "Epoch 470/500\n",
      "396/396 [==============================] - 4s 11ms/step - loss: 0.0744 - accuracy: 0.9778\n",
      "Epoch 471/500\n",
      "396/396 [==============================] - 5s 11ms/step - loss: 0.0782 - accuracy: 0.9792\n",
      "Epoch 472/500\n",
      "396/396 [==============================] - 4s 11ms/step - loss: 0.0571 - accuracy: 0.9857\n",
      "Epoch 473/500\n",
      "396/396 [==============================] - 4s 11ms/step - loss: 0.0363 - accuracy: 0.9885\n",
      "Epoch 474/500\n",
      "396/396 [==============================] - 4s 11ms/step - loss: 0.0584 - accuracy: 0.9883 0s - loss: 0.0583 - accuracy: 0.98\n",
      "Epoch 475/500\n",
      "396/396 [==============================] - 5s 11ms/step - loss: 0.1287 - accuracy: 0.9673\n",
      "Epoch 499/500\n",
      "396/396 [==============================] - 4s 11ms/step - loss: 0.0566 - accuracy: 0.9825\n",
      "Epoch 500/500\n",
      "396/396 [==============================] - 4s 11ms/step - loss: 0.0489 - accuracy: 0.9829\n"
     ]
    }
   ],
   "source": [
    "Batch_Size = 4\n",
    "Shift_Range = 1\n",
    "Brightness_Range = 0.3\n",
    "Rotation_Angle = 10\n",
    "ZoomRange = 0.4\n",
    "\n",
    "datagen = ImageDataGenerator(width_shift_range=[-Shift_Range,Shift_Range], \n",
    "                             height_shift_range=[-Shift_Range,Shift_Range],\n",
    "                             brightness_range=[1-Brightness_Range,1+Brightness_Range],\n",
    "                             zoom_range=[1-ZoomRange, 1+ZoomRange],\n",
    "                             rotation_range=Rotation_Angle)\n",
    "\n",
    "if (Training_Percentage > 0):\n",
    "    train_iterator = datagen.flow(x_data, y_data, batch_size=Batch_Size)\n",
    "    validation_iterator = datagen.flow(X_test, y_test, batch_size=Batch_Size)\n",
    "    history = model.fit(train_iterator, validation_data = validation_iterator, epochs = Epoch_Anz)\n",
    "else:\n",
    "    train_iterator = datagen.flow(x_data, y_data, batch_size=Batch_Size)\n",
    "    history = model.fit(train_iterator, epochs = Epoch_Anz)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Learing result\n",
    " \n",
    "* Visualization of the training and validation results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAEWCAYAAAB42tAoAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAABCiElEQVR4nO3dd3wb9fkH8M+jLXnbcRInTuLsQSYxGZCQsEPC3rtl09IWfgVaRiltoZQCBQqUDWU2QBmFQIASIIxmkZCETLJwduIRb1v7+/vjhu5OJ1kesmz5eb9eeUU6nU7fk+177rueLwkhwBhjjMViSXUBGGOMdW0cKBhjjMXFgYIxxlhcHCgYY4zFxYGCMcZYXBwoGGOMxcWBgrEOREQvEtE9Ce5bRkTHt/c4jCUbBwrGGGNxcaBgjDEWFwcK1uPITT63ENH3RNRIRM8TUR8i+oiI6oloERHlafY/jYg2EFENES0motGa1yYR0Xfy+94A4DJ81ilEtEZ+7xIiGt/GMl9NRNuI6BARvU9E/eTtREQPE1E5EdUR0ToiGiu/NpeINspl20tEN7fpC2M9HgcK1lOdDeAEACMAnArgIwC3AyiE9HfxKwAgohEA5gO4UX5tIYAFROQgIgeA/wB4BUA+gH/Lx4X83kkAXgBwLYACAE8DeJ+InK0pKBEdC+AvAM4DUARgJ4DX5ZdPBHC0fB458j5V8mvPA7hWCJEFYCyAz1vzuYwpOFCwnuoxIcRBIcReAF8DWC6EWC2E8AJ4F8Akeb/zAXwohPhUCBEA8CAAN4AjAUwDYAfwiBAiIIR4C8C3ms+4BsDTQojlQoiQEOIlAD75fa1xMYAXhBDfCSF8AG4DMJ2ISgAEAGQBGAWAhBCbhBD75fcFAIwhomwhRLUQ4rtWfi5jADhQsJ7roOZxs8nzTPlxP0h38AAAIUQYwG4A/eXX9gp9Zs2dmseDANwkNzvVEFENgAHy+1rDWIYGSLWG/kKIzwE8DuAfAMqJ6BkiypZ3PRvAXAA7iehLIpreys9lDAAHCsZasg/SBR+A1CcA6WK/F8B+AP3lbYqBmse7AfxZCJGr+ecRQsxvZxkyIDVl7QUAIcSjQojJAMZAaoK6Rd7+rRDidAC9ITWRvdnKz2UMAAcKxlryJoB5RHQcEdkB3ASp+WgJgKUAggB+RUR2IjoLwBTNe58FcB0RTZU7nTOIaB4RZbWyDPMBXE5EE+X+jXshNZWVEdER8vHtABoBeAGE5T6Ui4koR24yqwMQbsf3wHowDhSMxSGE+AHAJQAeA1AJqeP7VCGEXwjhB3AWgJ8COASpP+MdzXtXArgaUtNQNYBt8r6tLcMiAHcCeBtSLWYogAvkl7MhBaRqSM1TVQAekF+7FEAZEdUBuA5SXwdjrUa8cBFjjLF4uEbBGGMsLg4UjDHG4uJAwRhjLC4OFIwxxuKypboAydCrVy9RUlKS6mIwxli3sWrVqkohRKHZa2kZKEpKSrBy5cpUF4MxxroNItoZ6zVuemKMMRYXBwrGGGNxcaBgjDEWV1r2UZgJBALYs2cPvF5vqouSVC6XC8XFxbDb7akuCmMsTfSYQLFnzx5kZWWhpKQE+mSf6UMIgaqqKuzZsweDBw9OdXEYY2mixzQ9eb1eFBQUpG2QAAAiQkFBQdrXmhhjnavHBAoAaR0kFD3hHBljnatHBYqWVDb4UNPkT3UxGGOsS+FAoVHV4EdtcyApx66pqcETTzzR6vfNnTsXNTU1HV8gxhhLEAcKDQsByVqeI1agCAaDcd+3cOFC5ObmJqdQjDGWgB4z6ikRRIRwkiLFrbfeiu3bt2PixImw2+1wuVzIy8vD5s2bsWXLFpxxxhnYvXs3vF4vbrjhBlxzzTUAIulIGhoacPLJJ2PGjBlYsmQJ+vfvj/feew9utzsp5WWMMUVaBQoiOhXAqcOGDYu73x8XbMDGfXVR272BEADAZbe2+rPH9MvGXaceFvP1++67D+vXr8eaNWuwePFizJs3D+vXr1eHsb7wwgvIz89Hc3MzjjjiCJx99tkoKCjQHWPr1q2YP38+nn32WZx33nl4++23cckll7S6rIwx1hpp1fQkhFgghLgmJyen7cfowPLEM2XKFN1ch0cffRQTJkzAtGnTsHv3bmzdujXqPYMHD8bEiRMBAJMnT0ZZWVknlZYx1pOlVY0iUbHu/HdWNcIXDGNEn6yklyEjI0N9vHjxYixatAhLly6Fx+PB7NmzTedCOJ1O9bHVakVzc3PSy8kYY2lVo2gvAkEkqY8iKysL9fX1pq/V1tYiLy8PHo8HmzdvxrJly5JSBsYYa4seWaOIhQgIJ6ntqaCgAEcddRTGjh0Lt9uNPn36qK/NmTMHTz31FEaPHo2RI0di2rRpySkEY4y1ASXrDjqVSktLhXHhok2bNmH06NFx37e3ugm1zUGM6ZedzOIlXSLnyhhjWkS0SghRavYaNz1pJHN4LGOMdVccKDSSOeGOMca6qx4VKFpqZiMiCIikdWh3hu5cdsZY19RjAoXL5UJVVVXcC6mSeLW7XmuV9ShcLleqi8IYSyM9ZtRTcXEx9uzZg4qKipj7NPiCqGkKwFrrgsXSPdN1KyvcMcZYR+kxgcJut7e46tu/lu/C7e+vw/Lbj0OfbL4rZ4wxoAc1PSXCaZO+Dl8gnOKSMMZY18GBQsNplwNFMJTikjDGWNfBgULDaZOyxvqCXKNgjDEFBwoNtemJaxSMMabiQKGhBAov91EwxpiKA4VGhlMaBNboi788KWOM9SQcKDQy5UDRwIGCMcZUHCg0Ml1SoKj3cqBgjDEFBwqNLBfXKBhjzIgDhYbTZoXDauEaBWOMaXCgMMh02dDgC6S6GIwx1mVwoDDIctm4RsEYYxocKAwynTY0cKBgjDEVBwqDTKcN9dyZzRhjKg4UBtz0xBhjehwoDLJcdu7MZowxDQ4UBtxHwRhjehwoDKThscG4a2szxlhPwoHCIMtlQyAkeE0KxhiTdfk1s4koA8ATAPwAFgshXkvm52U5I/meXHZrMj+KMca6hZTUKIjoBSIqJ6L1hu1ziOgHItpGRLfKm88C8JYQ4moApyW7bJmc74kxxnRS1fT0IoA52g1EZAXwDwAnAxgD4EIiGgOgGMBuebekLz2X6bQDAOq9PPKJMcaAFAUKIcRXAA4ZNk8BsE0IsUMI4QfwOoDTAeyBFCyAOOUlomuIaCURrayoqGhz2dQMsjzyiTHGAHStzuz+iNQcAClA9AfwDoCziehJAAtivVkI8YwQolQIUVpYWNjmQiiLF/HsbMYYk3T5zmwhRCOAyzvr87hGwRhjel2pRrEXwADN82J5W6fKcnEfBWOMaXWlQPEtgOFENJiIHAAuAPB+ZxdCaXqq4xoFY4wBSN3w2PkAlgIYSUR7iOhKIUQQwC8AfAJgE4A3hRAbWnncU4nomdra2jaXzWGzIM9jx8E6b5uPwRhj6SQlfRRCiAtjbF8IYGE7jrsAwILS0tKr23oMAOiX68a+mub2HIIxxtJGV2p66jKKctzYX8s1CsYYAzhQmOqf68JerlEwxhgADhSm+uW6Ue8N8sgnxhgDBwpT/XLdAMDNT4wxhjQLFB0x6gkA+uW6AICbnxhjDGkWKIQQC4QQ1+Tk5LTrOEqNgkc+McZYmgWKjtI7ywWrhThQMMYYOFCYsloIfbNd2F/DfRSMMcaBIobBvTKw+UB9qovBGGMpx4EihsmD8rDpQB3qeIgsY6yH40ARw5TB+RACWLOrJtVFYYyxlEqrQNFRw2MBYFCBBwBwgOdSMMZ6uLQKFB01PBYACjKcAICqRn+7j8UYY91ZWgWKjuR2WOG2W1HV4Et1URhjLKU4UMSRn+HAIa5RMMZ6OA4UcfTKdHDTE2Osx+NAEQfXKBhjjANFXHkZDqzbW4v31uxNdVEYYyxlOFDEceKYPgCA11fsTnFJGGMsddIqUHTkPAoAmDO2CFfNGIxVO6vhDYQ65JiMMdbdpFWg6Mh5FIpJA/PgD4Wxo6Kxw47JGGPdSVoFimToky1NvKvk+RSMsR6KA0ULemVKgaKingMFY6xn4kDRgsIsOVBwjYIx1kNxoGhBhtMGt92KSq5RMMZ6KA4UCSjMcnKNgjHWY3GgSEBhlhPvrdmHHRUNqS4KY4x1Og4UCbjx+OFwWC2476PNeGVpGcJhkeoiMcZYp0mrQNHRE+4UM4cX4vwjBuC/Gw/izvc24IN1+zv0+Iwx1pWlVaBIxoQ7xYB8t/q4tokTBTLGeo60ChTJlOt2qI9D3PTEGOtBOFAkKNttVx/7Q+EUloQxxjoXB4oE5WgCRXVTIIUlYYyxzsWBIkG6QMGLGTHGehAOFAnK9UQCBa96xxjrSThQJEhbo9hW3sBzKRhjPQYHigR5HFb18Y7KRry8tCx1hWGMsU7EgSJBRAQAmDWiEBOKc/DKsp0QgmsVjLH0x4GiFTb+6SQ895NSnDmpP7ZXNGLUnR9jX01zqovFGGNJlVaBIlkpPBQehw12qwXjB+QCAHzBMD5efyApn8UYY11FWgWKZKbw0BpTlK0+5lnajLF0l1aBorO47FacMKYPAGAvNz0xxtIcB4o2evayUozsk8WBgjGW9jhQtEP/PDd2VTWluhiMMZZUHCja4cihBfjhYD1Wlh1KdVEYYyxpOFC0w8VTByHLacMVL36LJdsrU10cxhhLCg4U7eB2WDGybxbqvEFc9OxyrN5VneoiMcZYh+NA0U4/P2ao+vjMJ5aksCSMMZYcHCja6dhRffDBL2eoz72BUApLwxhjHS+hQEFENxBRNkmeJ6LviOjEZBeuuxjbPwe/nTMKAHCg1pvi0jDGWMdKtEZxhRCiDsCJAPIAXArgvqSVqhsa11+aDX7M3xajyR9McWkYY6zjJBooSP5/LoBXhBAbNNsYgL45TgCAEMCm/fUpLg1jjHWcRAPFKiL6L6RA8QkRZQEIJ69Y3U+fbJf6+IVvfkSzn/sqGGPpIdFAcSWAWwEcIYRoAmAHcHnSStVGyc4eG0+Wy44pg/MBAB+u249HPtuC7RUN2LivrtPLwhhjHSnRQDEdwA9CiBoiugTA7wB0/tW4BZ2VPTaWN66Zpj6uavDjnCeXYO6jX2P3IU7zwRjrvhINFE8CaCKiCQBuArAdwMtJK1U3payCBwBvrdqD6qYAAOC8p5dyOnLGWLeVaKAICmndz9MBPC6E+AeArOQVq/uaNaJQ9zzPY8f+Wi8ONfpTVCLGGGufRANFPRHdBmlY7IdEZIHUT8EMXrpiCpy2yNd64pi+AICKel+qisQYY+2SaKA4H4AP0nyKAwCKATyQtFJ1c09ecjiOG9UbP5s9FGdM6g8AqGjgQMEY655IalFKYEeiPgCOkJ+uEEKUJ61U7VRaWipWrlyZ6mIAAHZVNeHoB77AYf2yMb44F/ecMRZWC09BYYx1LUS0SghRavZaoik8zgOwAsC5AM4DsJyIzum4IqavwixpIt6GfXWYv2IX3l29F0IIzLz/c7y5cneKS8cYYy2zJbjfHZDmUJQDABEVAlgE4K1kFSxduB1W3fM91U3wBsLYfagZv3nre5xXOiBFJWOMscQk2kdhMTQ1VbXivT3elTMG476zxiHDYUW9N4hGzgXFGOtGEq1RfExEnwCYLz8/H8DC5BQp/dx5yhgAwCOLtqLeG0CjjwMFY6z7SKhWIIS4BcAzAMbL/54RQvw2mQVLR1kuG+q9QTRoAsWlzy/HzqrGFJaKMcbiS7RGASHE2wDeTmJZ0l6Wy4Zvyw7hhwOR7LJfb63EgrX78Itjh6ewZIwxFlvcGgUR1RNRncm/eiLibHetlOWyo7LBj1+/uVa3ffmPh1JUIsYYa1ncGoUQgtN0dCCPYQQUIA2fXb2rBuV1XvSWU5UHQ2HYrDxWgDHWNfDVqBM1maxRMW9cERp8QUy59zNsOViPL7dUYNgdH2HTfq6wMca6Bg4UnaimOaB7/tlNs1Bakqc+33qwAQvW7gMArDA0Rx1q9OO6V1ZhTzWnLGeMdS4OFJ2ozhAoBuR5UJTjVp8fqPMiGJIWDvzvxgP45fzVCMvpyR//fBs+3nAAr6/g2dyMsc7FgaIT3X36WIztn60+d9gsKMqJLKFaVtmIPdXNAID/bavCgrX7UCsHlxVlVQCAHyt5KC1jrHNxoOhEM4b3wge/nKnb1lvOBQUAryzbiZU7q3WvK3MuqhulgLF6l/51xhhLtrQKFKlcM7s1zistRkGGAwBgs1rw8PkTcNMJI9TXhxRmqI+VQFHnlQLFvlov/rJwE15aUoZGXxDPfb0DobDA5gN1OFjn7cSzYIz1FAlPuOsOhBALACwoLS29OtVlief+cybonp85qRgA8LdPtwAA7pw3Bpe/+C0AoN4bRCgs0OALYtaIQny5pQJPf7UDALBpfx1e/3Y3hvXOxE//+S1sFsK2e+d24pkwxnqCtKpRdHcPnjsBkwbmYtaIQjx+0SQAwCXPLceMv34OIYBpQwpgt0bWsli/T6o5KSuKBMMCH68/oL7+8tIybCtv6LTyM8bSEweKLuScycV49+dHwWIhjC6SOr39oTD210pNSgUZDvTKjPRpbNwnzbXQjqa67tVV0jZvAL9/bwMuf3FFZxWfMZamOFB0UVnO6FbBbLcNOe7IUuXyyNmoYbe+YAi7qqT5FuFw8srIGOsZOFB0UVkue9S2bJcdNmv0MqoVDX7d8301XpTJGWn7ZDuj9meMsdbgQNFFueyRH02mXLvIdtthtUS2DyrwwOOw4mCtfrTTj5UNuOu9DQCA3lmReRreQAjNJmlEGGMsHg4UXRRRpOaQ7ZIChcdhhd0S2W63WpDrtuNgvT5Q3PHuelQ1SrUMTVzBcX/7EmPu+jiJpWaMpSMOFN3Ar08cCQDone3SNT3ZLIQcjwMH63y6/ZXO716ZDl0NYm9NM4SI7LezqhGBEHdiMMbi40DRhb18xRR8dMNMnDO5GGX3zUOm04ZfahY4ctgsyHHbdBPtrp01BP1z3Th3cjGG9MqENxAdCP6+aCuqG/2Y9cBi/OH9DZ1yLoyx7osDRRd29IhCdZis4qhhvfDPy48AIDU9FeW4cagx0pnttlux+JbZ+OvZ4+FyWNEciO6TeHjRFlQ3Se9Z/ENFEs+AMZYOOFB0Qw55USObhTCsd6buNSsR7FYLLBaCy2aB1yRQAFADheJQo1/NXMsYY1ocKLohpV/BbrVEBQqLprPb7bDqAoVDs2peudyvIYRAvTeAw+/+FA988kMyi80Y66Y4UHRDEwfkIsdtxw3HD48KFBOKc9XHbnuk6UkIgUA4rCYcrGiQAkVYACvLpIy0S7ZXdXhZd1U1wR/kmgpj3RkHim4o1+PA2rtOxBEl+SjOiyx89NIVUzBjeC/1uctuxcE6H3ZUNGDinz6FEJG05mqNAgJfbZX6KUb0ib9E+m/eWotXlpYlXM7qRj+OfuAL3P3BxoTfwxjrejhQdHNOm1V9PDDfo3vNZZdeO/ZvX6oLIBXKE/DK5bkXTb4Q3l29FwAQaiHfx5sr9+BOeSJfvTeA615ZhfI4qc3L66Vg9M22yoTPhzHW9XCgSCNOmyXucwAolJMKVsgX8XpfEDVNUhBR1r6oavCpzUWHGv147LOtUfMt/rNmHz7ecAAPL9oaszzKZ2j7Rhhj3U9arUfR0yk1CEWtIVkgAPSWcz8pd/uKPI8dDb4g6r0BTL5nES6dNghzxxXhwmeXAQAmDMjV7a/0mQvtDD6NX81fjb010rKuTjsHCsa6Mw4UacRluCDvr22O2kfpo1DW5lYMLMhAgy+ID7/fDwBYvKUcmgFU6toXCoL04qYD9Wj0BZFhyHb7+eZytYZiVrNhjHUf/BecBq6dNQSAvr8CAI4b1QeA1Heh5IsqkJuetLUNIqA4z40GbxCr5DW7+2S5UNXoR0mBBzluO9bsqtEdu1EOAmt31+AKeTU+hT8YVoMEAARC5rUOxlj3wDWKNHDrnFH47UmjdHMoAODc0mKcPqkfnDYrLn1+Ob7eWomwSVNRnseBbJcdZVVNKJPXsVi5sxo5bjuGFmagV6YTa3bXqPu/vWoP/rxwk/p8+Y+HdMerMUzmM66X0RaLNh6E1UI4ZlTvFvcNyQt1WC3RKdnbQwiBTzYcwAlj+nb4sRnryrhGkQaIKCpIKNuVWsZZh/cHAAwuyIjaz2YhZDisUdtrmwPIz3BiQL5H16fxwv9+jNpXm3ywukkfGMz6SswIIWL2eVz18kp1HfGWjPvDJ5j94BcJ7dsaC9cdwHWvfodn5DXLGespOFD0EGdOKsbmu+egpFcGHjpvgu618npf1MVdUZDhUPs1FPkZjqj9fvP292qCQW3uKUAKFEIIbD1YrwaUD77fh7LKRnWfOm8Ag29biOe/iQ5CrdXkD2H3oWbM/fvX7T6WVqU8SXFvTVOHHpexro4DRQ+ijIo66/BiXDx1II4ZWQhAWu+iXLOmxRkT++km8vXOdumOk+uJDhQL1u7Di0vKEA6LqKanYFigrjmIEx7+Cte8shIA8It/rcbxD32p7lMtB5cXl5Sp22LVLhK1cX9du95vpNTawtzlwnoY7qPoof585jgAUmd0rscObyCMkx75CoCUvvyWk0bihtfXoLY5EFWjaNR0VD9y/kTc+MYa9fk9H27Cgu/3qc+tFkIoLLC7WroL/3prpTpHIyhfcdfvrYUvKNU0fJp0H1Pv/QwTBuTi2ctKWzyfNbtrkOWyYWihPqWJEEK3CFR7KK17YY4UrIfhQNHDaedHfP2bYzDz/i9w/hEDkeuR1uyeMbxXVKBQ1r946LwJOGNSf12gMPZfFOe5sbOqSTcct8kf1O1zymPfqI99cm6qZn8I5fU+fLrxoG7fWBf+M/7xPwDA7XNH6bYHwwJ2k3XG28JCSo2i4wKFLxhCeZ0PAwyz6hnrSrjpiakG5HtQdt88TB6Uh6GFmVj1u+Nx8dSB6GNoejpY58WsEYU46/DiFo/ZR04Z8m1ZZGTUR+sPqI9DhrtzpUaxdk+N6fGaWljz+96Fm3XPjQkJm/0hbDlYH7/QMSjhJtEKRSgsUHLrh3hk0ZaY+/z6zbWYef8Xao2Ksa6IAwWLqSDTCSLS9VcAQGWDH9luu/r885tm4f6zx5seQ+n41nZS3/bOOvXxj5oObSASKNbvlSb4GWsD9V59baQlPkOg+MP7G3Diw1+hqsEX4x2x+eU0JonWKJTMvY/ESXOi1Ji6e4bdUDj2iDXW/XGgYC2yWS1Y+/sT8fPZQ9VtygQ+ABhSmIlxxTmm783PjO741tpe0WC6fdchqU+DiHQXoCtfih4iW2YINlrGC/CmA1IH9w8J1irqvAHc9s46NPiC8MnLyiZ6PWxuofYDQF0sqjsHikZfEENvX4gnFm9PdVFYknCgYAnJ8dhx3OjIZDdtjQIAcgzPFQWGobTvXX+U7vnOKvOL/G45UPiDYd263xv21SEUFuoFts4bwOwHF8cst7FJZ5A8j2TT/uhAsWFfLTYYUpX885syzF+xCy8tKVMXgUq4RpFAoFCasfzdeHVBZXDD019yoEhXHChYwg7rF6k15BuGyCqd30alJfm658ZO2x8rzeck7NZ0fhuXbd1W3oBhd3yEN1fuVtfViOXd1Xvx3pq96nNlYuEPB6KHzs579BvMe/Qb3TZlpFOjL6g2Yxkz6cZitl55LIFgdPC59PnluPT55Qkfoy3+8tEm3P/x5pZ3jEMpeV0rmwVZxwmEwknt5+rygYKIhhDR80T0VqrL0tO57FZkysn/Tp3QT/ea25C5dkxRNr64eTZmjSjUbc9y6QfazV+xK+pzlmyvxJ7qJrVvZLUhz9SnG6XO8LsXbNTNxTDzyKKtuOH1NerzRvkuv9GX2B+VkvnWFwyrNYpEagpA9OiuePyh6GN+vbUSX29t3Voen28+iJJbP8SB2tjrhGg9/eWOdjcZJRo4O5o3EMKv31gTd02U7s4fDKu163h+/94GXPfKqqSVI6mBgoheIKJyIlpv2D6HiH4gom1EdGu8YwghdgghrkxmOVni3vrZdLx8xRT0zdGPhCIiXD1zMH513HAA0vyJwb2i04XYE1ib4qJnl8MbCGO2PCHw+n99p3v9wf9Ko4jqfa2/g22S3xOvqafOG5mlrrQy+YIhtUZhHHnlDYRw1hP/w/eGkVqtqVEox/YGQrh34SbUGmbKh8ICd3+wscWLxitLdwJAVBNae320br9pNmKlbB1hf22zbo33lny2qRzvrN6LPyzY0CGfnyrbyut1gzqa/SHsknOu3f7uOsy8/wvd3CUz+2qadbXwjpbsGsWLAOZoNxCRFcA/AJwMYAyAC4loDBGNI6IPDP9azgDHOtWovtk42lBLUNwxbwyOlZP2uTW5oz658ehWf06ex45fHTtct02ZSd5Wa3bX4LPN5QCi74JP1czlUP5IgUj7e5M/pFbtjReztbtr8N2uGvxxgX7JV23No7LBp47kMuMPhrHlYD0Ov/tTPPPVDvzt0x90r63fW4vnv/kR/6eZs2ImmISEiOGwwM9e+w7nPLnU9PWOyA4shMD0v3yO61/7ruWdZR6n9DvW2pFwXc3xD32FYzT9bNf/6zsc/cAX+GpLBd5atQcA0NhC7TQY7sZNT0KIrwAcMmyeAmCbXFPwA3gdwOlCiHVCiFMM/8qTWT7W8cb1z8GVMwbjb+dG8kmN7JuFj26YiecSmGE9qq+0bnffHDd6Z7vUFOoA0CvTGettCVEm5QH6QOENhLBOcxHfpblrV2ot1Y1+tVPdWKMIydUOq2EioLZGccJDX+omFhr5g2H83xtr1GMf1DSnlNd71X4A43Bfo6B80Q52YGr3gLxErrIQlZG2RqGdtV7T5Me2cvNRbVGfIZdXCeSJ8NjTI1AYfS5/B3/7NDL/xutv+eeuHfTR0VLRR9EfwG7N8z3yNlNEVEBETwGYRES3xdnvGiJaSUQrKyoqOq60rFWsFsKdp4yJ6rQeXZSN48f0MX3P8z+JBBClU1zpdHZqmqp6aWaIj+ijT9XRWtrO4ypDEsN9Nc0QQuDBT35QJ+dVNfrVOzZjk5Ky1LjFEslZBegDipJ08S8LN+FPhpoHIDWFZTgi/Tfa9TwOajrsBWIHgEUbD2Lpjqqo97dXS0N3tUG3RpMp+OLnluP4h75MaH5FW0Z9KQG63tv+NPad5X/bKvG2XEuIRakMbjkQGZnXco1CqFkNkqHLp/AQQlQBuC6B/Z4B8AwAlJaW8syfLuztn03H6l01OKxfDsJC4KhhvdTXlGG2yop5Ds3qeNqhtpnO1v3qGi9W2guTcfLdJxsOYGTfLDz+xTZ126FGP/LkkV4Har2o8waQ7ZLKqlwol+04hEl3f4r/XH8UJg7INW1vf1pOUf77U8fotgdCYd2Q429/rFYfn/3kEjVNfDjO9fSql1eqj439Nz97dRVG9c3Gw4u24OYTR+AXhma9eFpqWtLWKCrqfeokyw37pJFl5fW+qNn9Rm25yCm1po4Misl28XPSKLazJ0dnNWjwBZHptMFqIYRDQndD0lJGgmAoDG8S5+KkokaxF8AAzfNieRvrISYPysdVM4dg+tACXZAAgEynEiikGoU2UGgvpJku8+G4sRibbIKaK66xRvFtWTUufX6Fblu9N6jWKIJhgSXbKrGzqhFN/mDUH/HS7VUIhwXe/i7xX2t/MKybi2K8w35HPpb2kh1v/Y4Gb1BuLpPKtmxHFVaUSbWNp75s3Xoa2hqDMrlRCIFnv9qBQ41+3XdZoVm3pEge8BBrUqVWS01qZpTP7epNTw9/ugW/+8+6FvdTRm9ZTHKZtTSCLhgW8AfDSZsdn4pA8S2A4UQ0mIgcAC4A8H4KysG6II/c5KQ0wzg0TU9OmwXHj+6De88cpw6zLcpx4aoZg1s8rvFirmt6avAbd49S7w2gORDG9CEFsFkIT3+1A7MeWIyT//51VLPAur01+O/GA1irWRXQKBwWujtxXzAcNXTYjPZCUHrPIlzwzDKc8tjXeGvVHt37G31BTLr7U5zz1BKEwwK1zQH1gmp2MRl2+8KYn6ltelImN36/pxZ/XrgJN/97ra4/pKIh0rcyUG5+3F4RGdHjDYRMhxe3ZWa6X/4ZtnS33VGkm4LWB6W/f7YVry6LHgZupDQx2kwGIhjP8a731uOTDZGcacrPoC0BNxHJHh47H8BSACOJaA8RXSmECAL4BYBPAGwC8KYQonuPb2MdRg0UatNTZPSUzWLBcz8pxUVTByJLfj3bZcftc0dj7V0nxj2udr0NAKho8KlDTQ81tpz3KSyk/TKcNmS77ercjp1VTeqQW8WWgw0tpvmobQ7oLjr+YFg34zs7gaBR1ejH8h8PYf3eOtz877W6BaWU5pj1e+tQ5w0gLCIrDZoVLRgWqG70o+TWD/HmSqkLceO+OngDIdN5EkpZKxt86kgrQF+jUMqjTbEy8/4vMP6Pn0QdL94F7vHPt+KzTQejtgfjtcPF0eBr2wV/0p8+xfS/fN6mz4wlqPluK+QmULPVKo3lfWnpTlz7yip1GLXyXfiS1KGd7FFPFwohioQQdiFEsRDieXn7QiHECCHEUCHEnzvq84joVCJ6pra2Y8eQs86jDKtVFllyapqetAkClbvnDKcVFgshx23H8aPNO8sBYH+NPlAcavRj5v3ScqnaGsXFUwfGPMbuQ80YmO+Juogbmz6a/aEWM8xWNfp0k/78obB6V12Q4dA1uWnFC0AeTWd4pabfRRnRpQYKoRxLf7BJd38KAHht+S7UewOY++jXuPH1NaZ9FEqq90BIxAwUynbtMroV9T7T48Ub2vngf7fgypdWRm1v68iucX/4BGc9scT0tbW7a0yDiBACvmA44WV9E6XtV1CGYrdUo9COLFNqcMp37U3SENkuPzO7NYQQC4QQ1+TkmCeoY12f8kei/K1oL5h2zeM8+W5Ve5F69rLJGFOUbXrcffJksVtOGokzJ0UG2X2y4QD+o0nxMXdcEe6YO1r33n6ayYUzR/SKynNlXEbWGwi1OEqlssGvS00SCEqBItNpw+JbZqtNbof1059PvFFPVQ0+TByQi+I8t25m9ka5U7mmSalRSMeI1UntsJL62pdbKkxrFErfRyAU1t8VawOFvF1pe483mS6Rpqd6bwBLtkdmqmvLVXLrh7rgGMv6vbUQAth8IDrXV3WjH6f/43+46c21Ua8dbCFVTCLMmvy0zXBKoDCbA9OkuakIaGpSSvOb2vTUHWsUjLWW8rdEZoHCEnk8Q+4E/35PpPZIRDEnmu2Q28mnDSnQjZi69pVVuouA1UIoytWP0OmXG0mzPnVwvjraSWkm017wMxxWNGva4R85f6JpeS54Zhke+EQzqS4Uhi8URu9sJ7JcdtjkQHHCmD44XpOMUQjpjtLsolNe78OYftnIz3Bg5c7IqKk9MWbsxhqS6rBZ1It8cyBkup8yImdbeQMWaZqFGv0hhMNCV3NQahRlMRJAAom1rd/w+hpc9OxytXM8aKi2aSc0bj5Qh/s/3hz1PWl/X4wrFSrNdetMJkZuMskN1lraWoFSLm3wjNRMpd9hl91i+l5tgFeanJT/uUbB0tqb107Hc5eVqvfLJP+xOGI0PY3rL9UaT5+ozzkVa0LyP//3IwqznBjbPztuGpFQWOja+gGgSBMoPA6bGsSUIZ+HNKOmemU50aypUQyPM9/j662R+T5lVU3wB8NqTUL5DLvVAremSWlreQNG3fmxOszSKNNpw3Gj9E1wyjK0CuXaGYhxcXZYLbrgYLafdjjr/BW71c9u9ofw6OdbccSfF2GfPEFPqVFoZ7y/srQMX22JnH8is4qViWjH/e1LbD1Yr6vJAPr0MNe/9h2eWLwd+2u9qG0O4LZ3vseCtft0qTC0TWJApIZi9vuhlN2Y06w1NmuCjTKQQTsEVimb8t2OL85VX2sKRMqt/XlsOdiAcFhwjYL1DFMG5+P4MX3Ui5hywddOuNM2PRERNt89Bw+dN1F3nP87YYT6OMdtV2eDhwUw57C+cNqspkujKn0eQwoz1PkSin6GGobSkassEautUfTKdEIIoLYpAAtBrX1oPXrhJAwtzNDdGf5r+S58sblc7ZNRhkjaraTOQFb4Q2Es2V4VdVxAulj/6rhhum3G/FDKp8ZK5uewWXRlMzZRCSFM81hluWxo9AfxsbyC4U75cxv9IbkDOfKeO9/bgMteWIHL/7kCK3481OoL3I7KRvgN5dLWMJTh0zurmvDa8p2Yv2I3Xl22Uze/5PPN5Xjss63qRVup1SjNn7VNAbz5rRQElaCXGWOQwaKNB7Fg7T51pFttUyCqP+NsTQoU5TvVNT35gxBCoEn+bk8c0web756DXI9d3/Sk+bnd/O+1eOzzbeq2ZNUouvyEO9azqG3wFF2jcBju9Fwmd3ezR/bG4xdNwi/+tRozh/fC0N6RO3olBYjZHeMp44tw75njQERqbUbRL0eqUShNVsqFRUmMWKm5M+0lL9RU2eCHx2EzLWOm06rreFYEw0I9X6VGYbNYdHmzWpLhtEWtKR6VLE5NdGh+cbZZLLqla40BxWdYI0SR5bLpMv1qh//uPtRkGly++KECmw/U49aTI2udCyFw4bPLcPbhxTi3dIBu/5/NHoonF2/H3R9sxORBebrXtKPPinPdWLu7Bhc+uwxHDi2QjwtdjeL2d6W5DceM6o1QWKg1L5vVgj3VTbjrvQ34bHM5Jg3MVdOXGGsxgHSx1052LLtvHm54YzVsFgueuPjwqP0BKdi7YY1qevKHwgiFBW45aSSumimlr8lw2NQge6DWGzWC73/bKtXvujVJFVsjrQIFEZ0K4NRhw4a1uC/rmi6bXoJ1e2rx0yNLAABO7fBYk5qAGW2ztHaEUl6GdJdpDBTHjuqN648Zpl5gjWtrDMh348oZg3G2vEa4cv1Tah77NB3HhXIt41CjD26HVU1TruVx2DSjuyy6i64aKOTndiupfSGJyHRG72vsKI50ZpsHig/X7ceH6/arzy9/Ub+qYLPffC5ElkntaXjvTGwtb8B3u6pj1hpy3Hbda43+EJbtOIRlOw7hHMMMZmV2/p7q5qi+l0ZNmZSfNQC19tXoD6LBG4TNQrraR6MviPOfWaY+33qwHjP++oX6PBgW2C//jM063c1GSW0rb5Ca4mJcuAOaPiBtOZTvVdvEleO2o7rJj/s/3myaEt4XCiMQTm7TU1oFCiHEAgALSktLr051WVjb5Gc48PxPj1Cf6/soWt9Sqm1GypUf222RgDO6KBsvaD4PiK6pZLnsuPOUSMoNpekqw+SirNRaqhr9yHBYdcN7B+Z7sOtQE1x2q3rxz/M41IsQEKk1KU1PNqtFnVOi3SdWR7Qysz0Rbc362hwImTZxmE0YHNY7E4ca/VhVVq2r3Wllu+26PgptU5lxmLGx/0hLe8E2Wwiq2R9Cgz+IfrluXeJH46g1Yyd5kz+kNj35Q2Fc8txyEAFDCzPxkyNLotKsh8MCB+u8CGc6Y65dojYVyYEi12NHoy+k1hy0Nwd9c1z4sbJR7aMx8gVCkRoFd2aznihe01NLiEg3eSlPrikox8n12PHqlVNaPI4xr9R9Z43HT6YPwvQhvaL2VQNFgx9uh01XZmUpWZfdol4Icg39IdFNT6TeXSoXYmeMORZAJHgp52qWcVepcbV1ne4mfwjeBGsUdqsFY/pl4z9r9uIhTTZU3fucNl0zmJJxNttli5pUZ+w/0rrvo80oufVDCCFMA6lSo8gz1Bh3VMZPMVLXHFCz+QZCAt9skxaUenFJGY55cHFU6veqRj8CIYGa5kDMGsUFzyzDXxZuUl/vlenU9eO4TQJFLL5gONLPwp3ZrCfSBorCrMTSjE8ckAsAaiI9hXKRUWomx4/ug4IEUpcbA8WAfA/+ePpYdT0ELaWPoqrRhwyHVddfcMfc0Zh/9TSM6psNt90ml0l/0VJmomtHfSnNV+P65+DxiyZh/jXTWizr+7+YgUcvnIRsd/RdfkjOEaVdIlYxO4E1P5r9IXWi2O81NS2zGoXNSijMdCIsYi9wJK31EbnAvSF3IOdlOKLeY6ztaUe5KRfZBp+UlytL83PzOKxo8oXQ6AtG1dC2l8e+CANSrqqwiKQkMTIOp1XmsDT5Q7jo2WVmb8HOqiY8/dUOdbJnv1w3GjRNT9o+rKIWEipq+124RsF6JO0duVnHsJkB+R6U3TcPx4zUr3ul9D0ocy3MZsCaiZWDSXtn/2t5tJVSQ/AGwlGd0DarBdPljlVt09O2P5+s7mMcHmuzWNSLyVHDeuGU8f0wtn/sCaXKRXBAvgenTehnmmVXCOki9tw3P0a9Fq9pR9Hkly5oeR47TtCkjjf7nhxWCwoyzY/5zW+PQVGOC3XegBoohvXOxDfbpEl1uW57VDOQsc/HbFDA5gP1UpJFTRAuzHJKNQo5Q6tWSzUKpYYzqEAfKEoH5WFCcQ6MOfy0KwEqzYo/nz3U9Ngb99ehMMuJAXluNGpSixibnuLRjibjGgXrkcw6g9tKqVEol554q8C9cc009W7f7GIE6Dvaf3XccJTdN0/3Bx4vxYSyX45HmlynrL/hsBn7KAgXThmInx5ZgiuOajn5ofEiqCRXNHaIazuCn9UsKJVIx3lTIITmQAhuu76z3mwosM1KMWttxXkeTB2cjzpvAP5gGE6bRZdKPtttR8jwHRqb3WxWwoY/nqR737lPLcWqndW6QQlKraaq0R/1HW1vYXElZU2SkgL90r52qwV5GY6otCoHTNbwjhWAN+2vx+iibHgcVjT6g+rQWGPTUzzaLABco0gA53pKP63tlzBz9+mHoU+2U70IKjNy4wWKqUMKsPz247HstuNi5l0y6yvQjla5+ujYF3a3Q9+X4HHq+x+USeh2K6Ewy4k/nHaY7uJhNhcEiA4UygWqt6HZbtP+yOQv7ciwWEFRq94bhDcQgstu1dXylMf5GQ5MkJv/bBb9xT+yr3SC2W67msLdYbPoBiwIgajmMWOt0maxqIkataqbArqFoJRmy4p6X9RciLoYacpfvkLqv9p60LxGYbOS7jMUe01mwscKFNvLGzCqbxZcdit8wbCm6Slyntp+mf6ayZ8KbaDiGkUCONdT+umIQHHp9BIsv/14tb9Aac4wy/uv+2ybJe7dnFmg0F7Ijh0VO0mhtulJ+zwyPFZpHjM/f+MFUzkVY/v7n04/DD+bPTRqPoI2lYX2whlv5vERJdK8hZ2VjZFAoalVKWkpTpvQD4Pli6rDZt70pASkLJcNdc0BeANSjUI7BPrHykb8wbAaoPE7VwKmWU3IqTkXbaDUfkfxUruPL86B3Uqo9wWR67FHdaTbrRbTz9WmVVeM6JNl+hn+UBh5HgecNos0UVOepOexR8qlXadEO+zXDNcoWI9ksRCK89y4y7AiXHsoNYpE+yhi0TY9qdvkO+WWUj0oaTly1UChX39Dm8LDzEmH9QUAPHbhJNx28ig4bRY45H9aBZlO/HbOKPVuWOlsf+Nb8/UR4jU9vXntdPTPdWNbRQMqGvzIctl0NRtlxrrVQmrtx2Yh04CsfD/ZLjvCApi/YhfcDqsuMJo14UQFSPl/szt77U2GdvRXtsuOh86bgBnDepk2lymctsjEyJF9sqK+W7uVogIzIPV5GGtwvbNjD5rwOCI1s0cWSSPDtLVHbV9Lrjt+HxLXKFiP9c1vj8XlCbTPJyokWm56SoRZ/0lBhhNnTeqPN66NjEx65tLJeO2qqbr9lLQcStOTcqccGR4bSeFh5t4zx2HxzbNx6oR+uHbWULjs1rjLw1rl443pJ9W2dZPT5GB1REme2gRmhogwtHcmvttVjXV7ajB1cL5uVFdQE4CVC5/NasGkAXlRxxpfLJVj7rgidZvHrg88ZqOkjDWKZs08hKh9NT8f7Yi5HLcdZx1ejFevmmo6F0bhsEVqDKOLsqMChTTHJfr9Oyoa5abL49Rt8W4c3Jr5NkqCSm3AztQEwRyT89RKJGdWW3CgYD2OcgEyWyCmNcyaxawWwkPnT9QldDvxsL5RS74qFxilRlElL56kBA6lZMZ0HOpn2ywo6RXpXHXZrHEvepVy4sJ+hqa01XeegH65bqz63fF45cqpUXmljAFuVN8s7D7UjLAAZhmG0o7uK6VEnzAgV/1uHFZCjseuW1jqn5cfgQfPnQBAGp01fYg8EsxpVbPmxuK0WXHdrMgIIqV/QekwP790AIrz3PK+5oFCO2Q4Vp+MzSJlIlZGLY3pFx0oHCaTIRV9spxq0sjSQXlxR+xlOGxRtVNtYNH+nua6owNFhiaocI2CsQ4yTb4wzR7R8pyBeNoTaGYOL8QtJ43EBPnOevchqQN0tjykVzl0omsgu+yWuLOylWy7p03oh39efoR6EVXW9SjIdOpmjCuUO3+FksYEAA4fqK8pHDOqNxbfPBtzxxVFpVvRzmk4ZmRv3QVWeexxWGFv4Tu1Wwm3njwK86+WamxK0C+Um9ScdovazKQNFH00cxG0bf6xgqvy3nnjizC0MAPzxhXpElQCUjAxa/ICIpMPv73jeLx61dS4WQU8JqleYv1umdWctBMdOSkgYx1k8qA87Lh3brtrFIp4K+vFkuG04fpjIjnJbj15FD7bdBAD5EldSk0i0SQbUtNT7LvWiQNyseWek9W74g1/PMk0hYdx7ofxAjeybxbuPXMcxhfnmNZ2lFqO0tegZHhVvmuzzmPlYu2221rM56V8prHNv5dcY/AFwppAETkX7Zoi2n6JopzoUURApAnwsQsmgUj6XLOmJ21gPXZUbzXNhjKqK5FJom6HNeG1rs1mv3s6oUbBgYL1SB0VJNb/8aS4KTUSdd2sobomFaV0xsV1YhmQ7zFtltDSXuhsVgtM+uKjmmLsVgs+ufFoXW6ki+IsF6seX/5+tdlW/33ddNPhncpnZiTQ9KQwdhYrAag5EFLTwmvPVzsDXlujGBYj/5QSZLS/J8ag6bCSrl/o1yeM0ASKVmT8ddgSzvpq1typHd3FNYoEcPZY1tnidSC3x8ACD1burI7ZBm70xMWHoyNCn7HpyWohjOybhZF9zYd3xqKsHaKdWX1ESb7pvkobu8dhTXg4tPF7V1KiNPlDGC0vh6vNjqut/WgDxYA887QcZnNnjIMfbFaLen5HDi3QfXdmndc3HDccWS4b7vlwk77sDiucvsQCi1m5tDcqXKNIAGePZeninjPG4rhRfeKm69BqS2ZdM61JaR6PUqOIlcpcSwmGLrtVHZ3VEmOzl1JTGdzLo9Y2qhr8+OymWbr1QgB989eoIikAZrtsuol3ZrVE47nYrIRpQwowbUg+/nLWOF1Tl9mIuP87YQRqmvy458NNugzAGU4r6r3xf36PXjgJWw7UmwYKpZnLabPwPArGehKPw4Z544ta3jEJn9sRIk1PLTedaYNTok1PgLTY1FUzpGHT44pz8K+rp+KWk0apE/yqGn0YWpiJqfLgBbPPGFqYiY9vnIlfHjtct0+soc9aDqsFhVlOvH7NdAwqyND178RqelKCyQVTIhMgPXb9qCelo17rtAn9cPNJI+NO8rQQmS4o1RHSqkbBGGuf1qymp8hy2nRLjALapqeWL1zaC3eseSOvXjk1akW7xy/Srx535FBpCHKx3JxknEn95MWH47td1VHHHtU3G19srtBtM2sCG1jgwVe3HIMnv9yO+St2Rc2a9yQQKNwOK7678wTkuO14eelOdZs2MCmJI82YBQpd0xP3UTDGkq0tTU9Lbjs2quagtP0PMiTSM6OEBiFipyxxO6wJB7FhvTPx1CWHY/pQ/dyVk8cV4eRx5rU04wXWbNY9IAULpatCuwAWoG/+izfBzpj3yWGzJNz5bd70JL1XQCStRsFNT4wxld1qaXV+rSyXXZ2PoTh6RCH+dfVUXC2v+xwPaeaMxBoe29p0K3PGFuk6rVtivMDGSgSpZY8R1IBIv0GiEh0557BGB5RzJ0vNWNOHFMCXpDWzOVAwxnTa0vxk5sihvRJKk6LdI1bTU3vTrbTk3NJi5LjtmDtOyqGVyIU73pyPRGoIJx0WmX/T2hqFduLdjOG9UHbfPIzsm60uKNXRuOmJMaaT67FjfHEObjt5dKd83uBCaS7DqKLsmGtMtzQRr72GFmZi7V0n4q8fbwZgPgM6qkxxgldLSSEB4KlLJquPE61RKAsbjeqbFfXzcdos8AfDEELETP3SVhwoGGM6T1x8OPIzHDFnLXe0WSMK8cEvZ+Cwftl4dbl5Vtv2ZvpNVEC+I0+kbyVqaTuNRBbc0l7MEz2/wwflYdqQfNxzxlgM662f21KQ6UD/XDf8oXDMPpa2SqtAwRPuGGu/w/p1/nouynwRJdeTy26BLxhWF+WxxukP6Ej75dTmZjPIo8TJw9WamdlAJGgMyI//udkuO16/Zrrpa5dNL8Fl00ta9bmJSqs+Cl64iLHuTRkqO2NYL3x58zGR7Z1Uo9hXIyVnLGph+dGWJNL0ZPTBL2fgvetntOtzkyWtAgVjrHtTOrOJCAM1S48muzNbMVauTQ0ubF/TU1tmyo/tnxNzydRUS6umJ8ZY9xZrHkVn1Sh+d8po/OTIEvTOSqBGkWAK+HTANQrGWJcR60a8s2oUTps1ZkbZREwfEntWdXfGNQrGWJdjDAuxahopZdL09NIVU5KWRiOVOFAwxro8a5LnUXQUh82S0Kzu7ib9zogxlnY6q4+CmeMaBWOsy+usPopEXDdrKNbvrcUpMRIMpiOuUTDGuhyl+X+IvAZ3ogsadYYB+R6894sZUYkQ0xnXKBhjXYZxxOnr107Dhn11HbbGOWubtAoUnMKDse5NaWJSchX1znKh98j2zZJm7ZdWTU+cwoOx7u3YUb3xs9lD8YfTDkt1UZhGWtUoGGPdm81qwW/njEp1MZhBWtUoGGOMdTwOFIwxxuLiQMEYYywuDhSMMcbi4kDBGGMsLg4UjDHG4uJAwRhjLC4OFIwxxuIikYbL+RFRBYCdbXx7LwCVHVic7oDPuWfgc+4Z2nrOg4QQhWYvpGWgaA8iWimEKE11OToTn3PPwOfcMyTjnLnpiTHGWFwcKBhjjMXFgSLaM6kuQArwOfcMfM49Q4efM/dRMMYYi4trFIwxxuLiQMEYYywuDhQyIppDRD8Q0TYiujXV5elIRPQCEZUT0XrNtnwi+pSItsr/58nbiYgelb+H74no8NSVvG2IaAARfUFEG4loAxHdIG9P53N2EdEKIlorn/Mf5e2DiWi5fG5vEJFD3u6Un2+TXy9J6Qm0AxFZiWg1EX0gP0/rcyaiMiJaR0RriGilvC2pv9scKCD9ogH4B4CTAYwBcCERjUltqTrUiwDmGLbdCuAzIcRwAJ/JzwHpOxgu/7sGwJOdVMaOFARwkxBiDIBpAK6Xf57pfM4+AMcKISYAmAhgDhFNA/BXAA8LIYYBqAZwpbz/lQCq5e0Py/t1VzcA2KR53hPO+RghxETNfInk/m4LIXr8PwDTAXyieX4bgNtSXa4OPscSAOs1z38AUCQ/LgLwg/z4aQAXmu3XXf8BeA/ACT3lnAF4AHwHYCqkGbo2ebv6ew7gEwDT5cc2eT9KddnbcK7F8oXxWAAfAKAecM5lAHoZtiX1d5trFJL+AHZrnu+Rt6WzPkKI/fLjAwD6yI/T6ruQmxcmAViOND9nuQlmDYByAJ8C2A6gRggRlHfRnpd6zvLrtQAKOrXAHeMRAL8BEJafFyD9z1kA+C8RrSKia+RtSf3dtrW1pCx9CCEEEaXdOGkiygTwNoAbhRB1RKS+lo7nLIQIAZhIRLkA3gUwKrUlSi4iOgVAuRBiFRHNTnFxOtMMIcReIuoN4FMi2qx9MRm/21yjkOwFMEDzvFjels4OElERAMj/l8vb0+K7ICI7pCDxmhDiHXlzWp+zQghRA+ALSM0uuUSk3BBqz0s9Z/n1HABVnVvSdjsKwGlEVAbgdUjNT39Hep8zhBB75f/LId0QTEGSf7c5UEi+BTBcHi3hAHABgPdTXKZkex/AT+THP4HUjq9sv0weLTENQK2mStstkFR1eB7AJiHEQ5qX0vmcC+WaBIjIDalPZhOkgHGOvJvxnJXv4hwAnwu5Ebu7EELcJoQoFkKUQPqb/VwIcTHS+JyJKIOIspTHAE4EsB7J/t1OdcdMV/kHYC6ALZDade9IdXk6+NzmA9gPIACpjfJKSG2znwHYCmARgHx5X4I0Amw7gHUASlNd/jac7wxI7bjfA1gj/5ub5uc8HsBq+ZzXA/i9vH0IgBUAtgH4NwCnvN0lP98mvz4k1efQzvOfDeCDdD9n+dzWyv82KNeqZP9ucwoPxhhjcXHTE2OMsbg4UDDGGIuLAwVjjLG4OFAwxhiLiwMFY4yxuDhQMNaFENFsJQsqY10FBwrGGGNxcaBgrA2I6BJ5/Yc1RPS0nJCvgYgelteD+IyICuV9JxLRMnk9gHc1awUMI6JF8hoS3xHRUPnwmUT0FhFtJqLXSJukirEU4EDBWCsR0WgA5wM4SggxEUAIwMUAMgCsFEIcBuBLAHfJb3kZwG+FEOMhzY5Vtr8G4B9CWkPiSEiz5wEp2+2NkNZGGQIppxFjKcPZYxlrveMATAbwrXyz74aUhC0M4A15n1cBvENEOQByhRBfyttfAvBvOV9PfyHEuwAghPACgHy8FUKIPfLzNZDWEvkm6WfFWAwcKBhrPQLwkhDiNt1GojsN+7U1P45P8zgE/jtlKcZNT4y13mcAzpHXA1DWKx4E6e9JyVp6EYBvhBC1AKqJaKa8/VIAXwoh6gHsIaIz5GM4icjTmSfBWKL4ToWxVhJCbCSi30FaZcwCKSvv9QAaAUyRXyuH1I8BSGmfn5IDwQ4Al8vbLwXwNBH9ST7GuZ14GowljLPHMtZBiKhBCJGZ6nIw1tG46YkxxlhcXKNgjDEWF9coGGOMxcWBgjHGWFwcKBhjjMXFgYIxxlhcHCgYY4zF9f/PV9h/OxugOwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "loss_ges = np.append(loss_ges, history.history['loss'])\n",
    "plt.semilogy(history.history['loss'])\n",
    "\n",
    "if (Training_Percentage > 0):\n",
    "    val_loss_ges = np.append(val_loss_ges, history.history['val_loss'])\n",
    "    plt.semilogy(history.history['val_loss'])\n",
    "\n",
    "plt.title('model loss')\n",
    "plt.ylabel('loss')\n",
    "plt.xlabel('epoch')\n",
    "plt.legend(['train','eval'], loc='upper left')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Check the model by hand\n",
    "\n",
    "* The following code uses the trained model to check the deviation for each picture.\n",
    "* x-axis walks through each pixel, y-axis shows the result"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Check each image for expected and deviation\n",
    "* setting the switch \"only_deviation = true\" will only print the images for which the classification and the CNN-result deviates\n",
    "\n",
    "The output contains the following information:\n",
    "\n",
    "| Filename      | Expected Category           | Predicted Category        |\n",
    "|------------- |:-----------------------------:|--------------|\n",
    "| ziffer_sortiert_resize_NaN/5\\Ziffer_4_0034.jpg | 4  | -1 |\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ziffer_sortiert_resize\\0_0_digit7_20210504-183320.jpg 0 8\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAABQAAAAgCAIAAACdAM/hAAAGf0lEQVR4nAXBWYxdZQEA4PPvZ7n7nTt3tk5noNRxWsQBU5owmtSmhhBMBExs0Sa1koIYSHxspJgYBEmML33wgUejT7UExPhEQI0GjS3FznRaCV2mM3P37Sz/f86/Hb8PvP/6j4WSXqFsCSJ+IQcO56nVJudKJsIx2mptjIIEIwQTLhAMIER5ngshcMIlINgAqEweT0Jp8zzPXcRGUeRBUinXZCa01uNwBBByIEl4YozNc+C6FEMWiCzrtDrAdaHrE8rSLO3zETAgFiJMlUjiSqXSTzKAEca4Uq9nWRbHfBxOcKoAK1aRAw3GvXEMiCSEYOZzmYhUoVxFYayotzuJIISE0a279xnzKKUAABwm0pFm84svLKXd8Qh5nrVOEguXUMdYl7JMJDth1OrsAQAggYHrqXBirWWM4ZgLv1RMhCSIRkJSRBGhluAwk3EYOVZTSrVWrFgGyOGc97o9raVMVaZS/I31ry8fPLDw0IN7g8Hmu3/cvXfPKxRc1x/2Bxihoh9QH21sbTRnZxDBYRguLy83m43Z5hx1CXj1xHHien/560fluelOFFeaDeXAdrvtGLu+vr7+xBNBELz55i+lzhAl1slFnFJKGWOEEPDa008N4vDqzS3NkABAY9AbR42Z5ssvviyE6LT2lpaWjFVXrl798OMPMWWU+JyLRqPRaDRwKfAjHnkUTmSmQa4tfvjwyrET3xoOu7VabX5h1g9onpOVLx1Yfmjp7V/9hjKPEAohrNfruNdpd1p7RiqPsWqj3udJEPiEIMzIja2NP//pA63khQsXpho1IcRja4f7w2gyjm9t3dy+ew+WSiXmuQih/mgY8ySVmRf4QbHAvODy5ctSSs/zbty4EYZhkiRLS0vtdlsZvbi4uPbYo7A/HGXKTs/MPfLVR+cW93MpJ4lwg0KxXKF+0ViojXN/p7Uwv79UrPp+UKlVhUxjHu+0dsEvnn323Cs/aT5+JBv1nEKAS8H9bv+zjevQgZ12e9jrBcwdj4cLCwu7u/cncXR7e7fVac/Pzx86dAi8+s3jt27fvnL3TuY4IKCw4B08/PC3n/lOuVh6+623lMgOf3l1+96dU6dOTk9PX9/cuPTeu8VSKUmSyWSCCYKzM81FHgpCWjxyME61stZaa7XMKIZX/vOvWq22sLhv8383E5XZPD9//vzS0tK1a9dwwJwwUXE0lr4LGUpzbYzKrWKO1tHQ87ypIlM6GcfD0lQlUplS6qVzLzmZ8atFOGztZlE03ahDCK211WpVG2WMGvY71WLRSlEO3HA87nRapWrF9T0IUa1anVmc++GZM3C+OT9dq29vdx1tGKTt3RajrhBZDmChXCoUSrdu7TBK983tu7Vxc64xQzHBDpZcvvPbd/Cg04+tOfKVQ5vt1iiVy/sfkFJXq1UPoViIernKvD5woNVmujoVDifVcs0xlnN+9uxZuDi/mIyjz7c+f/21n3/w3vvHjh0/eHAFYaKMfe57J+M0nZqZeeGFc45xSoXyvTt3jVRaawDAxYsXwc8ePxpr0zNmL+VDCLyZRgby5777TMGl2Mk15wTAOOKdXn9qevYf//xkOOwnqUgTfvL7z4OfPrLmV6sfX//vxIHd3AxUBgN3YWnhR2dOi2gik5hB7FEPI3bp0uXdVgdQOBwPRJIUSiXwxpNPdsfjT7ZujhygAo8T2O11px/cF05G60ePlHxv5YEDv//dHzKe8SQrVso5sJnR9VrtB6dPg1eOfi2oVP726Wd9qYzvdXjsT9W5SorFAgKGQgBsnsWpiAUCxPU8B8E448P2wHEcXJoqEc8tFHyZZdb1uVHY6ESklWbdybVMszSOoIH1SpkCom0uVHrm1Km1tbXBYADeeP4pZfN/f7oZZgp5RWGtVynGaYIY7A87BOGZqbqR+c72LjROozlrcmtym+d5mqZYOyqV0iNAayCFsEIIniQiDkpug7GVlYOHV1f73cHfx9GgO2BKAYwGo/Hq6uqJEyfAr198GkGKLCkENQJcbR3CKM+4clIWuNrIVqcz6o8IYc3GbL1Sz5QZjUYYY4QQ9hhKwsQoNBpzMZFGGUqpttpADV1gQZ7KzM0dAlC4uze8s4MxjuNY8EwbiTEEaTKBKXAymE1kQAtFigABmTWD3iCzslgpU+omsVAqZ8RNIx4A6GHqYIKtlkZJn5Y8yqTRBGCb8FjEDjFlj2nIOE+ySEBMGUCa83TCC36gsswY838FtM5ScQHWmgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<PIL.JpegImagePlugin.JpegImageFile image mode=RGB size=20x32 at 0x2B428728F70>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ziffer_sortiert_resize\\1_D6_2020-08-23_11-50-46.jpg 1 0\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAABQAAAAgCAIAAACdAM/hAAAFB0lEQVR4nCXUTXtUxxGG4beqq7vPnPmShCQIdoxNgOCFkys/nD/hVTbJLiGBGLAtQDiKkDQz56O7qyoL9s/m3jz0ZnAj1IapVUCikBDQEAkBYIITKkMFEBDD59ZnmUpJKUlzJyYWBA9NURrMQQ4HBCCCESygOcxBjDrOrmZmUSAs1NSbUSOvcFM05wiyLzUBDAMaXKu5e98toZCAniEO3B2Gu91eCSF2gZgRlGJ0UjcCKMDM1by6ubvPfnt1zSzxu3uiwP+ur99fXkqM2+PTRe6EI6BEgdTY4QJEUvfmZgat9ed3HwA+Ob4nXUQrbToMi+U6SVjkLoAyhUwRrZoZBVBgT0HJzcgp7HajqgIQIeSQlqk/2mxPj45zTKQmTh0HZ2htFMASObIFqONu1Jubm1rrMDTRGdQsS1wv+lXqGNCmAQgOK9VLRSBhDoEdUEJOwV1Vq1kTzKhDqUNZcBZFl7A+i3e/tX/+7e/zfv/2pzcAvnvyh+c/fP/1t6c3M3JE09HRxulOJEBIMqcskYH97XBzOV+8efvXH3883WzqYXLX68vLz+enDx+crhLiEs+/f6Kqp2dHwg53B5BiXGRwS8M4udbTk+NHDx4Mu+H6+srLjFKiwxhK+OPzx2a2XCVxgjmZmZoxI/ey6U+iP/7h2bNt7l+9/Nerl4WSrNfrRcJdwew4Pz9yhyoEAcakjOJ6mEFuixVvjo9OlzECMUvxGpTA5IzWwIzIIKAyeDZUdw0BEpRRYdVBMVzvpv3k1c2YmttY5mIAwxlXn3efdzMTuBJmaDHVQNyBk1CEReYUjV3ZOUvoBAENMEYpePXqP69f/zROxosFQgjNatOiDoMpUFrZbIMTH8axNY055ZyZoYpS/ddfPr6/+DiNjbnB57Lq4v3zLcEXC67VU4p3O6yWyNKX2RZpFUOqBcsF4FQbjZNKTOIKb9W0WjVQtSAMd+cQyAy1qjc3AxGxo1bkjJgXwpwzxACFq5vCAQc7uYdAwgAQQogxhhBEJAhmQzMEIiIngjgDTMzMDGIKFEQgTm2ER3Rdt1wuQyB3J4JV2x8m1eqO2iAGGNwAJ4BgMAM7YSplwanv+9VqFUDaCgxqzd022x6AO6Q1qFYzAwyAw9W0WRCRmLHebrbbtaqKSM6IiJtNevToG1XtOggAAMzMzMYOwN1VdbWQL2aOonBJEQxm6jO+fvhVqVMmCBFCiEYthCBMDAY1fPmmYRjHcS6tlaZqhLlU6mK/JCmdA0yG/d1uPBxas3GcxzK7EbPsB0wVb3/5+c3bt5LicrN1IC1iAN69+3h1dbWfjIcdylRbURj13WKTskiY51kEatjvD7thUKfqGAqGsV7dlE+fPo3jmDNzl9DnBSO0sZZZG2CGFDMH1Ibbu2F/mCnGIKkYPMjFxcXrf7/676ffrEEiIXNSKkkSAdNspH7Sh+urxsUPcwFLkDSrthnGUPXd7jCOMwBpI8rQaq1CWHYYGx+GqeTAJP0KT589f/jV7795/DQvYwOaOzOHEFLqYoRkRhfz59uryw+fT+4fc/DpMN1YyIgx4/GTp0RYn2BWDNX24/7X9x+GwzjP8zhCXv7jE4PWy83t9e1hvFsfres0t9mo8f62q8VSSrcfylCH2Kf9sBNJp8cnq0Vv1eTFixd//sufHj5+uF6v78bbUkqrVatt+6Nhmq2aM+32B0Tcv7dZb1cPzu//7vzs7Oxsu+H/A0G4QGBSss50AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<PIL.JpegImagePlugin.JpegImageFile image mode=RGB size=20x32 at 0x2B428E8EA60>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ziffer_sortiert_resize\\2_4_nt7_20210702-033547.jpg 2 4\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAABQAAAAgCAIAAACdAM/hAAAEjklEQVR4nEVVTY8cuQ0lKUqqr253T4/tBZxdw3CSm2/BHnbzL/IDcsp/TJB7soAvhk8GfJ7MrDG99kxNd5UkisxBHi9PpZIofjy+J/zH3/+GiDHGGKOqrusKADFGM0PEWmutlYhCCG2pgPZo7L0nImZGRET03uujMbNzzsxUtZQCAKrKIQIAIpoZ55yZWVVTSgBARGZWSnHOtSURtbsAoG0BQPtm51w719Lz3rf0RKQ+mpk559p/JGqRAYCXlJ3U5oyIqUgL7pzLkpuz9x4dI4CoSV7g0bjlE0IIIdRaU0rNk5nbLapqZrVWABCRFvOr85MnT7z3u91ut9vlnI/HYyml67pSSowREVNK67o2Z0Tc7XatYADglFKD6uLiopSyruu6rtM0mdlms+m67nQ6ff78+XQ6qSoiLsvSnM2Mu36cpslxuJ9Py7KkLFLtvCQRMSAkjt0wTjVlKaUw8+Vhv91uQwgpJRaRdV3v7++XZREREWl4ikjOWVWHYWiNKKUQ0c3NzcPDAzMvy8JiupYs8z0ifpuWJAUdkWeOwQWP7BShgn1DGAC4HW1tNDPvfRuYnPN2u91uty3s3d1dzhkRi9Y/v37d932DhokYkdpsOsdmKFJFdLN5cnFx6b3/9On29vY3kRpCAND379+HEJxztVZuGDJzA7bV3Pf9s2fPxnFcluV8PotIjLHve1V58fw757AN3FcHVSWi1qpxHA+HwzAMd3d3x+NxWZYQAjObGRG9evWKCPjRqJSqCohmZog2DN3z509zXq+urq6vrwGgDzGllM4LM//r3/9shHHO8Zf5dhzHLsbTMoPZy5cvX/7hB0T88OHDck6boSeiWtQ5B4Sn5eEvP/44TkPf94jIIXApqaW9maZpGkTKPM+3t7cppcAxhABG3nvnOITw8ePHbuhCCKpK3gXJtaTchXg4PJ2m7bqmq6v/gRGCa0A2hqrkKpmJTFVKkVK4TWzf9/v9frvdIuI8z8fjMfiuKUmbglqrMZdS3rx5Aw4doCJwSoXZ7/eHw+HpEHtJpcnYuiyqigBEVEshoopIBr/8578KBmroiEsW733fDUM/ElGtNcZ4eXlZkqjqV0krWmttYrQf97lmVEN2TEYmWnM1MfAUQx/2cTNuG/4O8HQ6QYWHhwdQU6l/ev3HLAkNOHgmopLr+XxOKXnvu64L3jfdLaWgGhF9+XKfUrKqqvLu3btSUmsTiygRXV//enPzCQ2aRDMzqCKiStNt8N63nv/1p59Vtdba6B2IwDkPoKCGiG2PidrYp5QAVFWbzr59+/Z3JSmpOEfARISIhEgOiIxUamOvc46Jx36YpimEAISNAmbGKSUiYv6dWC1CWnPTUGb2jrqu6/u+67pu6JukqioTsqmVXFUVqpJBVmt0NTMm19o+z/O6riGEh/Ni+CgmrQBVrRXMVREhIgDzjg0qk2PmVjMiOs8vvv+hFayqXKsBoCfnvQ8hOEY0UK2ImLOmkmqtziEAkAMAPp1OotpeEjYzZh66uNlsxnGMMSJorXVZzvM8n08iIrWamTlGA5jnudpXJfw//nZwxfZ8OjQAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<PIL.JpegImagePlugin.JpegImageFile image mode=RGB size=20x32 at 0x2B4274F2490>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ziffer_sortiert_resize\\3_Ziffer_3_0145.jpg 3 -1\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAABQAAAAgCAIAAACdAM/hAAAF6klEQVR4nE1VS28byRHu6df0PDjkUKQo2SIFCrJoQDJg6yLAgXMw4Owpyc0XX3wykD35x/iwSC5JDv4LEeAAsmwoSI7yWrItS5tVZFF8D4dDzkz39PTk0MgidWigG12or+qr+sp4/fp1lmVKKcMwEELF/wwAQAhRSsVxLKUkhOhrXighBMYYQojzPP/FLcsyKSUAACE0n88hhAghQghjDACgY1Sq/mKxgBASQnAcx5zzJEk452maSikppZZlOY4jhCiKwjCMPM+FEIZhEELGwSQMQ0KI7/vYdd1fnDV4bVdXV5RSDbUoCowxpbQoiv39/fF4zBi7desW1l8ppYwxwzAWi8V8Pg/DsNlsmqaplEqSBGNsWRbnPAiCR48ejUYjQkij0cCLxSLPc0opACAIgouLi8vLyyiKRqORYRhFUTiOs7W1df/+/dXV1UajMZkGtm1DCJVSmFBmA8g5n0wmFz/9fHL6OQxDxphX9qWUUkou5OmnL6NxcO/evU6nY9u2UgoAgDHGeZ7neb5YLL59+3Z2dtbr9er1eqfT2d3dZYwxxvr9/uHh4cnJSZ7nnuc1VupFUSilOOdYAyCEdLvdy8tLQsjy8vLe3h7G2DTN2Wzm+/7Dhw9PTk6yLAuCoOQ5b968GQwG29vb0DRNQki/31dK+b5frVY3NzfL5fL6+vpisWi1Wr7vb2xsaPLevXvHGIvjGGOcpimehDPHcWqNld+sNX/168VkMllbW0szOZwE1foyJHQ0GEIIozj5z3W3VHI8zwvDEGN8fX2NdQEIIbZte57n+75t20IIy7IQQuPx2Lbt/f19Deru3a212y3XdRlj0+kUe543GAwwxmEYUkpLpRLn3HXdwWDQbrdfvXqVZZku1enp6XffPRkOh1mWWZYlpcRpmuZ57vu+bhLGWBRFGONWq3V1dZUkyXg8llJWKpUHDx6sr7cRgru7u6VSKUkSXACDUDOYhlmWAQBSLphlm8waTwJM6Fbn7tevXznn/X6fi0wpJXLZ6XQ01VApVSq5+iyXy4YBarUapaRcLlsWe/Lkyfff/+HOnTueV+KcHx0dIYSazaZlWaZpwpLrJPHcsa1//fMff/3Ln//0xx+Gg368iJJ4YQDlOrbKs9//7rdJPKcE//jjsWWb/774ySRUpBxPpxMhBELIdV0pxXg8vrm53t3dDcMQQhQE43q9Pp1OsyyrVs3z8/NFND8+Pi6KYjab4VqtNhwOp9NpvV5vtVrT6fTz58/VatVxnEajEYbhcDgUQvi+P5/PNzc3McbD4VAPP4zj2LZtjDEAYDgcDgaDs7OzNE1N0/z06dPGxoZt22/fvlVKzWazPM+llEIIQkipVMKcc8YYxthxnJWVlW63e3x8nKbpzc2Nfun3+5xz3/ebzebz588RQhBCCGFRFLBSqejBKpfLjx8/fvbs2c7Ozmg02tnZcV03CIKlpSWEUKVSefr0KSFEy0aWZVEUYc55lmWO46RpahhGu91+8eJFpVJJ01S3calUsm17NBr5vj8ajaSU9XpdShkEAaaUxnFsWZbmHQCg56xer/d6vWazGcdxmqaVSgUhBABwHGdnZ6dcLk8mE6hbknOuBVij0JLkeV6v1zNNM8syzvl8Pq/VakqpZrPJOfc8D2txBP9nRVHoKlBKKaUHBwcIobW1NU2qTJPxeIwQopRiQoiWZSFElmV6XSilbNu+vLx8//796elpmqYvX76czWau6wKTfvnyBSGklMIYY52MdgYAKKWklL1e7+jo6MOHD8vLy51Op9FoXFxcCCFkmhweHmqMMM9zpZTOinMupdTBIYQfP37EGE+n0729PV1t13WFEJPJJIoi27axYRgQQr2K0jTV7HPODw4OhBDb29vVanV1dZVzPpvNut2uY1LXdeM4FkJgnTpjzLIsAACEMMsyIcTt27fb7TZCqFwun5+fr66uEkKWlpZ+Pv9arVaFEFEUGX9783cAQJIks9lMi5mWcUKIZVkY49FoxBibz+fVajWKIoYRAEDv0P8Cmv2l8vzGo0EAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<PIL.JpegImagePlugin.JpegImageFile image mode=RGB size=20x32 at 0x2B42911FFA0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ziffer_sortiert_resize\\6_10_ht6_20210628-141923.jpg 6 8\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAABQAAAAgCAIAAACdAM/hAAAEPklEQVR4nF1Vy44cyQ0MMjPrMT09M8JqDRiW4cXeDAP+/8+wL96DDnuc3ZVmptWvqsokGT5kSxZMFAr5IhNBMiLl3x//JSKqmnOepmm3283zXKQYjGRE1FrXdb1cLtfrtdZKYzcAWUREBEBEbNvm7tfrVVVLKRHh7vWrmRnJfhgAyQwoCRIRYRbrWvvGPM8R0VozM3ePCAAiiQzg5p/dAyAg//dfljXC3cPdSPR1EXH3Hp1kdiPhDIGEIGlC0izKbW2QYAggoASNAcLCvoIms7uTJIMkEOrqSlU1s6/ZUDLCYdYiguaQuDmbBW4mANzp7oCLKAlASEYgAoACERHfsq09JT2NqioiJN29jzs8AH03pdKnt2yTTCmpaq9Wznm3203T1E/3yl2v115zd08p9eRHRB6GQURSSqWUcRzv7u52u904jqUUgXj4sizLslwul+PxeDqd8J1lVfSSjGPZ7eanp8dpmARSvfbL97vdw+5+e3oYx6KK0+FECilA5JSSR2OoJozjWIYUiGabtQgaqMOYp3xXcso593AdgqpmTRCDKIahzHdjRqrRLpfz29vB3UT0/n73/r0OWn54/y4X/eX1F3feMLdWzSylBCAlBeBu67oeDm8RISLuNgzD09MTyVJKa62XMyKye3NvpEdYhEFvK+/ePUaEmUXYy8unw+F127bLZYmIiK83T2VChCCJJEABVc2q+XB4SSWrqqoGBERARBOS6jfM27KFcxzzcr4WzR6WUr6/37++HfNQoLnWer5uAFQVaYAmqocgyKxEBEgiCKqIJE2a7B9//2eFAVjW+nJ4663y5e0wiPSGA5CN7ogUTgYkVJNAclZHW9dlnub9tEt/Su5+Pp/D6na63JggcisayQABBgBISgmIWmttFYisOpScUjKzb8QgqQ4L8RCHBCTQRYV+Oh8v19Phy+tp+0K2ATqMeb/fBRm4fZl0IEQYgJMiBqC29vHjRwdF0+Pj44cPHyTLw3j/888/ff7t92+9rQ4PiZCAkhJBa7bVtprXaRrM2+l8rG0ztFM9dv71JtXORAqgIkoAzrBws9rcx3HsBMw5KxTApJN+b2UcgtxqneZ52dYeTnLa7/fH45Hkw8NDa00gj8PeYBHBroaqed22Fp6YjEgpZQx5GKA3PVfRnHNKqaEt63L8coqI3tsk89rMI1pb/I/PJY9//kspUpz4699+qrUKdL/fz2kGcKyn78WAZDaniK7berl+AjDOdz/+8ONUJhERaH9umrTz+fz8/PvLp8+1Gs1vrFq2NgyDpGLNL5fldDzv7x/mcQ4noBGxrst6XU6n0+V0dWfO2UW6zOfX19dpmlJK27bVZatbe35+LqXM867L6La15XJd17Xj7K9PH8t//vj1O8XlN6GlsQNzo5t1YRCRUtL/nF94rVYjopSSZSAYCIEolCSoqpqgAAgGA4gOmOR/Aaja0PTC+KDxAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<PIL.JpegImagePlugin.JpegImageFile image mode=RGB size=20x32 at 0x2B42BCE07C0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ziffer_sortiert_resize\\7_D7_2020-08-20_15-23-28.jpg 7 2\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAABQAAAAgCAIAAACdAM/hAAAE40lEQVR4nC2USZMcWRGEPeLFy6UyK7u6qhokmIELhzHD+P9/gxsI0DAzGpDU6r0rl7dEBIfGzK9+cLfPnf7mDkANtcLViAgwdw8hiJAwABhgDiYw0ABLrgAACBwgwKClqiozMwNADAQCAAIi4AQF1JCBnA0AMwsMYLii5pJzFpEYIxHV5NWNYdLEPiIAqiibFfByUSKKMQgZQIB6TbmkxC0oSGCaL5daK4DdrotjLwLPyFtWa5ZZmRk9CwxEsGo1V1dncKAQQELBYTBnJaqAQ4tp9pR1XWoIIb414gaYwSwQ9bHpo5ihH3e1AmohsDgsA8VJKW11XUqMvOsh5ADgajCPIn3bdg1SQt/AI2CsCqvQ4p4tIJRtTWt2ZS0uAMhhZgBikC6iCTDAKoQBghtyRtmKqQcELbXWypxVVRhwR992y2VupXXH7ZcMNQA5pXVdrToRRQ4xtqFth7H7+8d/dl337v1ZRKAKAEQEIGc8PzxeLpcvnz+7Yl3XtKRSSiD+w/d//OEvfy55dlSQmhcBAw4iYhYiUvV5nh8fn+Hcd30rrba6LMvD3d3T/ZNmBWzo2r5rhSBmMAMAdwcQAvX9cHXl7//0Q4zRszLz8rp8+PAB7gD2u+F8c2yapusbqQYH1FFKSVzatpmmaZqmoe2JUF3WdX15uZSi07jf7TpmnI43RCRBGIA7VHXLaUmbKqRt9vt+XlOtiC1t2/bp06fL5bLbjUwgQtftQoi1QpygjlKrqro70VsENE0jgl0HZtm2PI7T+XxTCpLDzWuu5MT/N5s6WGIMEaBQC8aBasXTE77e3j29vIzjeDyPqsi51mJv4uSAIKl5kKo0L4iCvsfdnZ6O+OXn/3z+/Pl0c266DoxUi6r++ONP377dN03HtaIazIkoMAciuMMUbRPmGXd397XW0+l0OF1xRGxjSmme53meU0piFQyQ2hveVlEdThhGvDzh9vZrbMK797/Z74fqCBHOVLSyBQrMDUEcVOGlaqlezappAQPPz4/LemkaOR6ndhdTKsxgIWciotBE6RjVADWqFhwNcwwIjIfH/Hj/TVreH0bp2ByKHGMsWtRr9VA0cwSCgdUZHDm2EX2HvsevP//y8HB3fX043RwNZlRCZBYgcNd1bd+ZmcBADnYKDnLAoQoobm+/bNv2u+9/fz6fU9lIiCMntf1+ON3cxBhj2wo7yECAu5tqziibu/u6JoUdjtfjNLwur8KBhZd17Yfh6uoqhBAj8bIgRuRU3Whb8zJvux29vr6uOX333XfTNAGYpmkYBgQOUdzx23fjfuqrmhAhJYQoUqTb9X3fPb3o7d1D3w1tt5MmKkhLdXWwi4gCy4pSiohwu8NWnKOoW9Gaa/3475/+8a+Px/PN/urQNpEJWqkWwCVwSMkfvn3b5jkyCQRbLSBaclInlubp+TnlfDyd237wN+BA5lwqiHD75eunX/97vL4+HA6yVWSrCsuurmViOpxO/TjGtq9VLxehQEZQ97xupZT72/vnp6eh7bRWyYbYtdVSkEYVa8rD1WHvcMK65S0VjswS3G1Nadu2GOPpcD1NUytCf311Zjw/lPl1yWsmp6EfS1YReVt4CIECm9WitWomd2Yex3G/b6Uoxh4iMh2urNdtS4d99/JSYxSzt9uCQZk5ROHQpXUbhl0IoRT8DwjJRwDK20XrAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<PIL.JpegImagePlugin.JpegImageFile image mode=RGB size=20x32 at 0x2B428F95760>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ziffer_sortiert_resize\\NaN_20200910-153847_ROI5.jpg -1 2\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAABQAAAAgCAIAAACdAM/hAAAE8ElEQVR4nD2Uy24cyRFF45VZVc3qbpJNcQRhRvIYhgfaeQAv9SP2xxuiLZmmRLLJrurKykdEeCHDu7s4m7M4F//2109GONU1gfJu8zyf/vL77+M4lpzLOd19/pyXtOTVAS4vLxGZqtRc+76PMRIAICIRubuZHQ6HH/v6+jrnPE2TmX348OHdu3fjOKaUSimqCgAiIsho6MCoTa3VDz//UQIDekrL8/GplDyM299++3Ouxd1bs+lpQSJAVDNiZgBwd1U1s9vb2yhBiP/95WtKaRw2APA/BnGeTw4KhO6+lizAYKbmTYRiH/s+tlaY+fPdP8hh3AxmFgJf9juOYXu5f7w/IkAwcUABADNDxHEcdzc3rdYYIyKeji9diJqLiByPx3G3RdOqdbsf01yKNgbGv3/6tJZcCS7fvnn7/mcV2u12qlrXjABpPjPz4fZNA3eEf/3zfp3K1y8PKaUYI5mZupPwbre7ubmpa2ZmBowhDLEbuv5wOKhqSimE8PHjx/3VlYioKjOLA6k6KpzndH//wBK+3X8DgLEfclMze3x8vr65SSlTWL98fZhOS9eFvo+lrOKOBq5Np2kqABVarUrgF3Fj3lo1CjhcbDebDSI/Px2/3z+VVGqtRCTIIhwNfF1L0te15taUCL+v34WpljbuxvWXsunH1/l8d3dXk3YcY2B3FzNwInQw85qbuoGTIIU+RGGN/tNPt+O4Q6acazUnAGYmtzklKVXV3BBbMycGoMhBmAJHQletRPLt2+M5L8/zzBQcciBkDEPshJmd0MCIwBCEIyMRkKmq++l0il0339+f8+qBYwjitL0Y9/t93/cSQgem6mpuzARM6KBqUcJm6NdUiKTkFkJHfRy6njgHkSCy3+1EtZqbaVVUNEbi4qbalnXi7mBkzapaHSSa6fxy3HE3La/LNL88PUvXYVuytvzrn/5w9fbNXHOIsZnGGNd1fQe/MPKvaoEjmp/n+fH+QZlqref5JGjKBEMv+914e3N1RR4uBgPItdRcIvd9iFBBHC3rFPvX/3xHJEdCdOlidHdjH/th7IeKzl1n7oE5qRMQmkNzNzc1NP9/wkQkPQePbqBCjA6mCmupYGvKx+MxLxkV2trYMGIopcQYqykRNTcRZlYihMjSS0AnjqG6PT88TseXvGRXsKxs2HHXTEMMbmRE5C6x6xStgTEjM7MZEYlDzrnWiohDH4ERzAQYGzkTIyAiuYt0IhBUmyGoW1NlVQMV4R96EgMgWlNCDmxFG6ABGiJQAyum6s0RgJFj6IYYh34cx9hHQ6vaUltXrQ0dmAzc3QEAAKSBGrshNvCmqqjNQ3VbNV8ernXb0pI5KCi00mrOXRBE+fHTAkzIhOhAAAyI5ADu/v79e3dv1VspWiDnfD6dp9fJSjEzVzMzaabu7m7ujohESOhgLchgZsggfU9d6Lqu5TbjxMyuBmZgLuhAgATIDujOTuyAzUpLRZs3Fw7dJsbNRrfNaju/nMhcHSKh9EgaQq2AVducDFSDpJTm5ZyrWjWRsN3uN8PouYljQABCEgJEwaqEAKUtLy9W0lqLxHBOaV7OqlqLO8LFsN1sNgis1dSqqlpthiCeKwtjKXk6l+V8XlOIMdeCiEFYWJqpLjk3JxJ3L6UYuDVVN2GkQKISBMlNySEgOUta156YJXDD1rzmTKjujkwMSIJk9l/TVqGYGDxy9AAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<PIL.JpegImagePlugin.JpegImageFile image mode=RGB size=20x32 at 0x2B428D53A90>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ziffer_sortiert_resize\\NaN_20200911-182857_ROI5.jpg -1 3\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAABQAAAAgCAIAAACdAM/hAAAFPElEQVR4nE2TO48c1xGFq27V7dfM9LxJkDSlwBIMy4AAK7F/gQOHzuTAv9mAQTvW8r3iDne6p7vvq6ocLAz4xOeLvnPwrz/9yMxZLaToyG+Ph/P5ebtevX33cRiGsBQico5LKSbKzDlHIuKneO9jjLFI3TbPnr84PDszV0uKwzCEEEpRgAqRmBmcAQARAUApRUQYAACAmdfr9eFw6Ps+xhxj/PLlCyACOMzJU8XMJpBS6vu1aBYR5xz+5ccfnOeuW/f73fZw7FYrRZdLqdu2rlpEyjk7IGbOsUzT+Obf/8o55iwAyqqKZgAgIjFGR1R3q67rruOY64xAKSUTQMQY8zRNiIZoACqSue/7OYZxumWVJDrME9EjIBoiMyOQiJiAmZVSUkrzfEMHRI59xc5XmLKUtCxLEs2mIqZmRUXAEJ1zDgVFBAGIyKFW3rftitnx7XYTU19VyFREY4rouGpqQkgpIbqqqlAtpUTITVORs7Zt27Z1zjGSI8MiknMCJO89OMwlIvn/WckiIipIaKbTNIcwjyM75xgRn2yZmZmS46Zryfu2XakqACBiSul2u83jbbw9rurWDJ8oBnM5pSxC3m93u2evXrx49XK734cQxNTMEFFyeXh4eHd39/HjRzAAcGaoCuy9z1KI/G6/f/7yxeF0JKJxHG/z9ETWdb3uVufzOS7LPM/LNBM6RFRVDimKSLtevXr16jfffmMI7z68ff/p4+Xr16cxbrfb169fH4/Hft+fptP75R0CmYGqMRGFEOZ5fnx8bFZd0nJ/f3+5XHIpABBCMLP9fn8+n3e7XYnp3S/vFdTMVJXX67WITGG5u7u7e/9uTiGloAin81kRJMxznG/TEOJcsUdEIgIFUVExHq8DGuw2PfpqTpGKrPum7/vhNhIRCICZ5IIG3nvf1N57VQOALMXV6NkoxjwM4xIjMhvQ7bZsms3l86WjzgNvu56AYk6+qkJeihZwQJ4Y1QDAgXPsamIlBAA0KFl3/dZUQW273VdVFSSGEBRASyml5Jz56SJg4Bw5ZiWMJUvOWaXfbJZ5appmt9txXVHWYbrvus6K5ZyZmZ1zIKJgBiooRTHGGOflH3//ufbVcH2MORxO+2EaQlzevHmzDKHmer1eH49HFjD4v5iJmSnCFJbxOtSV77rudrstKTRtu8RwPJ8qrvu+P51O/FRFQEBERCauGyTC3W73+PDQtrVYyaJVVSHR4XDQpNfr9Xq93t/f49/+/KcMWgySc8akDg0BDU67/TyO67YRK7//ww/b0y5Kvlwub/75n2mcU0p1XTMzo6mqgkkpqs4RESAOw7Uscfh6uY6P3//uOzP98nB/PJxXq5UWQ0Tvvau9d86ZmZnVdb3drGvPJcXGVwBwOBy+/+13b3+5I3S1r5q6RgPviAA1F16mWRAconeUUw6qVVUddvs8h5yC1nSbA1X7FOaK2Bl0TWNJS8qlFG5qH3Iycu2qA/ahJClmRbq2Rlmtu9UlLOu6dYCb9doB5pikFDAj55gAl2lW5uOz8/nFyymFD+8/XS6XvmtLziBqIl3bkgEhTNfHskTNhRSQiCv2DXvXNOfD8dtvvkkqbd1sV93jl4fasXe0XW32/ZYdSZHxOjCRkpEHBWAQbeuGmjbOy6f3H9ShpLxerTQkbXNFXPebFOKvnz6P4fb5868STbKZqCHgz3/8SRCEUIgEnRI65xhdRcwIIYS2q0JOWdIwjyGkw+YkBR2gY+LNZqMOo5ZFsoGrvDeHVmRZlq6uSsp+uxrHsWAmotPpBNmhcwBA6P4L7SN4TwZFTN8AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<PIL.JpegImagePlugin.JpegImageFile image mode=RGB size=20x32 at 0x2B428CEF370>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXkAAAEWCAYAAACDoeeyAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAjYElEQVR4nO3de5wU9Znv8c8zw8A4oMjF+4iDBi+gKDpEUGOMJniJUbLJHjUaNFkPnuSsIWZjjmyMlzWX3cTdTdz1RNkToydeoqLHZTVGRaMeEwVBEVFQWMU4iHITQeQ6/ewfVT20MJee7qqu6urv+/Wa13RXV1c9U9P19K+e+tWvzN0REZFsqks6ABERiY+SvIhIhinJi4hkmJK8iEiGKcmLiGSYkryISIYpyYtEyMyeNLOLk45DJE9JXjLNzJaa2UYz+9DM3jWzW81sQIXWfZGZPVOJdYl0RUleasEX3H0AcBQwBpiabDgilaMkLzXD3d8FHiFI9pjZODP7k5mtNbOXzOyk/LxhK/wNM1tvZm+a2fnh9GvM7PaC+VrMzM2sT+G6zOww4CZgfHgUsTbuv0+kM0ryUjPMrBk4HVhiZvsBDwE/BAYD3wXuM7M9zKw/cANwurvvChwHzOvNutx9IfA/gGfdfYC77x7ZHyLSC0ryUgseMLP1wNvACuBq4ALgd+7+O3fPuftjwBzgjPA9OeBwM9vF3Ze7+yuJRC5SJiV5qQUTwxb5ScChwFDgAOAvw1LN2rCccgKwj7tvAM4haIkvN7OHzOzQZEIXKY+SvNQMd38KuBW4nqBV/xt3373gp7+7/3047yPu/jlgH2AR8G/hYjYATQWL3bu7VUb9N4j0lpK81JqfA58D/gR8wcxONbN6M2s0s5PMrNnM9jKzs8Pa/GbgQ4LyDQS1+RPNbJiZDaT7njrvAc1m1je2v0akB0ryUlPcfSXwf4FvAWcDfwusJGjZX06wT9QB3wHeAdYAnwa+Eb7/MeBuYD4wF3iwm9U9AbwCvGtmq2L4c0R6ZLppiIhIdqklLyKSYUryIiIZpiQvIpJhSvIiIhnWp+dZKmfo0KHe0tKSdBgiIlVj7ty5q9x9j65eT1WSb2lpYc6cOUmHISJSNczsre5eV7lGRCTDlORFRDJMSV5EJMNSVZPvzNatW2lra2PTpk1Jh5KoxsZGmpubaWhoSDoUEakiqU/ybW1t7LrrrrS0tGBmSYeTCHdn9erVtLW1MXz48KTDEZEqkvpyzaZNmxgyZEjNJngAM2PIkCE1fzQjIr2X+iQP1HSCz9M2EJFSpL5cIyJd27plM3Pv+QlsWhfpcoe0fpERR30q0mVKMpTkY5a/wGvo0KFJhyIZ9OaC5xi35J8ByHk0R3t15rz4/mugJJ8JSvK94O64O3V1VVHlkhqQ27oZgJdPvpUjTvxiJMtcct3R4LmeZ5SqoGzVg6VLl3LIIYcwadIkDj/8cK677jrGjh3L6NGjufrqqzvmmzhxIscccwyjRo1i2rRpCUYstSQXJmMz7crSuapqyV/7H6/w6jvR1h5H7rsbV39hVLfzLF68mNtuu41169Yxffp0Zs+ejbtz1lln8fTTT3PiiSdyyy23MHjwYDZu3MjYsWP50pe+xJAhQyKNVWRH3t4ePNDRpXRBn4wiHHDAAYwbN45HH32URx99lDFjxnD00UezaNEiFi9eDMANN9zAkUceybhx43j77bc7povEyT1I8nV10bbXDN0WNCuqqiXfU4s7Lv379weCmvzUqVO55JJLPvb6k08+ycyZM3n22WdpamripJNOUp92qQjPBUne1JKXLuiT0Qunnnoqt9xyCx9++CEAy5YtY8WKFXzwwQcMGjSIpqYmFi1axHPPPZdwpFIrPBfW5OvqE45E0qqqWvJJmzBhAgsXLmT8+PEADBgwgNtvv53TTjuNm266icMOO4xDDjmEcePGJRyp1IqOlrxOvEoXlOR70NLSwoIFCzqeT5kyhSlTpuw038MPP9zp+5cuXRpXaCJ42Lumrj7iXdlVk88Kff2LVDFv3waoJi9d0ydDpIp52OK2CHvXOBonKUuU5EWqWS5oydfVKTFL55TkRapYR01e/eSlC0ryIlUs37sGdaGULijJi1SzXL53TYRJXvcuyBQl+QpqaWlh1apVZc8jkre9XKNdWTqnT4ZIFds+rEHUl7yoJp8VsSZ5M7vMzF4xswVmdpeZNca5vjgsXbqUQw89lIsuuoiDDz6Y888/n5kzZ3L88cczYsQIZs+ezZo1a5g4cSKjR49m3LhxzJ8/H4DVq1czYcIERo0axcUXX9zR3Q3g9ttv55Of/CRHHXUUl1xyCe350QRFesHUkpcexHbFq5ntB3wLGOnuG83sHuBc4NaSF/rwFfDuy9EEmLf3EXD633c7y5IlS7j33nu55ZZbGDt2LHfeeSfPPPMMM2bM4Mc//jH7778/Y8aM4YEHHuCJJ55g0qRJzJs3j2uvvZYTTjiBq666ioceeohf/epXACxcuJC7776bP/7xjzQ0NPDNb36TO+64g0mTJkX7t0nm5VvyUV7xqn7y2RL3sAZ9gF3MbCvQBLwT8/p2ltsC1EEZh7PDhw/niCOOAGDUqFGccsopmBlHHHEES5cu5a233uK+++4D4OSTT2b16tWsW7eOp59+mvvvvx+Az3/+8wwaNAiAxx9/nLlz5zJ27FgANm7cyJ577lnGHyndmnUzDGqBg09NOpIOi56fydaZP8Io7w5Me29dCUCdxq6RLsSW5N19mZldD/wZ2Ag86u6P7jifmU0GJgMMGzas+4X20OLu1DsvBr/3HdP794b69evX8biurq7jeV1dHdu2baOhoaFXy3N3LrzwQn7yk5+UHJP0wsPfC35f80GycRR4/6WHGL/5BRY2lDd89ob63Znb/2CO3GOfiCILaeyazIizXDMIOBsYDqwF7jWzC9z99sL53H0aMA2gtbW1Kj9Zn/rUp7jjjjv4wQ9+wJNPPsnQoUPZbbfdOPHEE7nzzju58sorefjhh3n//fcBOOWUUzj77LO57LLL2HPPPVmzZg3r16/ngAMOSPgvkYrxHNu8jsO+/6ekI5GMi7Nc81ngTXdfCWBm9wPHAbd3+64qdM011/D1r3+d0aNH09TUxG233QbA1VdfzXnnnceoUaM47rjjOo5URo4cyQ9/+EMmTJhALpejoaGBG2+8UUm+luRy5NS5TSogziT/Z2CcmTURlGtOAebEuL5Y7DjU8K233trpaw888MBO7x0yZAiPPrpThQqAc845h3POOWen6RqauDaYt5PTCU6pgNiaEu4+C5gOvAC8HK5rWlzrE6kqnu6WvL5+siPW3jXufjVwdZzrEKlKKU/ykh1V8SlznenXNsgY83ZyKe32qH7y2ZLOT1mBxsZGVq9eXdNJzt1ZvXo1jY1Vd8GwdMVzqslLRaT+Hq/Nzc20tbWxcuXK0hawdkXw+4OF0QWVgMbGRpqbm5MOQ6KS+nJN7Taqsib1Sb6hoYHhw4eXvoBrxoW/03MhjEj6k7xkhT5lIgkIulCmdfdTGSlL0vopE8k2d53glIpQkhdJQLpb8qCafHak+VMmklnmOTylXSglW/QpE0lCik+8qoyULen8lIlknJFTMpWKUJIXSYB5jpzVJx1Gl6yGLz7MGiV5kSS4WvJSGUryIgmwFNfkMX35ZEnqr3gVSdrShXMYfPdZNPrmkpfR17YBsMWDXe5I2vnPPgdFEp9Id5TkRXqwtu11WtjA84NOZ1vTHr1+f7/2DRz9XnCj97n7faVj+oBDPhNZjNFTTT4rlORFeuCeA2Dwyd/ioNHH9X4Ba/8MPw+S/PjJ/xJlaCI9SmlRUCQ9PBeUWqwuvb1hoqQ2fLYoyYv0wHNB2qurLzXJ60SmJEdJXqQH7mFLvoaGITC15zOjdj61IqXKBTX5klvy6pIoCVKSF+mB59oBsBRfoRotfSlliZK8SE/C3jWln3hV0pTkKMmL9CRsydfVa3eR6qNPrUgPvNyWvGrykiAlecm2CEZTzNfk6+tr49pBDZyWLUrykm1RDJnb0ZIvdXdR0pTkKMlLxkWQ5PM1+Rq54hU0nnyWKMlLtkVRrglb8iUnedXkJUFK8pJtYYIuS76ffI3U5PWllC1K8pJx0dXk61STlyoUa5I3s93NbLqZLTKzhWY2Ps71iewkitpyR++a2qnJayzK7Ij7+PMXwO/d/ctm1hdoinl9IjtQTV5qW2xJ3swGAicCFwG4+xZgS1zrE8nzXI63XnuBrZs+wto384lw+uIXny5peXXr2oLfNVKTVxs+W+L81A4HVgK/NrMjgbnAFHffEOM6RXjlTw9y+Myv7jR9xL9/oeRlbvIGGkpO8mrJS3LiTPJ9gKOBS919lpn9ArgC+EHhTGY2GZgMMGzYsBjDkVqxZf1qAGYdNpV+Q1vos2U9Xt+X9vp+JS9zt72Gc2Cf2mjJB9Sez4o4P7VtQJu7zwqfTydI8h/j7tOAaQCtra36ZEnZ8sMQ7D36cxxw2DEJR4Nq8pKo2HrXuPu7wNtmdkg46RTg1bjWJ5LXMaBYTfWGiZK+lLIk7uPPS4E7wp41bwBfi3l9ItsvXkrNMARKmpKcWJO8u88DWuNch8iOPH+7vrpaqqFHS19L2aErXiVzvOxRIyOmmrwkKCV7gUiEOu7klJZyTbXRl1KWKMlL9pR7hapIhijJS+a4WvLl03jymaEkL9mTr8lbSj7eqslLgnrcC8zsYDN73MwWhM9Hm9mV8YcmUhqvwTs5Rcn1pZQpxTR1/g2YCmwFcPf5wLlxBiVSlnxNPjUDiilpSnKKSfJN7j57h2nb4ghGJBKevxgqJeWaKmQauyYzitkLVpnZQYQjFpnZl4HlsUYlUo6wJZ+am3yo/CEJKuZ49n8SDCB2qJktA94ELog1KpEybL/iNSVJXiRBPSZ5d38D+KyZ9Qfq3H19/GGJlMHT1oVSLXlJTo9J3syu2uE5AO7+dzHFJFIe9a6JgGryWVFMuabwTk6NwJnAwnjCEYlAeCFPfVp616gmLwkqplzzj4XPzex64JHYIpKa47kccx+8mW0fropkef1XzAW2H3VK77jKS5lSSlOnCWiOOhCpXcvfep3WF3a6aVhZ3rE92Tc1XSiVNCU5xdTkX2Z7ga4e2ANQPV4is3XzRwDMPuJaDjk5mo5bQ5sGRLKcWqWvpewopiV/ZsHjbcB77q6LoSQy+fHf63cZyMBBQxOOJgYqG0mCukzyZjY4fLhjl8ndzAx3XxNfWFJLcu35K1SVDNNB/4cs6a4lP5egTNPZf9yBA2OJSGqOd9yTNSW9YSKnpCnJ6XKvcvfhlQxEalcun+TTMjSwaDz5DCmq6WRmg4ARBP3kAXD3p+MKSmpLfhgCq89okldNXhJUTO+ai4EpBN0m5wHjgGeBk2ONTGpGLheexzddoZoGGk8+W4ppOk0BxgJvuftngDHA2jiDkhqT+QHFlDQlOcUk+U3uvgnAzPq5+yLgkHjDklqS70Kplnx6aDz57CimJt9mZrsDDwCPmdn7wFtxBiW1paMLpWryIpHrrp/85cBd7v7FcNI1ZvYHYCDw+0oEJ7XB83dyUks+JfSllCXdteT3BZ41s6XAXcC97v5URaKSmuLt+XuyZjXJK2lKcro8Pnb3y4BhwJXAEcB8M/u9mV1oZrtWKkCpAa5+8umjmnxWdLtXeeApd/8GQRfKfwa+DbxXgdikRuQvhiI1o0ZGTDV5SVCxF0MdAZwLnAOsAqbGGZTUmPzt+jI7rEG10ZdSlnR34nUEQWI/F2gHfgtMCO/5KhKZfE3estqSV9KUBHXXdPo9wQnXc9x9QakrsKDLxBxgmbuf2dP8Unvy/eSzezFU9VE/+ezoboCygyJaxxSCe8LuFtHyJGM8HNbAsprkVZOXBMVaBDWzZuDzwI+A78S5rpq3YiEsmQnHXVqxVb69+CVWzLgGy5V3D5n+m4Pz+NntQlld1IbPlrjPdP0c+B7QZZdLM5sMTAYYNmxYzOFk2M2fhvbNFU3ybbNnMH79E7xV14wXNUJG115tOJwDmj8RUWRpo5a8JCe2JG9mZwIr3H2umZ3U1XzuPg2YBtDa2qpGRKnaNwe/3SteHhgy5WkGDBxS0XWKSHG6611TeAPvj71E0IV+dA/LPh44y8zOIBiHfjczu93do7lTs3Sugkm+44SpGqrdU01eEtRdS76snjDuPpWwP33Ykv+uEnwlVO5gKBeuqk5XqmaLvpQypbveNRppshpV8rZtaskXR0lTEtRjE8zMxpnZ82b2oZltMbN2M1vXm5W4+5PqI18h+bHZK7Gq8PukPqtDBNcw9ZPPjmL2zn8FzgMWA7sAFwM3xhmUlKOS5ZpgXWrJi6RXUU0wd18C1Lt7u7v/Gjgt3rCkZBUt1wS/NHpk1uhbO0uK6UL5kZn1BeaZ2U+B5RT55SAJqGS5hsqtS0RKU0yy/mo4318DG4D9gb+IMygpR+Va8tsPGtTyy5xKHhFKrIpJ8hPdfZO7r3P3a939O5TZvVJiVMGd05UIRFKvmCR/YSfTLoo4DolMAkleXQQzxXVklindXfF6HvAVYLiZzSh4aVdgTdyBSYkS6EIpIunV3YnXPxGcZB0K/GPB9PXA/DiDkjIkUq5Ryy9r1E8+O3q64vUtYHzlwpHyVXLnVCIQSbvuyjXPuPsJZraej+/N+QHKdBOQNKpgSz6nmnw26f+ZKd215E8If3c5FrykkHrXiEiBHi+GMrPBnUxe7+5bY4hHyqZ+8iKyXTFdKF8AVgKvE4xfsxJYamYvmNkxcQYnJaho61oteZG0K2ZYg8eA6e7+CICZTQC+BPwa+N/AsfGFJ723c+J99uZLGbF8Rifzlud01gYPVMPNlF73k9+yAf6lFb74SzjwpFhiktIVk+THuft/zz9x90fN7Hp3v8TM+sUYm5Sik37yg1Y+T4463hx8QqSrGrom+i8OqUIrF8H6d2DmNTD5yaSjkR0Uk+SXm9n/An4bPj8HeM/M6kEjVKVOJ+WaOtp5t/FAjv3Wb6Jd1zUDwwdqyWeN+slnRzE1+a8AzcAD4c+wcFo98N/iCkxKtfPOaTiu4YBFalKPLXl3XwVc2sXLS6INR8rWSUvePIfHOTq0avIiqdXdxVA/d/dvm9l/0Enz0N3PijUyKU0nNXkjp5a8SI3qriWfL+BeX4lAJCqd1OQ9B7EmebXks0c1+azo7orXueHvp8xsj/DxykoFJiXqrFyjlrxIzep2zzeza8xsFfAa8LqZrTSzqyoTmpSkk3JNHarJS2/o/5klXe75ZvYd4HhgrLsPdvdBBBc+HW9ml1UqQOmtzk68Om71CcQiIknrrnn3VeA8d38zP8Hd3wAuACbFHZiUqIt+8vG2ttXyyxrT4HOZ0V2Sbwi7T35MWJdviC8kiVrQT14teZFa1F2S31Lia5KkzmrycfeuUU0+W/T/zJTuulAeaWbrOpluQGNM8Ui5Oi3XqHeNSK3qrgulju+rUufDGqglL72h/2h2qHmXNV225PWdLVKLlOSzptN+8jG35CVTej2evKRabHu+me1vZn8ws1fN7BUzmxLXuqRQV10oleRFalEx48mXahvwN+7+gpntCsw1s8fc/dUY1ymdDmugoYalt9RPPitiS/LuvhxYHj5eb2YLgf0AJflOfLBmJR+uXVHy+/cLf69oW8LWjz5+uD3I1ZKXmLjDNvWoTrM4W/IdzKwFGAPMqsT6qs2mjRvo84vD2c82lb2sPR+8cOeJBt5nl7KXLWUauH/SERSpFzX5J34I/18D1aZZ7EnezAYA9wHfdved+t2b2WRgMsCwYcPiDieVNn/0IQNtE3N2+yx+0CklLaPOt9G4aQUf7bLvTq+Z1XHYcRPLjFLK8o1nYde9k44iei9GfEtJiVysSd7MGggS/B3ufn9n87j7NGAaQGtra00WAnO5oEdM+76tHHv2NxOORmKx18ikI+gV3eM1O+LsXWPAr4CF7v5Pca0nC9rbtwUP6tSXXUSiFefZuOMJRrI82czmhT9nxLi+6hW25HVyVFKh1G7yGrkyleLsXfMMujq6KO25oCVvSvIiEjFllRTwsCVvKtdIapTQKtcYRqmkJJ8CufZ2AF2wJCKRU1ZJAfcgyaslL+nQixa56vCppySfArl21eRFJB7KKimQy4WtoXq15CUdim7Lqw6fekryKeDqXSMiMVFWSQHPqSYv6VHyePKqz6eSknwK5HvXoLs3iUjElORTwMO7OdWpJS+pUWSrvLD1rvp8KinJp0BHS75O/w4RiZaySgqoJi+p0psWuVrvqZftJF8lJ4Ly5Rr1rhGRqCmrpMD2lnxFbtQl0iONJ58dyiq9sOj5maxf9lrZy2nctIJtDQPYVt8EwLaVrwNgdTr0FZFoZTvJR1yuaX7wAgbYxkiXWaj/4H1iW7ZI8Uocu6ZKyqO1JttJPmKNbGbW0L+g+YzvlbyMuq0b2Oeu4D6uyyY91zG9X9MAhu9dLTd6FpFqoSTfC3U43jiI/Q48rPSFbNnQ8bCs5YjEyIptlRf2rlFPm1TK+InX6A4fPZejzrz8Md/Vg0ZEKkgZp0i5/H1Yy+3LrqELJOVKHrtGUklJvki5XH58mTJ3ALXkJUt04jX1sp1xIvzQtYc39ii7Ja6rWkWkgrKd5CPUcbPtsmvyOhSWjNJnO5WU5IvU0ZLXIGKSdaWOXaNyTSplPGNF96HrOPGqE6ciUkUynuSj4x01eW0ykU6pXJNKylhFyrfkNRyw1IKiByhT75rUU5IvUrta8iJShbKdsaJsWUR1MZSISAVlO8lHqD0XtOR1Yw+RAqrDp54yVpFcNXmpKSUcBSvhp1LGk3yEXSjDm22XPUCZSOqVmKx14jWVYs1YZnaamb1mZkvM7Io41xU3d91sW2QnSuypF1uSN7N64EbgdGAkcJ6ZjYxrfXHLtasmL9ItlWtSKc6bhnwSWOLubwCY2W+Bs4FXo17RrH/9GpbbutN08xxjw8ezb7igrHX02bKO/QHq1ZKX7CvpRt5q1adSnEl+P+DtgudtwLE7zmRmk4HJAMOGDStpRc2r/0hf39LtPMPXPFPSsgu9Y3sx+IDRZS+HEafCsJ02hUgqeKlj10gqJX77P3efBkwDaG1tLakpsN/Vr/c4zx6lLDgu59+TdAQiUiPiLDAvAwrvTN0cThMRkQqJM8k/D4wws+Fm1hc4F5gR4/pEJCJFF2EK6/Aq3aRSbOUad99mZn8NPALUA7e4+ytxrU9EoqJ+8lkSa03e3X8H/C7OdYiISNfU6VtESqcSTeopyYtIJ1R6yQoleRHZQS9a56rDp56SvIhIhinJi4hkmJK8iOzESinD6CRsKinJi8jHlZqsVZ9PJSV5ESmdWu+ppyQvIqVT6z31lORFZCdqn2eHkryIfIwrxWeKkryISIYpyYuIZFjid4YSkfQZ4mtY+neH9zhfS27F9ifL5xX1HtnZR/UDGfn9P8aybCV5Kd3XHoY1byQdhUSs/9gLePnZDRQzSNnG9j05bONcABY0HcuWun4xR5dN2xp2i23Z5inqAtXa2upz5sxJOgwRkaphZnPdvbWr11WTFxHJMCV5EZEMU5IXEckwJXkRkQxTkhcRyTAleRGRDFOSFxHJMCV5EZEMS9XFUGa2EnirxLcPBVZFGE5U0hoXpDe2tMYF6Y0trXFBemNLa1zQu9gOcPc9unoxVUm+HGY2p7urvpKS1rggvbGlNS5Ib2xpjQvSG1ta44JoY1O5RkQkw5TkRUQyLEtJflrSAXQhrXFBemNLa1yQ3tjSGhekN7a0xgURxpaZmryIiOwsSy15ERHZgZK8iEiGVX2SN7PTzOw1M1tiZlcksP79zewPZvaqmb1iZlPC6YPN7DEzWxz+HhRONzO7IYx3vpkdHXN89Wb2opk9GD4fbmazwvXfbWZ9w+n9wudLwtdbYo5rdzObbmaLzGyhmY1PwzYzs8vC/+MCM7vLzBqT2mZmdouZrTCzBQXTer2NzOzCcP7FZnZhTHH9LPxfzjez/2dmuxe8NjWM6zUzO7VgeuT7bmexFbz2N2bmZjY0fJ7oNgunXxput1fM7KcF06PbZu5etT9APfCfwIFAX+AlYGSFY9gHODp8vCvwOjAS+ClwRTj9CuAfwsdnAA8DBowDZsUc33eAO4EHw+f3AOeGj28CvhE+/iZwU/j4XODumOO6Dbg4fNwX2D3pbQbsB7wJ7FKwrS5KapsBJwJHAwsKpvVqGwGDgTfC34PCx4NiiGsC0Cd8/A8FcY0M98t+wPBwf62Pa9/tLLZw+v7AIwQXWw5NyTb7DDAT6Bc+3zOObRbbTlyJH2A88EjB86nA1IRj+nfgc8BrwD7htH2A18LHNwPnFczfMV8MsTQDjwMnAw+GH+ZVBTtjx/YLd4Dx4eM+4XwWU1wDCZKp7TA90W1GkOTfDnfuPuE2OzXJbQa07JAYerWNgPOAmwumf2y+qOLa4bUvAneEjz+2T+a3WZz7bmexAdOBI4GlbE/yiW4zgsbDZzuZL9JtVu3lmvxOmdcWTktEeLg+BpgF7OXuy8OX3gX2Ch9XMuafA98DcuHzIcBad9/Wybo74gpf/yCcPw7DgZXAr8NS0v8xs/4kvM3cfRlwPfBnYDnBNphLOrZZXm+3URL7yNcJWsipiMvMzgaWuftLO7yUdGwHA58KS31PmdnYOOKq9iSfGmY2ALgP+La7ryt8zYOv3Yr2VTWzM4EV7j63kustUh+CQ9dfuvsYYANB6aFDQttsEHA2wZfQvkB/4LRKxtAbSWyjnpjZ94FtwB1JxwJgZk3A3wJXJR1LJ/oQHDWOAy4H7jEzi3ol1Z7klxHU2vKaw2kVZWYNBAn+Dne/P5z8npntE76+D7AinF6pmI8HzjKzpcBvCUo2vwB2N7M+nay7I67w9YHA6hjigqAF0ubus8Ln0wmSftLb7LPAm+6+0t23AvcTbMc0bLO83m6jiu0jZnYRcCZwfvgFlIa4DiL40n4p3BeagRfMbO8UxNYG3O+B2QRH3EOjjqvak/zzwIiw90NfgpNfMyoZQPjN+ytgobv/U8FLM4D8WfkLCWr1+emTwjP744APCg6/I+PuU9292d1bCLbLE+5+PvAH4MtdxJWP98vh/LG0Et39XeBtMzsknHQK8CoJbzOCMs04M2sK/6/5uBLfZgV6u40eASaY2aDwSGVCOC1SZnYaQWnwLHf/aId4z7WgJ9JwYAQwmwrtu+7+srvv6e4t4b7QRtBR4l0S3mbAAwQnXzGzgwlOpq4i6m0WxYmOJH8IzpC/TnDW+fsJrP8EgkPm+cC88OcMgtrs48BigjPog8P5DbgxjPdloLUCMZ7E9t41B4YfmCXAvWw/s98YPl8Svn5gzDEdBcwJt9sDBL0YEt9mwLXAImAB8BuCHg6JbDPgLoJzA1sJktNflbKNCGrkS8Kfr8UU1xKCenF+H7ipYP7vh3G9BpxeMD3yfbez2HZ4fSnbT7wmvc36AreHn7UXgJPj2GYa1kBEJMOqvVwjIiLdUJIXEckwJXkRkQxTkhcRyTAleRGRDFOSl0wzs5+Y2WfMbKKZTQ2n3Wpmb5rZPDN7wczGh9N/ZwWjJ3ayrIlmNrJCoYtEQklesu5Y4Dng08DTBdMvd/ejCIZTuBnA3c9w97XdLGsiwQiBRSu4UlYkEUrykkkWjG8+HxgLPAtcDPzSzHYcw+Rp4BPhe5YWjDU+KRxj/CUz+42ZHQecBfwsPAI4yMyeNLPWcP6h4WXzmNlFZjbDzJ4AHjez/haMJz47HJDt7EpsAxEIBsgRyRx3v9zM7gEmEYyp/6S7Hw9BuaZg1i8QXO3YwcxGAVcCx7n7KjMb7O5rzGwGwZXD08P5ugvhaGB0+L4fEwx58PWwHDTbzGa6+4ZI/liRbijJS5YdTXBjhUOBhTu89jMzu5JgyOO/2uG1k4F73X0VgLuvKWHdjxW8bwLBYHHfDZ83AsM6iUkkckrykjlmdhRwK8EofauApmCyzSO48QIENfnpZa5qG9tLno07vFbYSjfgS+7+WpnrE+k11eQlc9x9XnhSNX8rxieAU939KHffWMQingD+0syGQHBf1XD6eoJbPOYtBY4JH3+Zrj0CXJofK9zMxhT5p4iUTUleMsnM9gDed/cccKi7v1rse939FeBHwFNm9hKQH0L6t8Dl4cnTgwjuIvUNM3uRYBzwrlwHNADzzeyV8LlIRWgUShGRDFNLXkQkw5TkRUQyTEleRCTDlORFRDJMSV5EJMOU5EVEMkxJXkQkw/4L4dXCyEfn3GMAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "Input_dir='ziffer_sortiert_resize'\n",
    "res = []\n",
    "only_deviation = True\n",
    "show_wrong_image = True\n",
    "\n",
    "files = glob.glob(Input_dir + '/*.jpg')\n",
    "\n",
    "for aktfile in files:\n",
    "    base = os.path.basename(aktfile)\n",
    "    target = base[0:1]\n",
    "    if target == \"N\":\n",
    "        zw1 = -1\n",
    "    else:\n",
    "        zw1 = int(target)\n",
    "    expected_class = zw1\n",
    "    image_in = Image.open(aktfile)\n",
    "    test_image = np.array(image_in, dtype=\"float32\")\n",
    "    img = np.reshape(test_image,[1,32,20,3])\n",
    "    classes = np.argmax(model.predict(img), axis=-1)\n",
    "    classes = classes[0]\n",
    "    if classes == 10: \n",
    "        classes = -1\n",
    "    zw2 = classes\n",
    "    zw3 = zw2 - zw1\n",
    "    res.append(np.array([zw1, zw2, zw3]))\n",
    "    if only_deviation == True:\n",
    "        if str(classes) != str(expected_class):\n",
    "            print(aktfile + \" \" + str(expected_class) +  \" \" + str(classes))\n",
    "            if show_wrong_image == True:\n",
    "                display(image_in)\n",
    "    else:\n",
    "        print(aktfile + \" \" + aktsubdir +  \" \" + str(classes))\n",
    "        \n",
    "\n",
    "res = np.asarray(res)\n",
    "\n",
    "\n",
    "plt.plot(res[:,0])\n",
    "plt.plot(res[:,1])\n",
    "plt.title('Result')\n",
    "plt.ylabel('Digital Value')\n",
    "plt.xlabel('#Picture')\n",
    "plt.legend(['real','model'], loc='upper left')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Save the model\n",
    "\n",
    "* Save the model to the file with the \"h5\" file format"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Assets written to: C:\\Users\\Muell\\AppData\\Local\\Temp\\tmphkpdxn0s\\assets\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "219292"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "FileName = TFliteNamingAndVersion\n",
    "\n",
    "converter = tf.lite.TFLiteConverter.from_keras_model(model)\n",
    "tflite_model = converter.convert()\n",
    "open(FileName + \".tflite\", \"wb\").write(tflite_model)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Assets written to: C:\\Users\\Muell\\AppData\\Local\\Temp\\tmp0zsu6ehb\\assets\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Assets written to: C:\\Users\\Muell\\AppData\\Local\\Temp\\tmp0zsu6ehb\\assets\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "dig1320s3q.tflite\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "63728"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "FileName = TFliteNamingAndVersion + \"q.tflite\"\n",
    "\n",
    "import tensorflow as tf\n",
    "\n",
    "def representative_dataset():\n",
    "    for n in range(x_data[0].size):\n",
    "      data = np.expand_dims(x_data[5], axis=0)\n",
    "      yield [data.astype(np.float32)]\n",
    "        \n",
    "converter2 = tf.lite.TFLiteConverter.from_keras_model(model)\n",
    "converter2.representative_dataset = representative_dataset\n",
    "converter2.optimizations = [tf.lite.Optimize.DEFAULT]\n",
    "converter2.representative_dataset = representative_dataset\n",
    "tflite_quant_model = converter2.convert()\n",
    "\n",
    "open(FileName, \"wb\").write(tflite_quant_model)\n",
    "print(FileName)\n",
    "Path(FileName).stat().st_size"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Checking the images shows, that this are border line images, which can be interpreted as a good digit or a faulty one."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}