Gesture Based Visually Writing System- Adding Virtual Animation, New Mode and New VUI.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Gesture Based Visually Writing System- Adding Virtual Animation, New Mode and New VUI"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Contents\n",
    "* [Introduction](#Introduction)\n",
    "    * [What now?](#What-now?)\n",
    "* [Before Anything](#Before-Anything)\n",
    "* [Credits](#Credits)\n",
    "* [New Mode](#New-Mode)\n",
    "    * [Concepts](#Concepts)\n",
    "    * [Prepare Dropdown Icons](#Prepare-Dropdown-Icons)\n",
    "    * [Attach colors image on VUI](#Attach-colors-image-on-VUI)\n",
    "* [Adding Animation Effect](#Adding-Animation-Effect)\n",
    "    * [Change Color](#Animation-Effect:-Change-Color)\n",
    "    * [Scale](#Animation-Effect:-Scale)\n",
    "* [Bonus Topic](#Bonus-Topic)\n",
    "    * [Final Result](#Final-Result)\n",
    "* [Finally](#Finally)\n",
    "    * [Codes](#Codes)\n",
    "    * [Features](#Features)\n",
    "    * [Shortcomings](#Shortcomings)\n",
    "* [What Next?](#What-Next?)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Introduction\n",
    "From last some weeks, i have been busy with some simple yet hard task of writing anything on canvas just by moving fingers (still is writable by moving anything) in front of the camera. Initially i got huge help to extract contours from [this article](https://gogul.dev/software/hand-gesture-recognition-p1). Then i was on my own to think something new and try to make it happen. Up to this point, a user can work with different modes and these modes are accessible by either keyboard or from the VUI region of the frame. But why to use keys when one can perform task using only waving fingers on the air?\n",
    "* exit:- Exit from system.\n",
    "* save:- Save current drawing(excluding pointer).\n",
    "* move:- Idle mode(move only pointer).\n",
    "* erase:- Erase mode.\n",
    "* color:- Color mode.\n",
    "* erase all:- Erase current canvas but leave the average image.\n",
    "* restart:- Restart the average and canvas.\n",
    "\n",
    "### What now?\n",
    "* Now on this blog, i will try to add some new features. When a user moves pointer on canvas above the modes icon, i want to make it look like animated. It can happen just by scaling that icon image little bit.\n",
    "* I will add new option to change color.\n",
    "\n",
    "This blog will be divided on to 3 parts. Simple to harder.\n",
    "* Introducing new Mode\n",
    "* New VUI\n",
    "* Virtual Animation on Icons\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Before Anything\n",
    "Before anything, i recommend you to view previous 2 parts from the beginning to make it easier to understand here. \n",
    "\n",
    "* [Gesture Based Visually Writing System Using OpenCV and Python](https://acharyaramkrishna.com.np/2020/08/01/gesture-based-visually-writing-system-using-opencv-and-python/)\n",
    "* [Gesture Based Visually Writing System: Adding Visual User Interface](https://acharyaramkrishna.com.np/2020/08/11/gesture-based-visually-writing-system-make-a-visual-user-interface/)\n",
    "\n",
    "Also while begining this blog, i only had idea on the mind so there is not much chance of being this system best but i will try to make it work. I have not proceeded to do codes yet but will figure out the ways later."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Credits\n",
    "I want to give credits of this blogs to everyone on [LinkedIn](https://linkedin.com/in/ramkrishna-acharya-91a217183/) who reacted, shared and commented my previous blog and on twitter also (most of retweets was from bots lol). My journey on AI was inspired by [mr.Pujan Thapa](#), (i have met him only once but he doesn't knows the truth) so i want to give him huge credits.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## New Mode\n",
    "New VUI is need for working with multi color. First i will make it work by keyboard. Then by contours. For simplicity, i will use 3 colors, Red, Green and Blue. So this new mode will be color mode. \n",
    "\n",
    "### Concepts\n",
    "Again, it will be very helpful if someone finds a better way of doing this task on lesser computation but i am writing an idea. From the last system, i am going to use the variable `vui` which is stackked horizontally before showing it to user. Now on this `vui`, we only stackked top part of it but we rejected the bottom part i.e. ```python new_canvas = np.vstack([vui[:100], drawn_new]).astype(np.uint8) ```. But now, we will use the other parts also on the case when we have to choose the color from virtual dropdown menu. What i mean by virtual is, it will act like a dropdown but in real, it will be just an horizontally stackked arrays. \n",
    "* Take a final `vui` and when the mode is color, \n",
    "    * Find where our pointer lies\n",
    "    * If pointer lies within the dropdown area then take the coordinate and the color related to the dropdown's that part.\n",
    "    * Then change the current drawing color.\n",
    "\n",
    "### Prepare Dropdown Icons\n",
    "A code is given below. I prepared an icon for colors also using paint. Please follow the comments for better understanding.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-08-14T00:55:11.491399Z",
     "start_time": "2020-08-14T00:55:08.679175Z"
    },
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlYAAABgCAYAAADfGTyGAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO2dfZgU1Z3vP4dXeRHHkRcnzCgoKBsCCTIgkQjGEB2RiLAmhLiJCb6ERK94496VxewSsonC+pgsZhPfbrjRXUPMjRCyxh00L4IPXpAZMCAKMiBmBpG3CRIBkZdz/6iu7tPVVd3V3VVd1T2/z/PUM92nTtU506dO1bd+53d+R2mtEQRBEARBEIqnU9QVEARBEARBqBREWAmCIAiCIASECCtBEARBEISAEGElCIIgCIIQECKsBEEQBEEQAkKElSAIgiAIQkCEIqyUUg1KqW1KqRal1NwwyhAEQRAEQYgbKug4VkqpzsCbwGeBNmA9MFNr/XqgBQmCIAiCIMSMMCxWY4EWrfVOrfWHwC+AqSGUIwiCIAiCECu6hHDOgUCr8b0NuNSZSSl1G3Bb4uvoEOpRNnTv+vGoq1ASjp/4U9RVEARBEIQgOKC17ue2IwxhpVzSMsYbtdaPAY8BKKU65Lo6nbooTp/UHD/xJ4bUHoy6OiGiaWnrC8Dpb91Cpx/874jrIwjxYkhtEzAYgJa2c1z2H3RNFwQhMt722hHGUGAbUGd8rwXeCaGcsmfyM4Pp1rsz4H4zrQxSourkXV9jw+vbI66PIMSTlrZzPO8DdrolwMaUsFaCIORLGMJqPTBUKTVYKdUN+CLwmxDKqQiueur85OdKFFe2qOrX4wxe3boz4toIQnkwpPYgfRJ/YQxDag8ypO9NwGCG1DZGXDtBELIR+FCg1vqkUuoOYCXQGViitd4SdDmVxJQVF7Dqm238dfeHtLSdUyHDgilL1Qe3f4XXdnhaTQVBcHBo3zn0rz1Iy75HsN5VoeXAEwyp/R+0tH0j2soJQohMWXFB0ed4dmq0L/Fh+FihtX4OeC6Mc1cqE39Sy5q7d/OXluMVIK5Souro7L8TUSXEBrtfxd06XFX9fQBq+s9mT9u9EddGiBtez4e4X9d+KVQYBSHKgiAUYSUUxvgHB7L23j0ceO1YGYurlKh6/+szeX1Xa478weLsWFG/uQjuDKk9SNv7b/HBofqSlef1PS4PozTR12V28h5wRlVTMk9bW704sndAnNevV/vH8bruiFSEsLq7CmaOmwhYcR6mNa6KtkJFMO77Naz/3rvsXX+0DMVVSlQdvvkLbH07/DkLud5QzP0isuJDS1sDsJ7zaw/y9l9+x5CzJwFw4v0GuvZupK3tHGpDEBBPPVQNwI13tifT4iBUWtrqHd/PSf97yEr/AHlgdiTytbCa+URkRUdFCCuTutxZYs+Yb5+bFAHvH3uO3j0mR1wjP6RE1YSP9OfN3XtDLc0pqO6pfsIz76L2m9KOEYEVH7oCHJkBZ1sPgb2H1lPbG2prbSvNTYB32/rB7eXkqYeqYyeuBMGk2GHr1EzSg3J9l5hYC6vpVTAvYYkyqS9ji5Rfpqy4gGev38m7B78MeI+px4PTtLRZcdIa6j7C94YPDbU0U1RlE1RueRa13yQCywM/11hQN+chtY3s+8CZ+jt69xrDkZNvsefd+kR9ghNVtrXK+d0WWOXifyVUNkFfh/bIh4ir0hHKIsxB4SaqAJZPcE+vNKb8+gI6dbXirca3Q6RE1Y1Dzi+ZqLqn+glfosqJeUxcHB1NLpixEoBRs1cyed5KGDWbGfOWAB9lxryVyXzm57AY/ZWm5GYTlMBvaTuHwwfOSRvyammbwYEj69nzbn0yrRiyiSoT5754v8QIlUxY4t6OkSbXdmmIpcVqJLCkwVs81fWEpoaJHcJyNflXg9n13Hu89ujBGHaMlKja/6XreLv9vdBKMgWVE3uoLxf2sfbfOFuvzqyGMwE2PgLXTGPGvB8Cu5kxezZvMo2nn1weaHn2dTX6K000P2kJm+Yn65Oiyv7b/GR5OE/7FVXOPFFYr4Ls03FvF8GbUlxz9jNErpNwiaXFKpuo6ogMmnwWl/yv/kCcbpwpUbV3xpRQRVU2TFFlW7Gcm1teO3/cGHMhTJ69ktX3XZ2x7+n7ZvHX6mlcVL0b2h4JrExTVNl/O99sCQ1bZNnYeeyhhXgJ/Uz8iKps+cvhfxTKH1vs5Ht/L6Qfxu8FvfJQWke/TJ9zrcAmn8LKtliZswLN9LiT71DUgT8dZe0/vwtEPVyRElXvfuFa2g6/n9fR+baPl7XKFkp+BZJXfjs9blarUmBeR7ZoerXrVcm0Uz9tT9tn4xRcEB/Rb/9P+YoqJ6ZzO4T3/wUVkygulojlE0ZQ1zP129ev3QiHDud/XJncx4slH0tVtrALfkMymPnjcL04mbLigqLjWJXoXt6stXaNFxM7i1VTw4ioqxBb+n68J2ee1w2A3ftviKwetqh6/JOX5C2q8sVNVC1qvylvUWXmNY830+Poc1UqnMIJ4BMnnk9+bn6yPm1zEpcbdJAvHKX2vYrLb1gspjgCaBo3CuiT9ZimhhEZxy3uQCMXfkSQKcDcrFvOdD+WLLFchUPshBWkd641R603F3tbs2VjRPWKBxN/VEvVRd05dvyPtO1rKGnZWp9KdtqnLx/LqLPOLEm5XsN5xTqv+/XNqlScQ4CQslbZosp0YO98czWdb67OEGFxEQSmpapYa5WN81xhDw3m+2AsFxY3jMqRI7O9xodTlVjhx2rkFFR+cF5HXnmEcIiVsJrukjZndbo5eE7rYeBYSeoTVz71wED6frwHH3y4vmSdQ+tT7Nht+Xk99+lPcmGvHqGX6WVBKnRGoHm8V1pHsVr5EVXlRNgCJCih5pdsQzzlSEcQSWFQqO+VTa7ZgOJvFQ6xElbzHMOAXouh1De+AkfbPfZ2DMZ9t4aay3oB4b95mKJq9aTx9O/eLdTyTMKyMJnDgh2NQkSVud8cCozDW2++MwALwelvVQoq1XolWOSyVgXpAyUCqrTkDLeglFoCTAH2aa0/lkirBp4GBgG7gC9orf+ilFLAYmAycBT4qtZ6g//qpN8UpzV6D/vVr97s/7QJ3Jzi69e+CYf25HVM61GYtnojkOmQaeevb7T2m8cH7Yw5+p4BaK357fVvhdZxTFG187pJtH94IvAy3IjCcnRP9RMVL7REVOWPLaqc0dpLhdm3KzGI6fJh2f2vOiJhOJa7hVrox0Lea3ucfixkP3MDLa8j48di9TPA6cwzF/i91noo8PvEd4BrgKGJ7Tbg4eKql3smiV8W17l33qZxF3kes3z0ENf0up6w3MVn4O6q1OeRw4YxvSojS+AopbjuuQuB4G+2pqjaPuUzJRNV2QgyRILXuTrKcCDkJ6pM4vBgr1RLlRum9SoOv32Q1A0aFnUVSk62l+Cwffjs8/djIUBSUNnfheLJabHSWq9WSg1yJE8Frkh8fgJ4Ebgnkf6ktmI4rFVKVSmlarTW3iahEjF+uLfz5PTRY1nW/EpaWlPDWMDbj8hak7APpvg7z+HL7RU5PmhOn9Bc/HfVbPvPdnbsruXCgW2BnNcWVW9MuZL3Tp4M5JyFEqYlaVH7TbGMaRU0TmuVGVbBDaeosq1VcXuwl8L/qdQ+VtmI6zR5d9pxc0xPJ3x/zTiSrQ3Dbl+nhWo/c+nHwpJZrsJ+ec12/lKEYijUx2qALZYSf/sn0geS7hrVlkjLQCl1m1KqSSmVOc8bWFNgxdzwslbZzOvn1rFzd/bpo3PNdCkdQz9fxYjb+6L1MVra+uc+IAd2x97xuc9w5OSpos9XCH7ETo/vf5se3/+2I+3HHnnd0/Mts9ww/XNyhVWwibuoCipWVTac1ir7e1x+g3JgzRbDZaMuuFA6TQ0TM7Zs+e4ePdZj/1iaGiay3OX46VWZ5eQKG1EOvNf2uKt42s/cpMASiiPoJW2US5prBFKt9WPAY5AZIBRgzpbgTPDZrFVJqobAoZa8zjuvHyzz2LegeG2TN+df1YdufTrTfP/egn2uTp8+yc53BgDw1nWTOBiD4b9sHLv3e8bbwQE6r/g3oAaAzi//mG5/tIylH87+XhTVixV+/ariPjuw1KIqTtaqcmNOKzQNtz4vH17NNK8ZSXmQTUTN2tLOplZLzJmzzGf268GDrkdZL9B1jtTlo2uo65fpJtLUMCrUwKVhWyP9WKRscRWm5erZqTuLCgSa7bxulHrpskItVnuVUjUAib/7EultpF+jtcA7fk6YEWqh9UCBVUtnsaPHmDGxTJrGuRrWAJi15U3qG1dx31r/MbTqekZj3q4Z14vLFn4EyP/N2hRVb0+Nv6iy6f79X9EZgL6cmpoSUKcuHsGxe63vpzqokSFfZ3W3fXGyVpViZlNHEVVTVlxQUn9Cp3iBdL/UpUdznyPXqhxLhqfay+ul1w9uospvHUJho4avXea+7/B2X+m2Jcoe8nPD3Be25coWV7muwenDxjJymLvF0Q9RrAdbqLD6DWA7vdwErDDSv6IsxgHv+fWvGpzh6H2kwKqlM35wygE9c3gxt1Wsfu1GNrVa/8KyQ4fLIsxD9d+cwZWP1gL+H4amqPrzdZPYf7w8RBXAsXtv4FTa99utD+dMSHz/Xnp6B6FYUWVGWY+bqCqFs3qliqpSC6pszByXGh58cNPukpbtFjcRYGTI5XpZpbK+NFwILLafYLfBYW1tTg5r+Jr7KeyhPj8WKz/5giTb9ThvUA82vWv1y6aGiTRNGMJ07HaqySp0oxBV4ENYKaWWAv8PuFgp1aaUuhlYCHxWKbUd+GziO8BzwE6gBXgc+KbfijgdvwObEdgzZYma47BSzWrMHrKhvnFVxhpX+YZ5qG/c6GohC5ue53aj4eeDAH8PRVtU7Zo6iX0xsVRFEfqg0sItFCOqbERUVQbxEFQ1ju/G73xoH9lwjj74L8M9faLH+RY4wj/Ur93IrLWluX979rUdwKSEp83Yj0IfZW2muPpV9Ov+5suzU3fmFD31jatoGjeQ6aMToYxWtzCvYQRDAKp65Xy2RrEGrJ9ZgTM9dn3GJa8GysYssCnLviCE0H37IciQEfnSpVcnpq68kBVX78gZfReg5XOf4UAZWaoEb5xtnWsGoBcdRVSZVJqocoop+0FTMpF1dHfyBXdxw0XMafQaxMh+rxw/PGWZaAWmGfdo02rR1HAR9S5l3D3hIh5cnUofPzw1vGTdqy3qBqV8cu3nwKbEZ7ucxQ0TM17U/ZJtqRnPOGWjDPflV+5Kfe5jpN/gcHHuM7Sg+kWBPSxof3Zit8Oy5PeEgcPDLzoqS5VNrCKvVxrLmt+Mugqc+kAzo8nyF3B7SNppW6dcyaET0YZUyEaYs/Wc546qMwZFtrAK5TYDsFSiygwCWik4LVR+rANhsHRT6uHntbRNrlngzuG5aU5Rs9/jXns0Nbw4s6dzZ8oP1r5Xew0POglqiR5z4eRilq4R4kXQswJjxd0lCNCZncjDdwFwZPdJvvr2cH52/pY0y5Xdif907ad5P6KQCl6YbzAmQcacchv2275hBxfPh20L3Ca4xh8RVflTqaLKJJugCtJy5VbOg4fAbdjDFDG5ZoE7h+ec1DfvoanBdDi3Ygwu3dTCzCwTk1JY9+ovG+WYVqxKJG5hFUxLaqEvAFFbqmwq2mKV6bfVcTnw6jG+sd967zPfjF65ZiInTp2OsmpZMcVPGFYr51qEQy+xothfPL/8/BW8YlV94sTzIqo8iEtk9aDIx0oV9cNn3mjD56k1u+9q3aDz8zt5lRXv5sFD7ru9LFNmOS1HjuVXZhmxn7mcVXtrmqO6/T0ORO8LWBwVbbE6r1f6W04h02QjmVobEq0vv8/d74/mwd7NAKy56nI6lZ9+CMRqlc1J/al1V3DjpS9y8XxdNparbDMA3RBRlZ+zepwFWOohNAbITzAFJa5yPwjNCOw1wB7IEtIgkzyvgzN7gIeognTLVPowZKqcJYN6wKDKuf+XC2YYhnyuzzDiYhVKbCxWqzICx3nN7CgdlSSqADhNUlQBjH/+pQgrkxu3TmILqmJm79nHOq1VAN952Joq9NS6K4DysFwFEasqjpRqaM6vqIqbD4zTOrX19vWxebBksN+IS1jXN/TiRvbyiiNoCaq6/qkIzqsqfMivHMl3gkXcLFyxEVYZVPUq+hR1vQo/R9ME9wWYyxWt4dlpOxOfNdYEzmBmP4aNl4gqRFxlO2b1uTu4csGLye/lIK7ytVS5EUdrVdiiyq9fVVwtVfn4UMWB+uaUv2nT8GrfTuJBk1yGzAjDs6y5dPdAe3mpXIFu/eSpdPyKq7j4VZnERlg5o+ROv9CPw2F2Wo8UEWS0Z/HlxwWt4bfXp0RVKj3+4sruLE5fK9Ny5UdgmfnM481z/2H+FQBlI67cbrzl7qz+XtvjvNf2eKjl+BVLZr44/D42w348Jvk5qpl+xWL6V4XuJG7MDJzXL5hT3re/MP8r8zryuqaS4W9CvOac9464LuydS1zFUVRBjISVkyA6wJ+POIJ7GsvZuG02TmvVrC27PZfCiTtaa1dRZe6H8hNXkDmU5yawnOlO3yznEGC5iSsofmHlOGCLKpsp0xeFUo5fv6q4iiqwhvxs4hH0swAM/6owwtLYkbohPdxDPszadczzWbGs+ZWC65YtjpWdXoprLm7XtRde4iquogpiLKwipWf6DXdTa2EdM3o0v73+LeuTi6hK5ipzceUmsNwElVNUXdvtU0BKVNmUg7hyC6vg5Tvlts+0VEV9g3WKKpugxVUliCob58OkLMVVEj9hafK0EB1KjVZ4zQwEN2tZqpwhnn5axWP3u3KxHEWNU1zFWVRB3IVVVXF+TpkO8X5JdahyjWXSvboTz07NLapsTHEV13lwprjyElhem4l5/Opzd7iWFWdxlU+sKrd9cRv+M3l22T2cVXtr8ntQ4qqSRJWNcxhw2I/HlLnAykKONVoz1/fzFmtmfMNlOxzrExrlfDmgYcNsmOKqVKLKWU65+HI5xVVcRRXEPNxC07iB1De6W4uaGibmtLAsA+YVWYdyjGWiOsEzl1u/mx9RZTNixAg2b97M6MZVrG+YGEuBZQYOdZvdlw2nGPvD/Cu4csGLXLngxaSQyrU/6lAMXrGqIPssv7jOAPRaaums2ltD8bWqFFFlYj9ghtQezBBXuR4+5SLEWvfto26Qt9/rZ/MIBp0W39C5PuGRI5CI0O57acIAcBM7pbwGy+l6h3iLKvC3CHOdUuqPSqk3lFJblFJzEunVSqkXlFLbE3/PTqQrpdRDSqkWpdQmpdQlvmtz1N/q5k0TRuTO5EaRFrByoFMXxX99ztunKhubNm1i7Fhr/awxjauIx6BXJs43dbehP7d92zfsYPuGHXzn4bqM4b9cmJYrm1JbroIIqxAna5WNU0DZ4se2XBVrtcp3Zl+cfpt8yWeIsNSiqnXXxoy0XEvZ2Ezbmu4vu9gRCmfmuNR3t3OaZZtrDjrXJzRnL1pkRnyfXtXHNb0QbAFVqmuu3Icay2Wyhh+L1Ungbq31BqXUmUCzUuoF4KvA77XWC5VSc4G5wD3ANcDQxHYp8HDib07qV7fQ1JD+VtLUMJH7drWzbOtmRlb1Ycm4UR5H58aygO3D2ZnseFVuFrAFg3owbauZd2xGnrjQqZviN9dYQ1v5iiqbdevWAaCUYkzjKtZfPREVR9MV7k6N2WYIbluguHi+5sZLX0wO6/m1WgHJPPaxpbRcVaqoMv2rnl12TzLdFlc33mlZrmxxZebxQ75hFeL02xRKvv4opXpQzd96mCWD0tNyLWXjxXhgZFUfNh06nHFPntOY6QzvVrYfmhpGsXRXOw86nj/LtrTnjBZfDLYlN8jr0e185S604kpOYaW13kNiwFpr/Vel1BvAQGAqcEUi2xPAi1jCairwpLae7GuVUlVKqZrEeQpi3qBq5hUcAdeM+AvLG0Yxf+1GNh06zPSqPszLIdRMu4Y1jh+eQ2MxdD5DseLq4kSVidbaElcr4y2uIL8HQ1DiyiRscZWvqHIjjqIKcBVVQZFvWIW4/TbFEjeH301uiXmIkzVbVqVZm7xfsjMfNa5le9C6ayN1g1LnnjmompkhRWDPJqCCFFduw+2l9K2K2i/Vi7Du23n5WCmlBgGjgHXAAFssaa33KKXsULYDAdNtvC2R5ktYOS/qYpnVuJklhtm4jmwdMsH+N9OmA8c9AnuXXopfTwpOVNmY4uqVqyfQKc7qKg8KFVdAWp4oLFfgb6kaiGdYBROv2YAmTz1UnbRagTUs6EeEFeKsXql4CaxyY04rNA0Pv5xCrVvFEKa48rJU2ecPG1tUxW15sIvn69Du275nBSqlegPPAHdprQ9ny+qSlvG0V0rdppRqUkqleeE6x9KLxXpTye2AbirBzHF2J/FxaNenCUVU2cyday3KOXbl6sDPHSV2Z7rx0heTaW4zAaMmnxmAJm6iKm4WGb/O6U89VJ02UzAfKtFZvVDi4Z8S7r1z1q7iz+/LuvXX4P6PXNeeLa7ytTDZx+QKRBomcRVVkKpTGNY0X8JKKdUVS1Q9pbW2g6TvVUrVJPbXAPb0ijbSR9BqgXec59RaP6a1rtdaZ75O55hamy/3rc0dzO0/yjCsQvezO/PbaYU5qvvl/vvv54EHHgAsH7TTIZUTBYWIq1KGYch3BqDX8GAcYlVlw48FyhRXU6YvyurQ7sevqqOJKpMoxdWaLbnuxdkFi+UH656nvnEVm7b6D9yZLZTOrFwx/Q4FG9swl3gy415ly2fu9+r3pfariqOosglLXKlcD2SllMLyoWrXWt9lpD8AHDSc16u11v+glLoWuAOYjOW0/pDWOqvHt1IqoxLLR9dQ57H6eetRmLY6deHfXZWaFdIKTHPpFE0TRmQE/rSZtaWdTS5j/W5DgPWNG2lqGIbta2U6vJv1cO5zoxiT/Bl9O/Or8duB8ESVyU9/+lNuueUWANZdPYHORQwLxi0Qqd2pbIEEKeHkNizotd8UaMXeTMwbp9NaVc7O6jb9WJj8nI9/1Y13tmf1y/IzBBg3UZXPsEy2vB3JEdn0j12zv505zeE4ki8eVsP4/nXQswetR48xbXXhEdf9UMi1YJLruFJfI1GFpcmHIqxqza6GIfwJq08BLwGbgdOJ5HlYfla/BM4D/gx8XmvdnhBi/w40AEeBr2mtM1+508vwrMTi0SMY38+6Qa7Zf4w5RSwlMBJYMGEEdQmBZc/2yMbyRP5iy3ajUGHV89yu/PLSbUBpRJXNM888ww033AAUJ67iJqwgeHFVzM2kUmcA2hQqqmy8xFU5iioQYSWkE5b/UxTXhy2sonZez3Y/jkRYlYJswqqSKURY9a7tyi9Gl15UmaiEoFp71QS6dMpfQMRRWEE8xFWliyooXlhBprgqV1EFIqwEd4ISWKV0VHdiCqqoLFe5hFMYwireS9oIafQZ3C1yUWWWPe751Zw8XTmauFifKzuP6XOVz5uam2m/UFFVLhQa/NMUT+Y5yk1UCYIX5mLNhYRGyOVrVSq2LVCRDgeG6aTuhQirMuH0Sc3PR1qRSuNgZdy82RpCHfe8zBZ0s2bZ4qqQG4rbDMB8ibN42M/ctO9BrQcookqoNExRlMtx3ZknakEVJ0ot7ERYlQGnT2ie+9vUgsp33XUH3/rWFwo+n1LnopRKDukVwsc+9jF27rRmF8V1aK9YihVXfjFvln4XVnZaq5qfrI/9EGA2cs30y4Wf0A3l+LsIAqQEltvsQDcxJdd6tMR6EWYB+n7iDH52/uvJ76YY+uEPrc9a7wf6lrpqDB48mL179zJgwADqG1fx8lWX061T+Wt103TsN4ioU3CZoiwbtqC68eaHANh64rK0/fkurFwuN9T9zE3ztbLxGwDUJtdizZUaVV3o2FTa9RxG0Noow4qIsIox/cf0ZMlHXstI//GCLzHgiq9zw0QrtMNL//BtLv/XR0pdPQD69+/P4cOH6dOnD5c9/1LFiCvIP0J7MVYrm2FdX7Y+nHDfX47O6n54dtk9SYvVlOmLOKv21qxxqPxYt0RUCUK8MQVVkEJoyooLmLLigsjElQirmFLzyZ483j9TVJn+Vdl8rYbPns0Z6zew+t9/RK9P+loDO8m8O+/kd2vX8n/mzWf49ddmzTvyhut49f7bOH78ON27d+ey519izVWX071CxJWNKa5sckVoz2dc37RUDev6clJgmemVJqpMq9WNd7YnLVVTpi9KLLzsPmvQFlW2+JoyPfPc5bhUTT4OyqVc500QwiDMtSvNZZyiEFeV9fSrEAZ+qndSVK01xNPTj96R81jbd+r1Rx9lw4b19L5sXGL4cLvvY+//0Y9Yv349H5s2BaUULz6RGrJZusfKN7ehN0opNj/zX3S+6HN069YtKfTGP/8SH5w67VVMWWGKI3t47w/zr8hqrfI7DOiFU2RB5YkqG9uR3Ws4rxC/K3FWFwQhSkRYxYxTx0/z6DnWalVaa759/cPJfV+47d6sx5r+V82vP0Pb228Z+9yj2Nv0HJTaP30w7NmTWi/x01/9x4z8i1YecT2PLa4+9cJLWcsrJ9zEFaQEVhBDgE5G3XoXo261FjqwxVU2f6tKwBZRTivVlOmLXC1QbkOF5SiqvByOnQ7L5n63feXy/wpCpSNDgTHi1Aen+e8Zu4CUQHnjvZVGjnN9ncc5XOhn9t+xty2L1um316HOG5tx7I7/+ygXfv7racc0f/cWLvmnTEuDfVx94yrXZYHKET+xUApd0sYMBjrzm6nVn5b+5BXgMoZ1fZkv8h3oalmz4rywciGYQ4K287o5LAgkhwZtci1nU66/i1vsonL9XwShoyLCKiac/OA0jQ5RlQ8r3sqdh6fHwozsy/LYoipFd+A4v3vunzOElZuosqlEcWWTa6iv0JgpmaLKYuuJy5JWq2FdX6aZ8h8C9Ivp1G7ilmYPJzrjZJUj9pR6kIjqglBuiLCKARdffTYPntEMZIqq0WfNZDcrrC/bfgEXf9H1HIda/5KznFd+/RZjZxRWxx1N+/I+xhRXqyeNp2eX8r/c7JmCQQac8xJUkPKjssVUpeJ0ZM82IzDXeSqFYiNvC0IciPNCzEUsZ5OV8m0ya7QAAAW7SURBVH/SlTl/M7maB7paQ0FulqoVv56BUpaYUsNmorW7sDrvo2fnLGvs0hcKqOFxACbdkeln5QdbXE343RpWTRpPrwoRV0HgfFj+/bjT4CGgOoLFwhZXzmE/r7wdBdN6JQjlSBTiygzl4ObCEZaoAlmEOVKuXjqIlTN3AdmH/5T6OGA5tD//o3/js3fMSe3rcik/uWUg33hkWdIfSq+9Di61rFyb22BknR1IVCfOdy6w15Fm5Vn3g56M/Z8px/TkORP5lu6BL30kPS0Xu3fvpra2FoCmhokVG6k9X9welh1BQOXCLXCoTUcSVE7M60WuE6FcyOaXOvSSC0Mte/uGHVn3FymqPBdhRmsd+Qbojrz5Idvxa//p64k8N3vmmVZbY5xrQEbZQ4f+p+exQ418P39H51Vvv/+DbLLZWz8W6n4sjLwesskmm2xZtiYvTSMWq4iJw+9fSopZn1AQBEEQYoKnxSouDi/vA9uirkQUlIHQ6AsciLoSgivSNvFF2ia+SNvEl3Jqm/O9dsRFWG3zUn5CtCilmqRt4om0TXyRtokv0jbxpVLaRiKvC4IgCIIgBIQIK0EQBEEQhICIi7B6LOoKCJ5I28QXaZv4Im0TX6Rt4ktFtE0sZgUKgiAIgiBUAnGxWAmCIAiCIJQ9IqwEQRAEQRACInJhpZRqUEptU0q1KKU67noVEaKU2qWU2qyUelUp1ZRIq1ZKvaCU2p74e3YiXSmlHkq01yal1CXR1r6yUEotUUrtU0q9ZqTl3RZKqZsS+bcrpW6K4n+pNDza5jtKqd2JvvOqUmqyse8fE22zTSl1tZEu97wAUUrVKaX+qJR6Qym1RSk1J5Eu/SZisrRNZfcbr5DspdiAzsAO4AKgG/An4KNR1qkjbsAuoK8j7V+BuYnPc4FFic+Tgf8GFDAOWBd1/StpAyYAlwCvFdoWQDWwM/H37MTns6P+38p982ib7wB/75L3o4n7WXdgcOI+11nueaG0Sw1wSeLzmcCbid9f+k1826ai+03UFquxQIvWeqfW+kPgF8DUiOskWEwFnkh8fgK43kh/UlusBaqUUjVRVLAS0VqvBtodyfm2xdXAC1rrdq31X4AXgIbwa1/ZeLSNF1OBX2itj2ut3wJasO53cs8LGK31Hq31hsTnvwJvAAORfhM5WdrGi4roN1ELq4FAq/G9jew/uhAOGnheKdWslLotkTZAa70HrM4B9E+kS5uVnnzbQtqotNyRGFJaYg83IW0TCUqpQcAoYB3Sb2KFo22ggvtN1MLKbaE8if9QesZrrS8BrgFuV0pNyJJX2iw+eLWFtFHpeBi4EPgEsAd4MJEubVNilFK9gWeAu7TWh7NldUmTtgkRl7ap6H4TtbBqA+qM77XAOxHVpcOitX4n8XcfsBzL7LrXHuJL/N2XyC5tVnrybQtpoxKhtd6rtT6ltT4NPI7Vd0DapqQopbpiPbif0lovSyRLv4kBbm1T6f0mamG1HhiqlBqslOoGfBH4TcR16lAopXoppc60PwNXAa9htYM9K+YmYEXi82+AryRm1owD3rPN7UJo5NsWK4GrlFJnJ0zsVyXShIBx+BdOw+o7YLXNF5VS3ZVSg4GhwCvIPS9wlFIK+Cnwhtb6B8Yu6TcR49U2ld5vukRZuNb6pFLqDqyLtzOwRGu9Jco6dUAGAMut658uwM+11o1KqfXAL5VSNwN/Bj6fyP8c1qyaFuAo8LXSV7lyUUotBa4A+iql2oD5wELyaAutdbtS6l+wbkYA39Va+3W6FjzwaJsrlFKfwBqW2AV8HUBrvUUp9UvgdeAkcLvW+lTiPHLPC5bxwJeBzUqpVxNp85B+Ewe82mZmJfcbWdJGEARBEAQhIKIeChQEQRAEQagYRFgJgiAIgiAEhAgrQRAEQRCEgBBhJQiCIAiCEBAirARBEARBEAJChJUgCIIgCEJAiLASBEEQBEEIiP8PnJDi9o4rFXIAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 720x720 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAOcAAAJCCAYAAAA7utBUAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAVZklEQVR4nO3af4xd5Z3f8fcndgwhCTE/AnVtt4BipYmqhpAR6zRttYWkDW618AdIRKtiIUuutmmblEpdp5XartQ/kqoKLVLFrrWkdVYpC5tNhLWlmyID6q60kJiEEIiX9YR28dSsveGHWRVlWeDbP+ZxMmsP9h17hvlm5v2SRvec5zz3znNGvH3OvdxUFZL6edtyL0DS/IxTaso4paaMU2rKOKWmjFNqakniTPLJJE8nmU6yayl+h7TSZbH/P2eSNcAfAJ8AZoBvAZ+qqu8v6i+SVriluHJeDUxX1TNV9Srw68D1S/B7pBVt7RK85kbg0Jz9GeBnTvWEi5P6y0uwEGk5/SHww6qc6fOXIs75FnPSvXOSncBOgL8EPLIEC5GW09azfP5S3NbOAJvn7G8CDp84qap2V9VUVU1dvASLkH7aLUWc3wK2JLk8yTrgZmDvEvweaUVb9NvaqnotyT8GvgGsAb5UVU8t9u+RVrqleM9JVd0P3L8Ury2tFn5DSGrKOKWmjFNqyjilpoxTaso4paaMU2rKOKWmjFNqyjilpoxTaso4paaMU2rKOKWmjFNqyjilpoxTaso4paaMU2rKOKWmjFNqyjilpoxTaso4paaMU2rKOKWmjFNqyjilpoxTaso4paaMU2rKOKWmjFNqyjilpoxTaso4paaMU2rKOKWmjFNqyjilpoxTaso4paaMU2rKOKWmjFNqyjilpoxTaso4paaMU2rKOKWmjFNqyjilpk4bZ5IvJTma5Mk5YxcmeSDJwfF4wRhPkjuSTCd5IslVS7l4aSWb5Mr5X4FPnjC2C9hXVVuAfWMf4Dpgy/jZCdy5OMuUVp/TxllV/wt44YTh64E9Y3sPcMOc8S/XrEeA9Uk2LNZipdXkTN9zXlpVzwGMx0vG+Ebg0Jx5M2PsJEl2JtmfZP8Pz3AR0kq22B8IZZ6xmm9iVe2uqqmqmrp4kRchrQRnGueR47er4/HoGJ8BNs+Ztwk4fObLk1avM41zL7B9bG8H7pszfsv41HYrcOz47a+khVl7uglJ7gZ+Frg4yQzwb4DPA/cm2QE8C9w0pt8PbAOmgVeAW5dgzdKqkKp53xK+pT6S1CPLvQhpkW0FHqua73OYifgNIakp45SaMk6pKeOUmjJOqSnjlJoyTqkp45SaMk6pKeOUmjJOqSnjlJoyTqkp45SaMk6pKeOUmjJOqSnjlJoyTqkp45SaMk6pKeOUmjJOqSnjlJoyTqkp45SaMk6pKeOUmjJOqSnjlJoyTqkp45SaMk6pKeOUmjJOqSnjlJoyTqkp45SaMk6pKeOUmjJOqSnjlJoyTqkp45SaMk6pKeOUmjJOqSnjlJoyTqkp45SaMk6pqdPGmWRzkoeSHEjyVJLPjPELkzyQ5OB4vGCMJ8kdSaaTPJHkqqU+CWklmuTK+Rrwz6vqA8BW4NNJPgjsAvZV1RZg39gHuA7YMn52Ancu+qqlVeC0cVbVc1X17bH9J8ABYCNwPbBnTNsD3DC2rwe+XLMeAdYn2bDoK5dWuAW950xyGfBh4FHg0qp6DmYDBi4Z0zYCh+Y8bWaMSVqAieNM8i7gN4HPVtXLp5o6z1jN83o7k+xPsv+Hky5CWkUmijPJ25kN8ytV9bUxfOT47ep4PDrGZ4DNc56+CTh84mtW1e6qmqqqqYvPdPXSCjbJp7UB7gIOVNUX5xzaC2wf29uB++aM3zI+td0KHDt++ytpcmsnmPMx4B8A30vy+Bj7l8DngXuT7ACeBW4ax+4HtgHTwCvArYu6YmmVSNVJbwffch9J6pHlXoS0yLYCj1XN9xnMRPyGkNSUcUpNGafUlHFKTRmn1JRxSk0Zp9SUcUpNGafUlHFKTRmn1JRxSk0Zp9SUcUpNGafUlHFKTRmn1JRxSk0Zp9SUcUpNGafUlHFKTRmn1JRxSk0Zp9SUcUpNGafUlHFKTRmn1JRxSk0Zp9SUcUpNGafUlHFKTRmn1JRxSk0Zp9SUcUpNGafUlHFKTRmn1JRxSk0Zp9SUcUpNGafUlHFKTRmn1JRxSk0Zp9SUcUpNGafUlHFKTZ02ziTnJvlmku8meSrJL43xy5M8muRgknuSrBvj54z96XH8sqU9BWllmuTK+afANVX1IeBK4JNJtgJfAG6vqi3Ai8COMX8H8GJVvQ+4fcyTtECpqsknJ+cBvwv8AvDfgb9QVa8l+Sjwb6vq7yb5xtj+vSRrgT8C3lun+EXr33dO/c0vbjqrE5G6+Z3bZnhp+k9zps+f6D1nkjVJHgeOAg8APwBeqqrXxpQZYOPY3ggcAhjHjwEXzfOaO5PsT7L/1ZffONP1SyvWRHFW1etVdSWwCbga+MB808bjfP9SnHTVrKrdVTVVVVPrzvdzKelEC6qiql4CHga2AuvHbSvMRnt4bM8AmwHG8fcALyzGYqXVZJJPa9+bZP3YfgfwceAA8BBw45i2HbhvbO8d+4zjD57q/aak+a09/RQ2AHuSrGE25nur6reSfB/49ST/DvgOcNeYfxfwa0mmmb1i3rwE65ZWvNPGWVVPAB+eZ/wZZt9/njj+I+CmRVmdtIr5SYzUlHFKTRmn1JRxSk0Zp9SUcUpNGafUlHFKTRmn1JRxSk0Zp9SUcUpNGafUlHFKTRmn1JRxSk0Zp9SUcUpNGafUlHFKTRmn1JRxSk0Zp9SUcUpNGafUlHFKTRmn1JRxSk0Zp9SUcUpNGafUlHFKTRmn1JRxSk0Zp9SUcUpNGafUlHFKTRmn1JRxSk0Zp9SUcUpNGafUlHFKTRmn1JRxSk0Zp9SUcUpNGafUlHFKTRmn1JRxSk1NHGeSNUm+k+S3xv7lSR5NcjDJPUnWjfFzxv70OH7Z0ixdWtkWcuX8DHBgzv4XgNuragvwIrBjjO8AXqyq9wG3j3mSFmiiOJNsAv4e8KtjP8A1wFfHlD3ADWP7+rHPOH7tmC9pASa9cv5H4F8Ab4z9i4CXquq1sT8DbBzbG4FDAOP4sTH/z0myM8n+JPtfffmNEw9Lq95p40zy94GjVfXY3OF5ptYEx34yULW7qqaqamrd+X4uJZ1o7QRzPgb8XJJtwLnA+cxeSdcnWTuujpuAw2P+DLAZmEmyFngP8MKir1xa4U57yaqqz1XVpqq6DLgZeLCqfh54CLhxTNsO3De29459xvEHq+qkK6ekUzub+8lfBG5LMs3se8q7xvhdwEVj/DZg19ktUVqdJrmt/bGqehh4eGw/A1w9z5wfATctwtqkVc1PYqSmjFNqyjilpoxTaso4paaMU2rKOKWmjFNqyjilpoxTaso4paaMU2rKOKWmjFNqyjilpoxTaso4paaMU2rKOKWmjFNqyjilpoxTaso4paaMU2rKOKWmjFNqyjilpoxTaso4paaMU2rKOKWmjFNqyjilpoxTaso4paaMU2rKOKWmjFNqyjilpoxTaso4paaMU2rKOKWmjFNqyjilpoxTaso4paaMU2rKOKWmjFNqyjilpiaKM8n/SfK9JI8n2T/GLkzyQJKD4/GCMZ4kdySZTvJEkquW8gSklWohV86/XVVXVtXU2N8F7KuqLcC+sQ9wHbBl/OwE7lysxUqrydnc1l4P7Bnbe4Ab5ox/uWY9AqxPsuEsfo+0Kk0aZwH/M8ljSXaOsUur6jmA8XjJGN8IHJrz3JkxJmkB1k4472NVdTjJJcADSX7/FHMzz1idNGk28p0A73jvpMuQVo+JrpxVdXg8HgW+DlwNHDl+uzoej47pM8DmOU/fBBye5zV3V9VUVU2tO98PjaUTnbaKJO9M8u7j28DfAZ4E9gLbx7TtwH1jey9wy/jUditw7Pjtr6TJTXI/eSnw9STH5/+3qvrtJN8C7k2yA3gWuGnMvx/YBkwDrwC3LvqqpVXgtHFW1TPAh+YZfx64dp7xAj69KKuTVjHf7ElNGafUlHFKTRmn1JRxSk0Zp9SUcUpNGafUlHFKTRmn1JRxSk0Zp9SUcUpNGafUlHFKTRmn1JRxSk0Zp9SUcUpNGafUlHFKTRmn1JRxSk0Zp9SUcUpNGafUlHFKTRmn1JRxSk0Zp9SUcUpNGafUlHFKTRmn1JRxSk0Zp9SUcUpNGafUlHFKTRmn1JRxSk0Zp9SUcUpNGafUlHFKTRmn1JRxSk0Zp9SUcUpNGafUlHFKTRmn1NREcSZZn+SrSX4/yYEkH01yYZIHkhwcjxeMuUlyR5LpJE8kuWppT0FamSa9cv4n4Ler6q8AHwIOALuAfVW1Bdg39gGuA7aMn53AnYu6YmmVSFWdekJyPvBd4IqaMznJ08DPVtVzSTYAD1fV+5P8yti++8R5b/Y7zl13ZW2+9MFFOB2pj0NHruFHrz6eM33+JFfOK4A/Bv5Lku8k+dUk7wQuPR7ceLxkzN8IHJrz/Jkx9uck2Zlkf5L9r7/x/JmuX1qxJolzLXAVcGdVfRj4f/zkFnY+8/1LcdLluap2V9VUVU2tedtFEy1WWk0miXMGmKmqR8f+V5mN9ci4nWU8Hp0zf/Oc528CDi/OcqXV47RxVtUfAYeSvH8MXQt8H9gLbB9j24H7xvZe4Jbxqe1W4Nip3m9Kmt/aCef9E+ArSdYBzwC3Mhv2vUl2AM8CN4259wPbgGnglTFX0gJNFGdVPQ5MzXPo2nnmFvDps1yXtOr5DSGpKeOUmjJOqSnjlJoyTqkp45SaMk6pKeOUmjJOqSnjlJoyTqkp45SaMk6pKeOUmjJOqSnjlJoyTqkp45SaMk6pKeOUmjJOqSnjlJoyTqkp45SaMk6pKeOUmjJOqSnjlJoyTqkp45SaMk6pKeOUmjJOqSnjlJoyTqkp45SaMk6pKeOUmjJOqSnjlJoyTqkp45SaMk6pKeOUmjJOqSnjlJoyTqkp45SaMk6pKeOUmjJOqanTxpnk/Uken/PzcpLPJrkwyQNJDo7HC8b8JLkjyXSSJ5JctfSnIa08p42zqp6uqiur6krgI8ArwNeBXcC+qtoC7Bv7ANcBW8bPTuDOpVi4tNIt9Lb2WuAHVfWHwPXAnjG+B7hhbF8PfLlmPQKsT7JhUVYrrSILjfNm4O6xfWlVPQcwHi8Z4xuBQ3OeMzPGJC3AxHEmWQf8HPAbp5s6z1jN83o7k+xPsv/1N56fdBnSqrGQK+d1wLer6sjYP3L8dnU8Hh3jM8DmOc/bBBw+8cWqandVTVXV1Jq3XbTwlUsr3ELi/BQ/uaUF2AtsH9vbgfvmjN8yPrXdChw7fvsraXJrJ5mU5DzgE8A/nDP8eeDeJDuAZ4Gbxvj9wDZgmtlPdm9dtNVKq8hEcVbVK8BFJ4w9z+yntyfOLeDTi7I6aRXzG0JSU8YpNWWcUlPGKTVlnFJTxik1ZZxSU8YpNWWcUlPGKTVlnFJTxik1ZZxSU8YpNWWcUlPGKTVlnFJTxik1ZZxSU8YpNWWcUlPGKTVlnFJTxik1ZZxSU8YpNWWcUlPGKTVlnFJTxik1ZZxSU8YpNWWcUlPGKTVlnFJTxik1ZZxSU8YpNWWcUlPGKTVlnFJTxik1ZZxSU8YpNWWcUlPGKTVlnFJTxik1ZZxSU8YpNWWcUlPGKTU1UZxJ/lmSp5I8meTuJOcmuTzJo0kOJrknybox95yxPz2OX7aUJyCtVKeNM8lG4J8CU1X1V4E1wM3AF4Dbq2oL8CKwYzxlB/BiVb0PuH3Mk7RAk97WrgXekWQtcB7wHHAN8NVxfA9ww9i+fuwzjl+bJIuzXGn1OG2cVfV/gf8APMtslMeAx4CXquq1MW0G2Di2NwKHxnNfG/MvOvF1k+xMsj/J/tffeP5sz0NacSa5rb2A2avh5cBfBN4JXDfP1Dr+lFMc+8lA1e6qmqqqqTVvO6ldadWb5Lb248D/rqo/rqo/A74G/HVg/bjNBdgEHB7bM8BmgHH8PcALi7pqaRWYJM5nga1JzhvvHa8Fvg88BNw45mwH7hvbe8c+4/iDVXXSlVPSqU3ynvNRZj/Y+TbwvfGc3cAvArclmWb2PeVd4yl3AReN8duAXUuwbmnFS4eL2rnrrqzNlz643MuQFtWhI9fwo1cfP+P/U+E3hKSmjFNqyjilpoxTaso4paaMU2rKOKWmjFNqyjilpoxTaso4paaMU2rKOKWmjFNqyjilpoxTaso4paaMU2rKOKWmjFNqyjilpoxTaso4paaMU2rKOKWmjFNqyjilpoxTaso4paaMU2rKOKWmjFNqyjilpoxTaso4paaMU2rKOKWmjFNqyjilpoxTaso4paaMU2rKOKWmjFNqyjilpoxTaso4paaMU2rKOKWmjFNqyjilpoxTaso4paZSVcu9BpL8CfD0cq/jLXYx8MPlXsRbbLWd8/ur6t1n+uS1i7mSs/B0VU0t9yLeSkn2e84rW5L9Z/N8b2ulpoxTaqpLnLuXewHLwHNe+c7qfFt8ICTpZF2unJJOsOxxJvlkkqeTTCfZtdzrWSxJvpTkaJIn54xdmOSBJAfH4wVjPEnuGH+DJ5JctXwrPzNJNid5KMmBJE8l+cwYX8nnfG6Sbyb57jjnXxrjlyd5dJzzPUnWjfFzxv70OH7ZKX9BVS3bD7AG+AFwBbAO+C7wweVc0yKe298CrgKenDP274FdY3sX8IWxvQ34H0CArcCjy73+MzjfDcBVY/vdwB8AH1zh5xzgXWP77cCj41zuBW4e478M/MLY/kfAL4/tm4F7Tvn6y3xyHwW+MWf/c8DnlvuPvojnd9kJcT4NbBjbG5j9/7sAvwJ8ar55P60/wH3AJ1bLOQPnAd8GfobZL1qsHeM//m8c+Abw0bG9dszLm73mct/WbgQOzdmfGWMr1aVV9RzAeLxkjK+ov8O4Xfsws1eSFX3OSdYkeRw4CjzA7J3gS1X12pgy97x+fM7j+DHgojd77eWOM/OMrcaPj1fM3yHJu4DfBD5bVS+fauo8Yz9151xVr1fVlcAm4GrgA/NNG48LOufljnMG2DxnfxNweJnW8lY4kmQDwHg8OsZXxN8hyduZDfMrVfW1Mbyiz/m4qnoJeJjZ95zrkxz/auzc8/rxOY/j7wFeeLPXXO44vwVsGZ9urWP2TfLeZV7TUtoLbB/b25l9X3Z8/JbxCeZW4NjxW8GfFkkC3AUcqKovzjm0ks/5vUnWj+13AB8HDgAPATeOaSee8/G/xY3AgzXegM6rwRvpbcx+svcD4F8t93oW8bzuBp4D/ozZfzF3MPv+Yh9wcDxeOOYG+M/jb/A9YGq5138G5/s3mL1FewJ4fPxsW+Hn/NeA74xzfhL412P8CuCbwDTwG8A5Y/zcsT89jl9xqtf3G0JSU8t9WyvpTRin1JRxSk0Zp9SUcUpNGafUlHFKTRmn1NT/B+eEQoLdsw6fAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 720x720 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import os\n",
    "import numpy as np\n",
    "import cv2\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "def show(img, figsize=(10, 10)):\n",
    "    figure = plt.figure(figsize=figsize)\n",
    "    plt.imshow(img)\n",
    "    plt.show()\n",
    "\n",
    "# read icons\n",
    "icon_dir = \"icons/\"\n",
    "icons = []\n",
    "for image_name in os.listdir(icon_dir):\n",
    "    icon = cv2.imread(icon_dir+image_name)\n",
    "    icons.append(icon)\n",
    "\n",
    "icons = np.array(icons)\n",
    "# stack it horizontally\n",
    "icons_holder = np.hstack(icons)\n",
    "show(icons_holder)\n",
    "\n",
    "# read colors\n",
    "color_dir = \"colors/\"\n",
    "colors = []\n",
    "for color_name in os.listdir(color_dir):\n",
    "    color = cv2.imread(color_dir+color_name)\n",
    "    colors.append(color)\n",
    "colors = np.array(colors)\n",
    "# stack it vertically\n",
    "colors_holder = np.vstack(colors)\n",
    "show(colors_holder)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Attach colors image on VUI\n",
    "I am using the Icons panel or VUI ROI above the Draw ROI. Also i have modified little bit of `run_system` method.\n",
    "```python\n",
    "def run_system(count_mode=5):\n",
    "    #accumulate weight variable\n",
    "    aweight = 0.5\n",
    "    # strat the camera\n",
    "    cam = cv2.VideoCapture(0)\n",
    "    # ROI box\n",
    "    top, right, bottom, left = 250, 400, 480, 681\n",
    "\n",
    "    # ROI for GUI\n",
    "    gtop, gright, gbottom, gleft = 150, 400, 240, 681\n",
    "    gcolors = np.random.randint(0, 255, (9, 3), dtype=np.uint32).tolist()\n",
    "    gb_indices = int((gleft-gright)/9)\n",
    "    gb_indices = np.arange(gright, gleft, gb_indices)\n",
    "    gb_indices[-1] = gb_indices[-1]+1\n",
    "    # take each box's y coordinates\n",
    "    # read icons\n",
    "    icon_dir = \"icons/\"\n",
    "    icons = []\n",
    "    for image_name in os.listdir(icon_dir):\n",
    "        icons.append(cv2.imread(icon_dir+image_name))\n",
    "    icons = np.array(icons)\n",
    "\n",
    "    gui_boxes = [(gb_indices[i], gb_indices[i+1]) for i in range(len(gb_indices)-1)]\n",
    "    modes = os.listdir(\"icons/\")\n",
    "    modes = [mode.split(\".\")[0] for mode in modes]\n",
    "    gui_modes_position = {modes[i]:gui_boxes[i] for i in range(len(gb_indices)-1)}\n",
    "    gui_modes_icons = {modes[i]:icons[i] for i in range(len(gb_indices)-1)}\n",
    "    #print(gui_modes_position)\n",
    "    # read icons\n",
    "    icon_dir = \"icons/\"\n",
    "    icons = []\n",
    "    for image_name in os.listdir(icon_dir):\n",
    "        icon = cv2.imread(icon_dir+image_name)\n",
    "        icons.append(icon)\n",
    "\n",
    "    icons = np.array(icons)\n",
    "    # stack it horizontally\n",
    "    icons_holder = np.hstack(icons)\n",
    "    show(icons_holder)\n",
    "\n",
    "    # read colors\n",
    "    color_dir = \"colors/\"\n",
    "    colors = []\n",
    "    for color_name in os.listdir(color_dir):\n",
    "        color = cv2.imread(color_dir+color_name)\n",
    "        colors.append(color)\n",
    "    colors = np.array(colors)\n",
    "    # stack it vertically\n",
    "    colors_holder = np.vstack(colors)\n",
    "    show(colors_holder)\n",
    "\n",
    "    # count frame\n",
    "    num_frames=0\n",
    "    # writing canvas\n",
    "    canvas = None\n",
    "    # thickness\n",
    "    t=3\n",
    "    # draw color(ink color)\n",
    "    draw_color = (0, 255, 0)\n",
    "    # pointer color\n",
    "    pointer_color = (255, 0, 0)\n",
    "    # mode flag\n",
    "    erase = False\n",
    "    # flag to indicate take average\n",
    "    take_average=True\n",
    "    #bg image\n",
    "    draw_bg=None\n",
    "    gui_bg = None\n",
    "\n",
    "    previous_mode = \"move\"\n",
    "    running_mode = \"move\"\n",
    "    drawn = None\n",
    "    count_modes = 0\n",
    "    vui_canvas = None\n",
    "    current_color = (20, 100, 50)\n",
    "    colors_array = np.array([(255, 0, 0), (0, 255, 0), (0, 0, 255)]).tolist()\n",
    "    while True:\n",
    "        ####################################\n",
    "        ####################################\n",
    "```\n",
    "\n",
    "\n",
    "I am assuming boxes as modes icons region, and icon's index starts from 0. Just below the key checking codes below codes must be added.\n",
    "```python\n",
    "                    if chr(key) == \"v\" or running_mode == \"color\":\n",
    "                        # add color region on VUI\n",
    "                        # only use few portion of vui for dropdown\n",
    "                        xs = vui.shape[0]-300\n",
    "                        # what will be the length of each box on modes panel?\n",
    "                        # since our colors mode lies on 1st box\n",
    "                        ys = int(vshape[1]/len(icons)) * 2 - int(vshape[1]/len(icons))\n",
    "                        # length of each box\n",
    "                        c1 = int(vshape[1]/len(icons))\n",
    "                        # col. num. of 1st box's end\n",
    "                        c2 = c1 * 2\n",
    "                        # for every row below the modes region up to xs, for only 1st box \n",
    "                        vui[100:100+xs, c1:c2] = cv2.resize(colors_holder, (ys, xs))\n",
    "                        \n",
    "                        # we don't want yet to draw so we will move cursor instead\n",
    "                        erase = None\n",
    "                        \n",
    "                        # out of xs length on vui, how much region will a single color can hold?\n",
    "                        cboxes = int(xs/len(colors))\n",
    "                        \n",
    "                        # take parts of rows from the draw ROI because our ROI for color lies on Draw ROI part. \n",
    "                        # if gshape[0] rows contains xs parts for colors then what must contain for bottom-top?\n",
    "                        crb = int(xs/gshape[0] * (bottom-top))\n",
    "                        \n",
    "                        # divide that crb rows into num colors part\n",
    "                        cboxes = np.linspace(0, crb, len(colors)+1).astype(np.int64)\n",
    "                        # find each box's extreme columns on Draw ROI   \n",
    "                        cboxes = [(cboxes[i], cboxes[i+1]) for i in range(len(cboxes)-1)]\n",
    "                        \n",
    "                        # in which box is pointer now?\n",
    "                        # check the rows, on which rows among colors ROI is pointer now?\n",
    "                        cpointer = [cbox for cbox in cboxes if (cbox[0] < m[1] <= cbox[1])]\n",
    "                        \n",
    "                        # check the rows\n",
    "                        cdb = (left-right)/len(icons) # draw box, divide original col draw ROI into num icons pcs\n",
    "                        if  len(cpointer) > 0 and cdb<=m[0]<=cdb*2: # since color lies on 2nd box check if pointer is on that col\n",
    "                            current_color = colors_array[cboxes.index(cpointer[-1])]\n",
    "                            \n",
    "```\n",
    "\n",
    "And finally, and the codes where `erase==None` was present, it must be slightly modified.\n",
    "```python\n",
    "                    elif erase == None:\n",
    "                        canvas_shape = canvas.shape\n",
    "                        clone_shape = clone.shape\n",
    "                        eshape = (clone_shape[0]/canvas_shape[0], clone_shape[1]/canvas_shape[1])\n",
    "                        m[0] = int(eshape[1]*m[0])\n",
    "                        m[1] = int(eshape[0]*m[1])\n",
    "\n",
    "                        if drawn is None:\n",
    "                            e = cv2.erode(canvas, (3, 3), iterations=5)\n",
    "                            drawn = cv2.resize(e, (clone.shape[1], clone.shape[0]))\n",
    "\n",
    "                        drawn = cv2.resize(drawn, (clone.shape[1], clone.shape[0]))\n",
    "                        dc = drawn.copy()  \n",
    "                        cv2.circle(dc, (m[0], m[1]), 10, pointer_color, -3)\n",
    "                        drawn_new = dc\n",
    "                        new_canvas = np.vstack([vui[:100], drawn_new]).astype(np.uint8)\n",
    "                        if running_mode == \"color\":\n",
    "                            new_canvas[100:100+xs, c1:c2] = vui[100:100+xs, c1:c2]\n",
    "                        cv2.imshow(\"Drawing\", new_canvas)\n",
    "```\n",
    "\n",
    "#### What is happening?\n",
    "* Initially, i prepared icons for colors and also prepared image for it.\n",
    "* Then i added new variable `current_color`, which will be responsible for changing color.\n",
    "* Then i added new variable to store RGB values for 3 colors.\n",
    "* When pressing key `v` or going to the mode named `color`, we will change our running mode to `color`.\n",
    "* When running mode is `color` then, show the `vui` image with dropdown like colors panel (look at `erase==None`)\n",
    "* On running mode, we find the current position of pointer on ROI, then compare it with Drawing canvas. Because `vui` and drawing canvas are same sized.\n",
    "    * We found the exact column where a colors icon is and where it can locate to ROIs.\n",
    "    * Next we found where the each color's icon is and where it can locate to ROIs.\n",
    "    * When the current pointer is on the dropdown ROIs, then we check on which color's region is the pointer.\n",
    "    * Change `current_color` according to the pointer positon.\n",
    "\n",
    "The final code must show the result like below.\n",
    "<figure>\n",
    "<video src = \"Assets/dropdown.mp4\" width=\"100%\" controls autoplay loop> </video>\n",
    "<figcaption style = \"text-align:left; font-style:italic\">Dropdown working</figcaption>\n",
    "</figure> "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Adding Animation Effect\n",
    "The term animation will be nothing but a fake animation here. I have only one thing on mind to do as animation. I have never used Animation on OpenCV but used with Matplotlib many times. But here i have few ideas with performing animation like features.\n",
    "* Resize the portion/box which holds the icon. It makes look like scaled.\n",
    "* Rotate the portion/box which holds the icon. It makes look like spinning.\n",
    "* Change the color.\n",
    "\n",
    "Among these, easiest one is to chane the color. I will start with this one.\n",
    "\n",
    "### Animation Effect: Change Color\n",
    "One thing a clever man (performer or magician) always tries to take an attention of everyone by guessing what might surprise the audience. There is nothing like a magic but all is the since used to fool people. And here i will use the term Animation but everything i will be doing is change the color. Why does it looks like animating? Well lets see on the output. How to make a box/icon change its color when a pointer is above it? Lets write it on steps:-\n",
    "* Since we have 9 icons, we will have 9 boxes on both the ROI and VUI. So divide the VUI's icon panel on 9 boxes. (note that we need 10 cuts to make 9 sticks from 1)\n",
    "* We already have used the box from ROI to find the current mode. Now we need to find the index of current mode.\n",
    "* Since current mode and boxes are indexed in same order, we can use index of current mode to get index of box on VUI.\n",
    "* Using index of box, we will slice the VUI's icons panel to get exactly the portion of icon where pointer lies currently.\n",
    "* Now is the time to change color. Just add some color. Adding is most suitable here because we have black background image.\n",
    "\n",
    "#### Lets see it on code.\n",
    "Only few parts of code has to be added.\n",
    "```python\n",
    "                    ############################################\n",
    "                    ############################################\n",
    "        \n",
    "                    # check if the y axis lies on any of gui boxes\n",
    "                    current_box = [box for box in gui_boxes if box[0] <= m[0] <= box[1]][0] \n",
    "                    current_mode = [mode for mode, pos in gui_modes_position.items() if current_box==pos][0]\n",
    "                    \n",
    "                    ####### Look from here ########\n",
    "                    ## Change color on current box\n",
    "                    ccolor = np.array([[10, 10, 50]]).astype(np.uint8)\n",
    "                    if current_mode == previous_mode:\n",
    "                        count_modes += 1\n",
    "                        # change color everytime, i want to make it more red\n",
    "                        ccolor = count_modes * np.array([[10, 10, 50]]).astype(np.uint8)\n",
    "                    if count_modes > count_mode:\n",
    "                        running_mode = current_mode\n",
    "                    previous_mode = current_mode\n",
    "                    \n",
    "                    # to make animate like features, we will have to find part where cursor is on canvas\n",
    "                    # index of current mode\n",
    "                    mind = modes.index(current_mode)\n",
    "                    # to make 9 boxes, we need 10 lines on left and right\n",
    "                    vboxes = np.linspace(0, vui.shape[1], len(icons)+1).astype(np.int64)\n",
    "                    vboxes = [(vboxes[i], vboxes[i+1]) for i in range(len(icons))]\n",
    "                    vbox = vboxes[mind]\n",
    "                    vui_canvas_anim = vui_canvas.copy()\n",
    "                    # increase brightness? instead of original vui_canvas, use its copy to edit \n",
    "                    vui_canvas_anim[:, vbox[0]:vbox[1]] += ccolor\n",
    "                    \n",
    "                    if vui_canvas is not None:\n",
    "                        gshape = gray.shape\n",
    "                        dummy = np.zeros_like(clone)\n",
    "                        dummy_copy = dummy.copy()\n",
    "                        dummy[:100, :] += vui_canvas_anim \n",
    "                        ##### above here #########\n",
    "                        cv2.circle(dummy_copy, (m[0], m[1]), 10, (250, 250, 150), -3)\n",
    "                        d = dummy_copy[gtop:gbottom, gright:gleft]                                                \n",
    "                        d = cv2.resize(d, (gshape[1], vshape[0]))                        \n",
    "                        dummy_temp = dummy[:100, :]\n",
    "                        dummy_temp[d!=[0, 0, 0]] = 100\n",
    "                        dummy[:100, :] += dummy_temp\n",
    "                        vui = dummy\n",
    "                       \n",
    "                    cv2.circle(clone, (m[0], m[1]), 10, pointer_color, -3)\n",
    "\n",
    "```\n",
    "\n",
    "The result must look like below.\n",
    "\n",
    "<figure>\n",
    "<video src = \"Assets/anim_color.mp4\" width=\"100%\" controls autoplay loop> </video>\n",
    "<figcaption style = \"text-align:left; font-style:italic\">Animation: Color</figcaption>\n",
    "</figure> \n",
    "\n",
    "The color is changing rapidly, which is not helping that much so we can use less value of pixels to add."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Animation Effect: Scale\n",
    "Again, i am going to use simple technique to perform scale like feature.\n",
    "\n",
    "```python\n",
    "                    ## Change color on current box\n",
    "                    ccolor = np.array([[5, 5, 8]]).astype(np.uint8)\n",
    "                    sf = 0.3\n",
    "                    if current_mode == previous_mode:\n",
    "                        count_modes += 1\n",
    "                        ccolor = count_modes * ccolor\n",
    "                        sf  *= count_modes \n",
    "                    if count_modes > count_mode:\n",
    "                        running_mode = current_mode\n",
    "                        count_modes = 0\n",
    "                    previous_mode = current_mode\n",
    "                    \n",
    "                    # to make animate like features, we will have to find part where cursor is on canvas\n",
    "                    mind = modes.index(current_mode)\n",
    "                    # to make 9 boxes, we need 10 llines on left and right\n",
    "                    vboxes = np.linspace(0, vui.shape[1], len(icons)+1).astype(np.int64)\n",
    "                    vboxes = [(vboxes[i], vboxes[i+1]) for i in range(len(icons))]\n",
    "                    vbox = vboxes[mind]\n",
    "                    vui_canvas_anim = vui_canvas.copy()\n",
    "                    \n",
    "                    ## Anim 2 scale##\n",
    "                    ### Scale down to up and repeat\n",
    "                    icon_box = vui_canvas_anim[:, vbox[0]:vbox[1]].copy()\n",
    "                    zeros_icon = np.zeros_like(icon_box)\n",
    "                    icshape = icon_box.shape\n",
    "                    icon_box = cv2.resize(icon_box.astype(np.uint8), (int(icshape[1]*sf), int(icshape[0]*sf)))\n",
    "                    # take only part of new icon that fits to its original part\n",
    "                    \n",
    "                    if sf > 1:\n",
    "                        rd = int((icon_box.shape[0] - icshape[0])/2)\n",
    "                        cd = int((icon_box.shape[1] - icshape[1])/2)\n",
    "                        zeros_icon[:, :] = icon_box[rd:icshape[0]+rd, cd:icshape[1]+cd] \n",
    "                    else:\n",
    "                        rd = int((icshape[0] - icon_box.shape[0])/2)\n",
    "                        cd = int((icshape[1] - icon_box.shape[1])/2)\n",
    "                        zeros_icon[rd:abs(icon_box.shape[0]-rd), cd:abs(icon_box.shape[1]-cd)] = icon_box[rd:abs(icon_box.shape[0]-rd), cd:abs(icon_box.shape[1]-cd)] \n",
    "                    \n",
    "                    vui_canvas_anim[:, vbox[0]:vbox[1]] = zeros_icon\n",
    "                    \n",
    "                    ## ANim 1, change color##\n",
    "                    # increase brightness?\n",
    "                    vui_canvas_anim[:, vbox[0]:vbox[1]] += ccolor\n",
    "```\n",
    "\n",
    "Just after we changed the color property, we will perform our animation. What happened there?\n",
    "* Take a part of box which where pointer was located.\n",
    "* Make zeros array just like that box.\n",
    "* Resize that part of box with some factor. \n",
    "* The scale factor is increased with modes counts. \n",
    "* When image is shrinked:\n",
    "    * Find the row divide i.e rd and column divide. TO make our icon fit just like previous icon. i.e. center of original and shrinked is same.\n",
    "* When image is enlarged:\n",
    "    * i.e rd and column divide. TO make our icon fit just like previous icon. i.e. center of original and shrinked is same.\n",
    "\n",
    "Now the result should look like below:-\n",
    "\n",
    "<figure>\n",
    "<video src = \"Assets/anim_scale.mp4\" width=\"100%\" controls autoplay loop> </video>\n",
    "<figcaption style = \"text-align:left; font-style:italic\">Animation: Scale</figcaption>\n",
    "</figure> \n",
    "\n",
    "Currently, the scaling happens real fast. Tuning the values will make it smoother."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Bonus Topic\n",
    "For the bonus topic, i am willing to make little easier vui. I want this system to be able to run with dual hand but only move/draw is usable by left hand. Doing this way, i can do other things like choose color from right hand and move/draw mode by left. And the final code looks like below. Honestly, i can not even understand the codes below after a day because it is so messy. I hope i can make it more factored.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-08-14T02:15:37.351887Z",
     "start_time": "2020-08-14T02:13:16.332673Z"
    },
    "code_folding": [
     4,
     11,
     24,
     154,
     377
    ],
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "import imutils\n",
    "import time\n",
    "from dcr.recognition import *\n",
    "\n",
    "def running_average(bg_img, image, aweight):\n",
    "    if bg_img is None:\n",
    "        bg_img = image.copy().astype(\"float\")\n",
    "    else:\n",
    "        cv2.accumulateWeighted(image, bg_img, aweight)    \n",
    "    return bg_img\n",
    "\n",
    "def get_contours(bg_img, image, threshold=10):\n",
    "    \n",
    "    # abs diff betn img and bg\n",
    "    diff = cv2.absdiff(bg_img.astype(\"uint8\"), image)    \n",
    "    _, th = cv2.threshold(diff, threshold, 255, cv2.THRESH_BINARY)\n",
    "    (cnts, _) = cv2.findContours(th.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)\n",
    "    \n",
    "    if len(cnts) == 0:\n",
    "        return None\n",
    "    else:\n",
    "        max_cnt = max(cnts, key=cv2.contourArea)\n",
    "        return th, max_cnt\n",
    "\n",
    "def run_system(count_mode=5, avg_frames=100, hroi=[250, 400, 480, 681],\n",
    "               groi=[150, 400, 240, 681], icon_dir=\"icons/\", color_dir=\"colors/\"\n",
    "              ):\n",
    "    #accumulate weight variable\n",
    "    aweight = 0.5\n",
    "    # strat the camera\n",
    "    cam = cv2.VideoCapture(0)\n",
    "    # ROI box\n",
    "    top, right, bottom, left = hroi\n",
    "    # move ROI\n",
    "    mtop, mright, mbottom, mleft =  100, 10, 250, 160\n",
    "    \n",
    "    # ROI for GUI\n",
    "    gtop, gright, gbottom, gleft = groi\n",
    "    gcolors = np.random.randint(0, 255, (9, 3), dtype=np.uint32).tolist()\n",
    "    gb_indices = int((gleft-gright)/9)\n",
    "    gb_indices = np.arange(gright, gleft, gb_indices)\n",
    "    gb_indices[-1] = gb_indices[-1]+1\n",
    "    # take each box's y coordinates\n",
    "    # read icons\n",
    "    icon_dir = icon_dir\n",
    "    icons = []\n",
    "    for image_name in os.listdir(icon_dir):\n",
    "        icons.append(cv2.imread(icon_dir+image_name))\n",
    "    icons = np.array(icons)\n",
    "\n",
    "    gui_boxes = [(gb_indices[i], gb_indices[i+1]) for i in range(len(gb_indices)-1)]\n",
    "    modes = os.listdir(icon_dir)\n",
    "    modes = [mode.split(\".\")[0] for mode in modes]\n",
    "    gui_modes_position = {modes[i]:gui_boxes[i] for i in range(len(gb_indices)-1)}\n",
    "    gui_modes_icons = {modes[i]:icons[i] for i in range(len(gb_indices)-1)}\n",
    "    \n",
    "    # stack it horizontally\n",
    "    icons_holder = np.hstack(icons)\n",
    "    #show(icons_holder)\n",
    "\n",
    "    # read colors\n",
    "    color_dir = \"colors/\"\n",
    "    colors = []\n",
    "    for color_name in os.listdir(color_dir):\n",
    "        color = cv2.imread(color_dir+color_name)\n",
    "        colors.append(color)\n",
    "    colors = np.array(colors)\n",
    "    # stack it vertically\n",
    "    colors_holder = np.vstack(colors)\n",
    "    #show(colors_holder)\n",
    "\n",
    "    # count frame\n",
    "    num_frames=0\n",
    "    # writing canvas\n",
    "    canvas = None\n",
    "    # thickness\n",
    "    t=3\n",
    "    # draw color(ink color)\n",
    "    draw_color = (0, 255, 0)\n",
    "    # pointer color\n",
    "    pointer_color = (255, 0, 0)\n",
    "    # mode flag\n",
    "    erase = False\n",
    "    # flag to indicate take average\n",
    "    take_average=True\n",
    "    #bg image\n",
    "    draw_bg=None\n",
    "    gui_bg = None\n",
    "    move_bg = None\n",
    "    \n",
    "    previous_mode = \"move\"\n",
    "    running_mode = \"move\"\n",
    "    drawn = None\n",
    "    count_modes = 0\n",
    "    vui_canvas = None\n",
    "    current_color = (20, 100, 50)\n",
    "    colors_array = np.array([(255, 0, 0), (0, 255, 0), (0, 0, 255)]).tolist()\n",
    "    # loop while everything is true\n",
    "    while True:\n",
    "        # read the camera result\n",
    "        (ret, frame) = cam.read()\n",
    "        # if camera has read frame\n",
    "        if ret:\n",
    "            # wait for 1ms to key press\n",
    "            key = cv2.waitKey(1) & 0xFF\n",
    "            frame = imutils.resize(frame, width=700)\n",
    "            # flip to remove mirror effect\n",
    "            frame = cv2.flip(frame, 1)\n",
    "            # clone it to not mess with real frame\n",
    "            clone = frame.copy()\n",
    "            gray = cv2.cvtColor(clone, cv2.COLOR_BGR2GRAY)\n",
    "            h, w = frame.shape[:2]\n",
    "            # take roi, to send it onto contour/average extraction\n",
    "            draw_roi = frame[top:bottom, right:left]\n",
    "            # roi to grayscale\n",
    "            #draw_gray = cv2.cvtColor(draw_roi, cv2.COLOR_BGR2GRAY)\n",
    "            draw_gray = gray[top:bottom, right:left]\n",
    "            # add GaussianBlur to eliminate some noise\n",
    "            draw_gray = cv2.GaussianBlur(draw_gray, (7, 7), 0)\n",
    "\n",
    "\n",
    "            gui_roi = frame[gtop:gbottom, gright:gleft]\n",
    "            #gui_gray = cv2.cvtColor(gui_roi, cv2.COLOR_BGR2GRAY)\n",
    "            gui_gray = gray[gtop:gbottom, gright:gleft]\n",
    "            gui_gray = cv2.GaussianBlur(gui_gray, (7, 7), 0)\n",
    "            \n",
    "            move_gray = gray[mtop:mbottom, mright:mleft]\n",
    "            \n",
    "            if vui_canvas is None:\n",
    "                gshape = gray.shape\n",
    "                vui_canvas = cv2.resize(icons_holder, (gshape[1], 100)).astype(np.uint8)\n",
    "                vshape = vui_canvas.shape\n",
    "                vui = np.zeros_like(frame)\n",
    "                vui[:100, :] = vui_canvas\n",
    "\n",
    "            # if to take average and num frames on average taking is lesser than \n",
    "            if num_frames<avg_frames and take_average==True:\n",
    "                # perform running average\n",
    "                draw_bg = running_average(draw_bg, draw_gray, aweight)                \n",
    "                gui_bg = running_average(gui_bg, gui_gray, aweight=aweight)\n",
    "                move_bg = running_average(move_bg, move_gray, aweight=aweight)\n",
    "                # put frame number on frame\n",
    "                cv2.putText(clone, str(num_frames), (100, 100),\n",
    "                                       cv2.FONT_HERSHEY_SIMPLEX, 2, (0, 0, 255), 1)\n",
    "                num_frames+=1\n",
    "            # if not to take average\n",
    "            else:\n",
    "                num_frames=0                        \n",
    "                # take our segmented hand\n",
    "                gui_hand = get_contours(gui_bg, gui_gray)\n",
    "                draw_hand = get_contours(draw_bg, draw_gray)\n",
    "                move_hand = get_contours(move_bg, move_gray)\n",
    "                take_average=False\n",
    "\n",
    "                if gui_hand is not None:\n",
    "                    # get the position of contours\n",
    "                    gthresholded, gsegmented = gui_hand\n",
    "                    cv2.drawContours(clone, [gsegmented+(gright,gtop)], -1, (0, 0, 255))   \n",
    "                    original_gcontour = gsegmented + (gright, gtop)\n",
    "\n",
    "                    tshape = gthresholded.shape\n",
    "                    sshape = gsegmented.shape\n",
    "                    new_segmented = original_gcontour.reshape(sshape[0], sshape[-1])\n",
    "                    m = new_segmented.min(axis=0)\n",
    "                    # check if the y axis lies on any of gui boxes\n",
    "                    current_box = [box for box in gui_boxes if box[0] <= m[0] <= box[1]][0] \n",
    "                    current_mode = [mode for mode, pos in gui_modes_position.items() if current_box==pos][0]\n",
    "                    \n",
    "                    ## Change color on current box\n",
    "                    ccolor = np.array([[5, 5, 8]]).astype(np.uint8)\n",
    "                    sf = 0.2\n",
    "                    if current_mode == previous_mode:\n",
    "                        count_modes += 1\n",
    "                        ccolor = count_modes * ccolor\n",
    "                        sf  *= count_modes \n",
    "                    if count_modes > count_mode:\n",
    "                        running_mode = current_mode\n",
    "                        count_modes = 0\n",
    "                    previous_mode = current_mode\n",
    "                    \n",
    "                    # to make animate like features, we will have to find part where cursor is on canvas\n",
    "                    mind = modes.index(current_mode)\n",
    "                    # to make 9 boxes, we need 10 llines on left and right\n",
    "                    vboxes = np.linspace(0, vui.shape[1], len(icons)+1).astype(np.int64)\n",
    "                    vboxes = [(vboxes[i], vboxes[i+1]) for i in range(len(icons))]\n",
    "                    vbox = vboxes[mind]\n",
    "                    vui_canvas_anim = vui_canvas.copy()\n",
    "                    \n",
    "                    ## ANim 1, change color##\n",
    "                    # increase brightness?\n",
    "                    vui_canvas_anim[:, vbox[0]:vbox[1]] += ccolor\n",
    "                    \n",
    "                    ## Anim 2 scale##\n",
    "                    ### Scale down to up and repeat\n",
    "                    icon_box = vui_canvas_anim[:, vbox[0]:vbox[1]].copy()\n",
    "                    zeros_icon = np.zeros_like(icon_box)\n",
    "                    icshape = icon_box.shape\n",
    "                    icon_box = cv2.resize(icon_box.astype(np.uint8), (int(icshape[1]*sf), int(icshape[0]*sf)))\n",
    "                    # take only part of new icon that fits to its original part\n",
    "                    \n",
    "                    \n",
    "                    if sf > 1:\n",
    "                        rd = int((icon_box.shape[0] - icshape[0])/2)\n",
    "                        cd = int((icon_box.shape[1] - icshape[1])/2)\n",
    "                        #print(icon_box.shape, icshape, rd, icon_box.shape[0]-rd, cd,icon_box.shape[1]-cd)\n",
    "                        # edit here.....\n",
    "                        zeros_icon[:, :] = icon_box[rd:icshape[0]+rd, cd:icshape[1]+cd] \n",
    "                    else:\n",
    "                        rd = int((icshape[0] - icon_box.shape[0])/2)\n",
    "                        cd = int((icshape[1] - icon_box.shape[1])/2)\n",
    "                        \n",
    "                        #print(icon_box.shape, icshape, rd, icon_box.shape[0]-rd, cd,icon_box.shape[1]-cd)\n",
    "                        zeros_icon[rd:abs(icon_box.shape[0]-rd), cd:abs(icon_box.shape[1]-cd)] = icon_box[rd:abs(icon_box.shape[0]-rd), cd:abs(icon_box.shape[1]-cd)] \n",
    "                    \n",
    "                    vui_canvas_anim[:, vbox[0]:vbox[1]] = zeros_icon\n",
    "                    vui_canvas_anim[:, vbox[0]:vbox[1]] += ccolor\n",
    "                    if vui_canvas is not None:\n",
    "                        gshape = gray.shape\n",
    "                        dummy = np.zeros_like(clone)\n",
    "                        dummy_copy = dummy.copy()\n",
    "                        dummy[:100, :] += vui_canvas_anim\n",
    "                        cv2.circle(dummy_copy, (m[0], m[1]), 10, (250, 250, 150), -3)\n",
    "                        d = dummy_copy[gtop:gbottom, gright:gleft]                                                \n",
    "                        d = cv2.resize(d, (gshape[1], vshape[0]))                        \n",
    "                        dummy_temp = dummy[:100, :]\n",
    "                        dummy_temp[d!=[0, 0, 0]] = 100\n",
    "                        dummy[:100, :] += dummy_temp\n",
    "                        vui = dummy\n",
    "                        \n",
    "                    \n",
    "                    cv2.circle(clone, (m[0], m[1]), 10, pointer_color, -3)\n",
    "                \n",
    "                if move_hand is not None:\n",
    "                    mthresholded, msegmented = move_hand\n",
    "                    sshape = msegmented.shape\n",
    "                    new_segmented = msegmented.reshape(sshape[0], sshape[-1])\n",
    "                    m = new_segmented.min(axis=0)\n",
    "                    cv2.drawContours(clone, [msegmented+(mright,mtop)], -1, (0, 0, 255))   \n",
    "                    if len(msegmented)>50:\n",
    "                        if m[0]+mright > int((mright+mleft)/2):\n",
    "                            running_mode = \"draw\"\n",
    "                            vui[:100] = vui_canvas\n",
    "                            vui[100:] = 0\n",
    "                        else:\n",
    "                            running_mode = \"move\"\n",
    "                            vui[:100] = vui_canvas\n",
    "                            vui[100:] = 0\n",
    "                if draw_hand is not None:\n",
    "                    dthresholded, dsegmented = draw_hand\n",
    "                    cv2.drawContours(clone, [dsegmented+(right,top)], -1, (0, 0, 255))   \n",
    "\n",
    "                    tshape = dthresholded.shape\n",
    "                    sshape = dsegmented.shape\n",
    "                    new_segmented = dsegmented.reshape(sshape[0], sshape[-1])\n",
    "                    m = new_segmented.min(axis=0)\n",
    "\n",
    "                    \n",
    "                    # if pressed x, erase\n",
    "                    if chr(key) == \"x\" or running_mode==\"erase\":\n",
    "                        draw_color = (255, 255, 255)\n",
    "                        pointer_color = (0, 0, 255)\n",
    "                        erase = True\n",
    "                    if chr(key) == \"c\" or running_mode==\"draw\":\n",
    "                        draw_color = current_color\n",
    "                        pointer_color = (255, 0, 0)\n",
    "                        erase = False\n",
    "                    #idle\n",
    "                    if chr(key) == \"z\" or running_mode==\"move\":\n",
    "                        erase = None\n",
    "                        pointer_color = (0, 0, 0)                   \n",
    "                    # restart system\n",
    "                    if chr(key) == \"r\" or running_mode == \"restart\":\n",
    "                        take_average=True\n",
    "                        running_mode = \"move\"\n",
    "                        canvas = None\n",
    "                    if chr(key) == \"e\" or running_mode == \"clear\":\n",
    "                        canvas = None\n",
    "                        drawn = np.zeros(drawn.shape)+255\n",
    "                        running_mode=\"move\"\n",
    "                    if chr(key) == \"v\" or running_mode == \"color\":\n",
    "                        # add color region on VUI\n",
    "                        # only use few portion of vui for dropdown\n",
    "                        xs = vui.shape[0]-300\n",
    "                        # what will be the length of each box on modes panel?\n",
    "                        # since our colors mode lies on 1st box\n",
    "                        ys = int(vshape[1]/len(icons)) * 2 - int(vshape[1]/len(icons))\n",
    "                        # length of each box\n",
    "                        c1 = int(vshape[1]/len(icons))\n",
    "                        # col. num. of 1st box's end\n",
    "                        c2 = c1 * 2\n",
    "                        # for every row below the modes region up to xs, for only 1st box \n",
    "                        vui[100:100+xs, c1:c2] = cv2.resize(colors_holder, (ys, xs))\n",
    "                        \n",
    "                        # we don't want yet to draw so we will move cursor instead\n",
    "                        erase = None\n",
    "                        \n",
    "                        # out of xs length on vui, how much region will a single color can hold?\n",
    "                        cboxes = int(xs/len(colors))\n",
    "                        \n",
    "                        # take parts of rows from the draw ROI because our ROI for color lies on Draw ROI part. \n",
    "                        # if gshape[0] rows contains xs parts for colors then what must contain for bottom-top?\n",
    "                        crb = int(xs/gshape[0] * (bottom-top))\n",
    "                        \n",
    "                        # divide that crb rows into num colors part\n",
    "                        cboxes = np.linspace(0, crb, len(colors)+1).astype(np.int64)\n",
    "                        # find each box's extreme columns on Draw ROI   \n",
    "                        cboxes = [(cboxes[i], cboxes[i+1]) for i in range(len(cboxes)-1)]\n",
    "                        #print(len(cboxes))\n",
    "                        \n",
    "                        # in which box is pointer now?\n",
    "                        # check the rows, on which rows among colors ROI is pointer now?\n",
    "                        cpointer = [cbox for cbox in cboxes if (cbox[0] < m[1] <= cbox[1])]\n",
    "                        \n",
    "                        # check the rows\n",
    "                        cdb = (left-right)/len(icons) # draw box, divide original col draw ROI into num icons pcs\n",
    "                        if  len(cpointer) > 0 and cdb<=m[0]<=cdb*2: # since color lies on 2nd box check if pointer is on that col\n",
    "                            current_color = colors_array[cboxes.index(cpointer[-1])]\n",
    "                            \n",
    "                    if type(canvas) == type(None):\n",
    "                        canvas = np.zeros((tshape[0], tshape[1], 3))+255\n",
    "                    c = np.zeros(canvas.shape, dtype=np.uint8)\n",
    "                    cv2.circle(c, (m[0], m[1]), 5, pointer_color, -3)\n",
    "                    cv2.circle(clone, (right+m[0], top+m[1]), 5, pointer_color, -3)\n",
    "                    #print(right+m[0], top+m[1])\n",
    "                    if erase==True:\n",
    "                        cv2.circle(canvas, (m[0], m[1]), 5, draw_color, -3)\n",
    "                        erimg=cv2.circle(canvas.copy(), (m[0], m[1]), 5, (0, 0, 0), -3)            \n",
    "                        cv2.circle(c, (m[0], m[1]), 5, (0, 0, 255), -3)\n",
    "                        e = cv2.erode(erimg, (3, 3), iterations=5)\n",
    "                        drawn = cv2.resize(e, (clone.shape[1], clone.shape[0]))\n",
    "                        c = cv2.resize(c, (clone.shape[1], clone.shape[0]))\n",
    "                        drawn_new = drawn+c\n",
    "                        new_canvas = np.vstack([vui[:100], drawn_new]).astype(np.uint8)\n",
    "                        cv2.imshow(\"Drawing\", new_canvas)\n",
    "                        #cv2.imshow(\"Drawing\", drawn+c)\n",
    "                        erimg=cv2.circle(canvas.copy(), (m[0], m[1]), 5, (255, 255, 255), -3)            \n",
    "                        cv2.circle(c, (m[0], m[1]), 5, (0, 0, 255), -3)\n",
    "                        e = cv2.erode(erimg, (3, 3), iterations=5)\n",
    "                        drawn = cv2.resize(e, (clone.shape[1], clone.shape[0])) \n",
    "                        dc = drawn.astype(np.uint8)\n",
    "                        #cv2.imshow(\"dc\", drawn.astype(np.uint8))\n",
    "                        #exception from here, xs not defined\n",
    "                        try:\n",
    "                            new_canvas[100:100+xs] += vui[100:100+xs]\n",
    "                        except:\n",
    "                            pass\n",
    "                        #show(vui[100:100+xs])\n",
    "                    elif erase==False:\n",
    "                        cv2.circle(canvas, (m[0], m[1]), 5, draw_color, -3)\n",
    "                        #print(mm)\n",
    "                        e = cv2.erode(canvas, (3, 3), iterations=5)\n",
    "                        drawn = cv2.resize(e, (clone.shape[1], clone.shape[0]))\n",
    "                        c = cv2.resize(c, (clone.shape[1], clone.shape[0]))\n",
    "                        drawn_new = drawn+c\n",
    "                        new_canvas = np.vstack([vui[:100], drawn_new]).astype(np.uint8)\n",
    "                        try:\n",
    "                            new_canvas[100:100+xs] += vui[100:100+xs]\n",
    "                        except:\n",
    "                            pass\n",
    "                        #show(vui[100:100+xs])\n",
    "                        \n",
    "                        cv2.imshow(\"Drawing\", new_canvas)\n",
    "                        dc = drawn.astype(np.uint8)\n",
    "                        #cv2.imshow(\"D\", drawn.astype(np.uint8))\n",
    "                        #cv2.imshow(\"c\", c)\n",
    "                    elif erase == None:\n",
    "                        canvas_shape = canvas.shape\n",
    "                        clone_shape = clone.shape\n",
    "                        eshape = (clone_shape[0]/canvas_shape[0], clone_shape[1]/canvas_shape[1])\n",
    "                        m[0] = int(eshape[1]*m[0])\n",
    "                        m[1] = int(eshape[0]*m[1])\n",
    "\n",
    "                        if drawn is None:\n",
    "                            e = cv2.erode(canvas, (3, 3), iterations=5)\n",
    "                            drawn = cv2.resize(e, (clone.shape[1], clone.shape[0]))\n",
    "\n",
    "                        drawn = cv2.resize(drawn, (clone.shape[1], clone.shape[0]))\n",
    "                        dc = drawn.copy()  \n",
    "                        cv2.circle(dc, (m[0], m[1]), 10, pointer_color, -3)\n",
    "                        drawn_new = dc\n",
    "                        new_canvas = np.vstack([vui[:100], drawn_new]).astype(np.uint8)\n",
    "                        if running_mode == \"color\":\n",
    "                            new_canvas[100:100+xs, c1:c2] = vui[100:100+xs, c1:c2]\n",
    "                        #show(vui[100:100+xs])\n",
    "                        \n",
    "                        cv2.imshow(\"Drawing\", new_canvas)\n",
    "                        #cv2.imshow(\"Drawing\", dc)\n",
    "\n",
    "            if chr(key) == \"s\" or running_mode==\"detect\":\n",
    "                if drawn is not None:\n",
    "                    show(drawn)\n",
    "                    d = drawn.copy().astype(np.uint8) \n",
    "                    r = recognition(cv2.cvtColor(d, cv2.COLOR_BGR2GRAY), 'show')\n",
    "                    cv2.imshow(\"Detection\", r)\n",
    "                    running_mode=\"move\"\n",
    "\n",
    "            # draw a ROI for Draw\n",
    "            cv2.rectangle(clone, (left, top), (right, bottom), (0, 255, 0), 2)\n",
    "            cv2.putText(clone, str(\"GUI ROI\"), (400, 140),\n",
    "                                       cv2.FONT_HERSHEY_SIMPLEX, 2, (0, 0, 255), 2)\n",
    "            cv2.putText(clone, f\"Curr. Mode: {running_mode}\", (400, 100),\n",
    "                                       cv2.FONT_HERSHEY_SIMPLEX, 1, (0, 0, 255), 2)\n",
    "            cv2.putText(clone, str(\"Move\"), (int((mright)/1), int((mtop+mbottom)/2)),\n",
    "                                       cv2.FONT_HERSHEY_SIMPLEX, 1, (0, 0, 255), 2)\n",
    "            cv2.putText(clone, str(\"Draw\"), (int((mleft + mright)/2), int((mtop+mbottom)/2)),\n",
    "                                       cv2.FONT_HERSHEY_SIMPLEX, 1, (0, 0, 255), 2)\n",
    "            cv2.rectangle(clone, (mleft, mtop), (int((mleft + mright)/2), mbottom), (0, 255, 0), 2)\n",
    "            cv2.rectangle(clone, (int((mleft + mright)/2), mtop), (mright, mbottom), (0, 255, 0), 2)\n",
    "            \n",
    "            # draw a ROI boxes for GUI\n",
    "            for i in range(len(gb_indices)-1):\n",
    "                _gleft = gb_indices[i]\n",
    "                _gright = gb_indices[i+1]\n",
    "                cv2.rectangle(clone, (_gleft, gtop), (_gright, gbottom), gcolors[i], 3)\n",
    "                cv2.putText(clone, modes[i][:2], (_gleft+2, int((gtop+gbottom)/2)),\n",
    "                                       cv2.FONT_HERSHEY_SIMPLEX, 0.75, gcolors[i], 2)\n",
    "            # show live feed\n",
    "            cv2.imshow(\"Feed\", clone)\n",
    "            if key == 32 or running_mode==\"save\":\n",
    "                cv2.imwrite(\"Captured.png\", dc)\n",
    "                cv2.imshow(\"captured\", dc)\n",
    "                running_mode = \"move\"\n",
    "            # if pressed  escape, loop out and stop processing\n",
    "            if running_mode==\"exit\" or key==27:\n",
    "                break\n",
    "    cam.release()\n",
    "    cv2.destroyAllWindows()\n",
    "run_system(avg_frames=30)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To make it work from left and right side, i have added new ROI on left. \n",
    "* When there is more than 50 contour points detected then one of move/draw is chosen.\n",
    "* We are dividing the box of move/draw on 2 equal parts, left part for moving cursor and right for draw mode.\n",
    "* Then we re assigned `vui`, else the color dropdown will be visible whenever we draw.\n",
    "* Also i have made few modifications inside each condition of `erase`, to save the drawn image."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The output of above code should look like below. <b> Note that i have disabled some modes to make it fast and easier to show demo. Because my room was having frequent light change due to rainy day and blackout.</b>\n",
    "\n",
    "### Final Result\n",
    "<figure>\n",
    "<video src = \"Assets/anim1_final.mp4\" width=\"100%\" controls autoplay loop> </video>\n",
    "<figcaption style = \"text-align:left; font-style:italic\">Animation and New VUI</figcaption>\n",
    "</figure> "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-08-13T07:46:55.034523Z",
     "start_time": "2020-08-13T07:46:20.579Z"
    }
   },
   "source": [
    "## Finally\n",
    "I am going to point out features and shortcomings of this system now. \n",
    "\n",
    "### Codes\n",
    "Codes to current version of the system is available on link below and if it is not, then hit the comment or leave me message (LinkedIn or Twitter or mail me).\n",
    "* [Contour Based Writing](https://github.com/q-viper/Contour-Based-Writing/)\n",
    "\n",
    "### Features\n",
    "* Dual hand can be used to draw/move pointer.\n",
    "* Color can be picked from dropdown like fashion.\n",
    "* Total of 9 modes available.\n",
    "\n",
    "### Shortcomings\n",
    "Shortcomings are most helpful to find new feature on next version. Well here are plenty of them.\n",
    "* Code is looking messy. Refactoring is needed and must be made modules.\n",
    "* The idea to chose color is not always useful. Use the color array instead of color image. Or use color pallete for R, G, B and pick the color when pointer lies above it.\n",
    "* The system is usable by contour of anything. Hence some gesture confirming model must be used.\n",
    "* The background has to be taken many times to take average.\n",
    "\n",
    "## What Next?\n",
    "I will try to solve shortcomings on next time. But i am interested to make this system run on mobile phones to. As per now, i am thinking of taking frames from device camera and process it. Then use some API call to get those frame. I might use Unity. I am highly excited to try using LSTMs and other state of the art Deep Learning Algorithms to make this system more awesome but i don't have internet access (other than cellular data) to do broad research."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Why not read more?\n",
    "* [Deploying a RASA Chatbot on Android using Unity3d](https://acharyaramkrishna.com.np/2020/08/04/deploying-a-simple-rasa-chatbot-on-unity3d-project-to-make-a-chatbot-for-android-devices/)\n",
    "* [Gesture Based Visually Writing System Using OpenCV and Python](https://acharyaramkrishna.com.np/2020/08/01/gesture-based-visually-writing-system-using-opencv-and-python/)\n",
    "* [Logistic Regression from Scratch](https://acharyaramkrishna.com.np/2020/08/11/writing-a-logistic-regression-class-from-scratch/)\n",
    "* [Linear Regression from Scratch](https://acharyaramkrishna.com.np/2020/08/07/writing-a-linear-regression-class-from-scratch-using-python/)\n",
    "* [Writing Popular ML Optimizers from Scratch](https://acharyaramkrishna.com.np/2020/06/05/writing-popular-machine-learning-optimizers-from-scratch-on-python/)\n",
    "* [Feed Forward Neural Network from Scratch](https://acharyaramkrishna.com.np/2020/05/31/writing-a-deep-neural-network-from-scratch-on-python/)\n",
    "* [Convolutional Neural Networks from Scratch](https://acharyaramkrishna.com.np/2020/06/05/convolutional-neural-networks-from-scratch-on-python/)\n",
    "* [Writing a Simple Image Processing Class from Scratch](https://acharyaramkrishna.com.np/2020/05/31/image-processing-class-from-scratch-on-python/)\n",
    "* [Naive Bayes for text classifications: Scratch to Framework](https://acharyaramkrishna.com.np/2020/03/04/text-classification-using-naive-bayes-scratch-to-the-framework/)\n",
    "* [Simple OCR for Devanagari Handwritten Text](https://acharyaramkrishna.com.np/2020/02/25/building-ocr-for-devanagari-handwritten-character/)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  },
  "latex_envs": {
   "LaTeX_envs_menu_present": true,
   "autoclose": false,
   "autocomplete": true,
   "bibliofile": "biblio.bib",
   "cite_by": "apalike",
   "current_citInitial": 1,
   "eqLabelWithNumbers": true,
   "eqNumInitial": 1,
   "hotkeys": {
    "equation": "Ctrl-E",
    "itemize": "Ctrl-I"
   },
   "labels_anchors": false,
   "latex_user_defs": false,
   "report_style_numbering": false,
   "user_envs_cfg": false
  },
  "nbTranslate": {
   "displayLangs": [
    "*"
   ],
   "hotkey": "alt-t",
   "langInMainMenu": true,
   "sourceLang": "en",
   "targetLang": "fr",
   "useGoogleTranslate": true
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {
    "height": "calc(100% - 180px)",
    "left": "10px",
    "top": "150px",
    "width": "291.319px"
   },
   "toc_section_display": true,
   "toc_window_display": true
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}