19-Intro2ML-HodaDataset.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"http://akhavanpour.ir/notebook/images/srttu.gif\" alt=\"SRTTU\" style=\"width: 150px;\"/>\n",
    "\n",
    "[![Azure Notebooks](https://notebooks.azure.com/launch.png)](https://notebooks.azure.com/import/gh/Alireza-Akhavan/class.vision)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbpresent": {
     "id": "588ee1fa-64b5-453b-ade7-8e6b2515821c"
    }
   },
   "source": [
    "# <div style=\"direction:rtl;text-align:right;font-family:B Lotus, B Nazanin, Tahoma\">معرفی مجموعه داده هدی و تغییر اندازه و شکل برای استفاده در الگوریتم‌های یادگیری ماشین\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <div style=\"direction:rtl;text-align:right;font-family:B Lotus, B Nazanin, Tahoma\">معرفی مجموعه داده ارقام دستنویس هدی</div>\n",
    "\n",
    "<div style=\"direction:rtl;text-align:right;font-family:Tahoma\">\n",
    "مجموعه ارقام دستنویس هدی که اولین مجموعه‌ی بزرگ ارقام دستنویس فارسی است، مشتمل بر ۱۰۲۳۵۳ نمونه دستنوشته سیاه سفید است. این مجموعه طی انجام یک پروژه‏ی کارشناسی ارشد درباره بازشناسی فرمهای دستنویس تهیه شده است. داده های این مجموعه از حدود ۱۲۰۰۰ فرم ثبت نام آزمون سراسری کارشناسی ارشد سال ۱۳۸۴ و آزمون کاردانی پیوسته‏ی دانشگاه جامع علمی کاربردی سال ۱۳۸۳ استخراج شده است[1].\n",
    "</div>\n",
    "<p>\n",
    "[1] <a href=\"https://www.sciencedirect.com/science/article/pii/S0167865507000037\" target=\"_blank\">\n",
    "H. Khosravi and E. Kabir, \"Introducing a very large dataset of handwritten Farsi digits and a study on their varieties,\" Pattern Recognition Letters, vol. 28, pp. 1133-1141, 2007. ]\n",
    "</a>\n",
    "</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <div style=\"direction:rtl;text-align:right;font-family:B Lotus, B Nazanin, Tahoma\">لود کتابخانه‌های مورد استفاده</div>\n",
    "<div style=\"direction:rtl;text-align:right;font-family:Tahoma\">\n",
    "برای اجرای این نوت‌بوک باید دو کتابخانه ی scipy و matplotlib را نصب کنید.\n",
    "بدین منظور به اینترنت متصل شود و در cmd دو دستور زیر را بنویسید:\n",
    "</div>\n",
    "<p>pip install scipy</p>\n",
    "<p>pip install matplotlib</p>\n",
    "\n",
    "<div style=\"direction:rtl;text-align:right;font-family:Tahoma\">\n",
    "میتوانید به جای pip از کلمه ی conda استفاده کنید. (اگر از آناکوندا استفاده میکنید.)\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "nbpresent": {
     "id": "de829a92-1fb6-44ad-a2c6-fc1001e1f6e1"
    }
   },
   "outputs": [],
   "source": [
    "import cv2\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from scipy import io\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <div style=\"direction:rtl;text-align:right;font-family:B Lotus, B Nazanin, Tahoma\">لود مجموعه داده (dataset)</div>\n",
    "<div style=\"direction:rtl;text-align:right;font-family:Tahoma\">\n",
    "این مجموعه داده در قالب فایل mat متلب منتشر شده است. در پایتون میتوانیم آن را با کتابخانه scipy لود کنیم.\n",
    "</div>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "#load dataset\n",
    "dataset = io.loadmat('./dataset/Data_hoda_full.mat')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <div style=\"direction:rtl;text-align:right;font-family:B Lotus, B Nazanin, Tahoma\">انتخاب مجموعه داده آموزش و مجموعه داده آزمون(تست)</div>\n",
    "<div style=\"direction:rtl;text-align:right;font-family:Tahoma\">\n",
    "در این مجموعه داده 60000 داده تصادفی وجود دارد.\n",
    "<br>\n",
    "برای آزمایشاتمان 1000 داده‌ی اول را برای آموزش و 200 داده‌ی بعدی را برای آزمون انتخاب می‌کنیم.\n",
    "<br>\n",
    "**داده های آزمون نباید قبلا در داده های آموزشی دیده شده باشند.**\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "#test and training set\n",
    "X_train_orginal = np.squeeze(dataset['Data'][:1000])\n",
    "y_train = np.squeeze(dataset['labels'][:1000])\n",
    "X_test_original = np.squeeze(dataset['Data'][1000:1200])\n",
    "y_test = np.squeeze(dataset['labels'][1000:1200])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div style=\"direction:rtl;text-align:right;font-family:Tahoma\">\n",
    "نگاهی به مجموعه داده بیندازیم!\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "cv2.imshow('Hoda[0]',X_train_orginal[0])\n",
    "cv2.waitKey()\n",
    "cv2.destroyAllWindows()      "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div style=\"direction:rtl;text-align:right;font-family:Tahoma\">\n",
    "بیایید به جای opencv از matplotlib استفاده کنیم که در خود نوت‌بوک خروجی را ببینیم!\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAMYAAAD8CAYAAAAsetuWAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAACm1JREFUeJzt3VGoHOUZxvHnaaw3qVDF5BA0baSE\ngghNm+CNpURKJJVC9EIxVykVjhcNtHcNvTFQCrmwlV6UQmxDctFqBRVDKsYQSm2htJ6I1di0TQip\nHhNylFgar0R9e3HmwJq8u7PZmd359pz/D5bdnbNn5s3ZPHwz8803nyNCAD7tM10XAJSIYAAJggEk\nCAaQIBhAgmAACYIBJAgGkCAYQOK6SW7Mdqfd7Js3b268jhMnTrRQCboUEa77jJtcEmJ7u6SfS1ol\n6VcRsa/m850Go43LX+zavykKN9Zg2F4l6d+Stkmal/SKpJ0R8Y8Bv0Mw0LlhgtHkGONOSWci4mxE\nfCjpKUk7GqwPKEaTYNwi6e2e9/PVsk+xPWt7zvZcg20BE9Xk4Dtrjq7aV4mI/ZL2S93vSgHDatJi\nzEta3/P+Vknnm5UDlKFJMF6RtNH2bbavl/SQpMPtlAV0a+RdqYj4yPZuSUe1eLr2QES82Vplo9XU\n+TY4a7U8NOrHuOaNjfkYo4RhugSjfOM+XQssWwQDSBAMIEEwgATBABIEA0hMdDxGUyWcjsX4Nf2e\n2zhlTosBJAgGkCAYQIJgAAmCASQIBpAgGECiqH6Maein4LLy5sb9PQ9a/5YtW4ZaBy0GkCAYQIJg\nAAmCASQIBpAgGECCYACJovoxukYfRXMl9EUxHgMYE4IBJAgGkCAYQIJgAAmCASQIBpBYUf0Y9FM0\nV0I/xSQ0Cobtc5IuS/pY0kcRMdwoEKBwbbQYd0fEey2sBygGxxhAomkwQtJLtk/Ynm2jIKAETXel\n7oqI87bXSjpm+58R8XLvB6rAEBpMldYmp7S9V9IHEfHYgM8M3Ni4z3hwVqq5aTgrVfc9j3VyStur\nbd+w9FrSPZJOjro+oCRNjjFmJP3Z9t8l/U3S7yPixUG/sHnzZkVE3we6N+j7KeU7sj3w0YaRjzEi\n4qykr7RSBVAYTtcCCYIBJAgGkCAYQIJgAAmCASQIBpBYVgOVuOSjXimddIOU8D3SYgAJggEkCAaQ\nIBhAgmAACYIBJAgGkFhW/RgrwTT0QywHtBhAgmAACYIBJAgGkCAYQIJgAAmCASToxyjMSuinKGG8\nRR1aDCBBMIAEwQASBANIEAwgQTCABMEAErX9GLYPSPq2pIWIuKNadpOk30naIOmcpAcj4v3xlTmc\nldAHULpp6KMYxjAtxkFJ269YtkfS8YjYKOl49R5YNmqDUc3CeumKxTskHapeH5J0X8t1AZ0a9Rhj\nJiIuSFL1vLa9koDujf3g2/as7Tnbc+++++64Nwe0YtRgXLS9TpKq54V+H4yI/RGxJSK2rFmzZsTN\nAZM1ajAOS9pVvd4l6fl2ygHKUBsM209K+oukL9uet/2wpH2Sttk+LWlb9R5YNmr7MSJiZ58ffbPl\nWjAFlks/RR16voEEwQASBANIEAwgQTCABMEAEgQDSHiSYxhsD9wY4ym6txL6KSKi9h9JiwEkCAaQ\nIBhAgmAACYIBJAgGkCAYQKKo+THGfQ6dfhIMixYDSBAMIEEwgATBABIEA0gQDCBBMIBEUf0Y4zaJ\nsQal95WshPEWbaDFABIEA0gQDCBBMIAEwQASBANIEAwgsaL6MVYC+inaMcyMSgdsL9g+2bNsr+13\nbL9WPe4db5nAZA2zK3VQ0vZk+eMRsal6vNBuWUC3aoMRES9LujSBWoBiNDn43m379WpX68Z+H+qd\n57vBtoCJGuqmzrY3SDoSEXdU72ckvScpJP1Y0rqI+O4Q6yn7CrsWdH0RIQff9cZ2U+eIuBgRH0fE\nJ5KekHTnKOsBSjVSMGyv63l7v6ST/T4LTKPafgzbT0raKulm2/OSHpW01fYmLe5KnZP0yBhrLErX\nu0qYjKImjpkGpQeDY4x6TBwDjIhgAAmCASQIBpAgGECCYAAJxmNMGU7HTgYtBpAgGECCYAAJggEk\nCAaQIBhAgmAACfoxepR+STkmhxYDSBAMIEEwgATBABIEA0gQDCBBMIAE/RiFYbxFGWgxgATBABIE\nA0gQDCBBMIAEwQASBANIEAwgMcw83+tt/8H2Kdtv2v5+tfwm28dsn66e+05QCUyb2oljqmnF1kXE\nq7ZvkHRC0n2SviPpUkTss71H0o0R8cOadRU9RK6EEXz0fI9fKxPHRMSFiHi1en1Z0ilJt0jaIelQ\n9bFDWgwLsCxc07VS1bTGX5X0V0kzEXFBWgyP7bV9fmdW0myzMoHJGnoOPtufk/RHST+JiGdt/zci\nPt/z8/cjYuBxBrtS9diVGr/W5uCz/VlJz0j6TUQ8Wy2+uDStcfW8MGqhQGmGOStlSb+WdCoiftbz\no8OSdlWvd0l6vv3ygG4Mc1bq65L+JOkNSZ9Ui3+kxeOMpyV9QdJbkh6IiEs16+p+X2UAdqVWhmF2\npZjnuwfBWBmY5xsYEcEAEgQDSBAMIEEwgATBABIEA0hww7UJo59iOtBiAAmCASQIBpAgGECCYAAJ\nggEkCAaQWFH9GCWMt8B0oMUAEgQDSBAMIEEwgATBABIEA0gQDCBBMIAEwQASBANIEAwgQTCABMEA\nEgQDSBAMINFknu+9tt+x/Vr1uHf85ZbP9sAHpkOTeb4flPRBRDw29MY6njhmEgOV+M9fvmEmjqkd\nwVdNWbw0bfFl20vzfAPL1jUdY1wxz7ck7bb9uu0DtgdOZQxMk6GDUc3z/YykH0TE/yT9UtKXJG3S\nYovy0z6/N2t7zvZcC/UCEzHU5JTVPN9HJB29YkrjpZ9vkHQkIu6oWQ/HGOhcK5NT9pvnuzooX3K/\npJOjFAmUqMk83zu1uBsVks5JeqQ6UB+0LloMdI55vq9RG38LglE+5vkGRkQwgATBABIEA0gQDCBB\nMIAEwQASk54f4z1J/+l5f3O1rAhJH0RR9fVReo2l1ffFYT400Q6+qzZuz0XEls4KqFF6fVL5NZZe\nXz/sSgEJggEkug7G/o63X6f0+qTyayy9vlSnxxhAqbpuMYAidRIM29tt/8v2Gdt7uqihju1ztt+o\nbg3U+bDcalz9gu2TPctusn3M9unqudNx931qnMrbLE08GLZXSfqFpG9Jul3STtu3T7qOId0dEZsK\nOd14UNL2K5btkXQ8IjZKOl6979JBXV2jJD1e/R03RcQLE65pJF20GHdKOhMRZyPiQ0lPSdrRQR1T\nJSJelnTpisU7JB2qXh/S4v2+OtOnxqnURTBukfR2z/t5lXmfqpD0ku0Ttme7LqaPmaXhxNXz2o7r\n6WfqbrPURTCyYYUlnhq7KyK+psVdvu/Z/kbXBU2poW6zVJougjEvaX3P+1slne+gjoEi4nz1vCDp\nOS3uApbm4tLdWqrnhY7ruUpEXIyIjyPiE0lPqMy/41W6CMYrkjbavs329ZIeknS4gzr6sr26uk+v\nbK+WdI/KvD3QYUm7qte7JD3fYS2pab3N0qSvrlVEfGR7t6SjklZJOhARb066jhozkp6rrra9TtJv\nI+LFLguy/aSkrZJutj0v6VFJ+yQ9bfthSW9JeqC7CvvWuNX2p26z1FmB14CebyBBzzeQIBhAgmAA\nCYIBJAgGkCAYQIJgAAmCAST+DxekFdIEhJE2AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x5891a25828>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Label 0th image is: 6\n"
     ]
    }
   ],
   "source": [
    "i=0\n",
    "image = X_train_orginal[i]\n",
    "label = y_train[i]\n",
    "plt.imshow(image, cmap='gray')\n",
    "plt.show()\n",
    "print(\"Label {}th image is: {}\".format(i, label))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div style=\"direction:rtl;text-align:right;font-family:Tahoma\">\n",
    "در کد بالا مقدار i را تغییر دهید و تصاویر را ببینید</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <div style=\"direction:rtl;text-align:right;font-family:B Lotus, B Nazanin, Tahoma\">تغییر اندازه تصاویر مجموعه داده</div>\n",
    "<div style=\"direction:rtl;text-align:right;font-family:Tahoma\">\n",
    "اندازه تصاویر مجموعه داده متفاوت است. <br>\n",
    "تمام تصاویر را به یک اندازه مشخص (5*5) تغییر اندازه می‌دهیم.\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#resize\n",
    "X_train_5by5 = [cv2.resize(img, dsize=(5, 5)) for img in X_train_orginal]\n",
    "X_test_5by_5 = [cv2.resize(img, dsize=(5, 5)) for img in X_test_original]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <div style=\"direction:rtl;text-align:right;font-family:B Lotus, B Nazanin, Tahoma\">تغییر شکل مجموعه داده</div>\n",
    "<div style=\"direction:rtl;text-align:right;font-family:Tahoma\">\n",
    "حالا داده های تصویری در مقیاس 5x5 است. برای استفاده در الگوریتم‌های خود آن ها را به یک بردار 25 تایی تغییر شکل میدهیم.\n",
    "</div>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#reshape\n",
    "X_train = [x.reshape(25) for x in X_train_5by5]\n",
    "X_test = [x.reshape(25) for x in X_test_5by_5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "<div class=\"alert alert-block alert-info\">\n",
    "<div style=\"direction:rtl;text-align:right;font-family:B Lotus, B Nazanin, Tahoma\"> دانشگاه تربیت دبیر شهید رجایی<br>مباحث ویژه - آشنایی با بینایی کامپیوتر<br>علیرضا اخوان پور<br>96-97<br>\n",
    "</div>\n",
    "<a href=\"https://www.srttu.edu/\">SRTTU.edu</a> - <a href=\"http://class.vision\">Class.Vision</a> - <a href=\"http://AkhavanPour.ir\">AkhavanPour.ir</a>\n",
    "</div>"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  },
  "nbpresent": {
   "slides": {
    "300ee14f-a043-486e-b274-7ff253907cd7": {
     "id": "300ee14f-a043-486e-b274-7ff253907cd7",
     "prev": "cb74e0bc-4513-4d13-b7f1-14c3078a7927",
     "regions": {
      "26dc3f39-a230-447c-af4c-f5e5b2fb7835": {
       "attrs": {
        "height": 0.8,
        "width": 0.8,
        "x": 0.1,
        "y": 0.1
       },
       "content": {
        "cell": "c58440a5-3f8f-4f37-9c79-6bf766209406",
        "part": "whole"
       },
       "id": "26dc3f39-a230-447c-af4c-f5e5b2fb7835"
      }
     }
    },
    "878aa53a-1444-4100-8f50-7a408191c579": {
     "id": "878aa53a-1444-4100-8f50-7a408191c579",
     "prev": null,
     "regions": {
      "a6c6843a-5ea6-4fbc-b890-3b4b8ae475b3": {
       "attrs": {
        "height": 0.8,
        "width": 0.8,
        "x": 0.1,
        "y": 0.1
       },
       "content": {
        "cell": "588ee1fa-64b5-453b-ade7-8e6b2515821c",
        "part": "whole"
       },
       "id": "a6c6843a-5ea6-4fbc-b890-3b4b8ae475b3"
      }
     }
    },
    "96ffe88e-7b50-43de-afdd-942e564f4e3e": {
     "id": "96ffe88e-7b50-43de-afdd-942e564f4e3e",
     "prev": "878aa53a-1444-4100-8f50-7a408191c579",
     "regions": {
      "b7e52e12-489a-468d-b10c-af2024fd2856": {
       "attrs": {
        "height": 0.8,
        "width": 0.8,
        "x": 0.1,
        "y": 0.1
       },
       "content": {
        "cell": "de829a92-1fb6-44ad-a2c6-fc1001e1f6e1",
        "part": "whole"
       },
       "id": "b7e52e12-489a-468d-b10c-af2024fd2856"
      }
     }
    },
    "cb74e0bc-4513-4d13-b7f1-14c3078a7927": {
     "id": "cb74e0bc-4513-4d13-b7f1-14c3078a7927",
     "prev": "96ffe88e-7b50-43de-afdd-942e564f4e3e",
     "regions": {
      "444878ee-68f3-4abb-acff-a7079b21e86d": {
       "attrs": {
        "height": 0.8,
        "width": 0.8,
        "x": 0.1,
        "y": 0.1
       },
       "content": {
        "cell": "25f3f538-1ee8-4d98-a6bb-14cbeb7a702d",
        "part": "whole"
       },
       "id": "444878ee-68f3-4abb-acff-a7079b21e86d"
      }
     }
    }
   },
   "themes": {}
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}