Full-Example.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## MLQuest Quick Start"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Some Example Functions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Suppose we have any set of functions such as the following and we interesed in running a Python file or notebook that calls them multiple times with different arguments and we would like to keep track of the arguments that they take and their corresponding outputs for each run. This problem is exactly what this package attempts to solve.\n",
    "\n",
    "In this example, we will try to demonstrate the high-level features of the package. For a more detailed explanation of the package, please refer to the [documentation](https://essamwisam.github.io/MLPath/mlpath.html). You can also run this notebook firsthand by simply downloading it from the Github repo [here](https://github.com/EssamWisam/MLPath/blob/main/Example.ipynb)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "### Define some functions with default and non-default parameters\n",
    "def Preprocessing(x_data, alpha, beta_param, c=0, \n",
    "                  depth_ratio=4, a_num=5, b_gum=7, c_hum=12):\n",
    "    return [2, 3, 4]\n",
    "\n",
    "def FeatureExtraction(x_data, x_param, y_param, \n",
    "                      z_param, a_num=5, b_gum=7, c_hum=12, **kwargs):\n",
    "    return [3, 4, 5]\n",
    "\n",
    "def RadialBasisNet(x_data, p_num, k_num, l_num, \n",
    "                   a_num=5, b_gum=7, c_hum=12, **kwargs):\n",
    "    return None\n",
    "\n",
    "def train_model(model):\n",
    "    return 701"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Imports"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The main functions responsible for logging functions is simply called `l` and is defined under the class `mlquest`. You can ignore the second line and use `mlq.l` to log your functions. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "from mlpath import mlquest as mlq\n",
    "l = mlq.l"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Starting a Quest"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A quest simply corresponds to a table where each row is an experiment corresponding to a run of the file. When you call `mlq.start_quest` it simply creates an empty table or loads it if it was created already already."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Start a new quest, this corresponds to a table where every run will be logged.\n",
    "mlq.start_quest('Radial Basis Pipeline')     "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Core Pipeline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Suppose the following is our pipeline, we can log the input to each function by simple using `l(func)(args)` instead of `func(args)`. This logs the arguments regardless to whether they are given by name or not and even if they are default arguments. Note that non-scalar arguments are always ignored by `l()`. We will later see how the log looks like."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Preprocessing\n",
    "x_data_p = l(Preprocessing)(x_data=[1, 2, 3], alpha=14, beta_param=9)\n",
    "\n",
    "# Feature Extraction\n",
    "x_data_f = l(FeatureExtraction)(x_data_p, 20, 340, 10)\n",
    "\n",
    "# Model Initialization\n",
    "model = l(RadialBasisNet)(x_data_f, 55, 25, 26)\n",
    "\n",
    "# Model Training\n",
    "accuracy = train_model(model)  + 15"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Model Evaluation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To log a metric we use the `log_metrics` function. If you pass it variables with scalar context directly then a column will be created with that variable's name to log a variable with a different name you can pass it as a keyword argument.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# log the accuracy\n",
    "mlq.log_metrics(accuracy)        # can also do mlq.log_metric(acc=accuracy) if want to log it as acc"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Extra Logging"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Suppose we have another piece of information that we would like to log. We can use the `to_log` function. The first argument it takes is the column header and the rest are any number of key value pairs passed as keyword arguments where a column under the column header will be created for each key."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "mlq.to_log('New Column', User=\"Malzahar\", Ult=\"Yes\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Save and Display Logs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once the quest is ended with `mlq.end_quest`, the run is converted to a row and logged to a table and saved. The first argument is where we would like to save the markdown corresponding to the quest, the second argument specifies whether to show or hide the default arguments of functions and the third helps blacklist further arguments"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "mlq.end_quest('./', log_defs=True, blacklist=['alpha'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can use `mlq.show_logs` to show the logs of the last `last_k` below the notebook cell that calls it. To see it, please consider running the notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<tr>\n",
       "<th colspan=4 style=\"text-align: center; vertical-align: middle;\">info</th>\n",
       "<th colspan=6 style=\"text-align: center; vertical-align: middle;\">Preprocessing</th>\n",
       "<th colspan=6 style=\"text-align: center; vertical-align: middle;\">FeatureExtraction</th>\n",
       "<th colspan=6 style=\"text-align: center; vertical-align: middle;\">RadialBasisNet</th>\n",
       "<th colspan=1 style=\"text-align: center; vertical-align: middle;\">metrics</th>\n",
       "<th colspan=2 style=\"text-align: center; vertical-align: middle;\">New Column</th>\n",
       "</tr>\n",
       "<th style=\"text-align: center; vertical-align: middle;\">time</th>\n",
       "<th style=\"text-align: center; vertical-align: middle;\">date</th>\n",
       "<th style=\"text-align: center; vertical-align: middle;\">duration</th>\n",
       "<th style=\"text-align: center; vertical-align: middle;\">id</th>\n",
       "<th style=\"text-align: center; vertical-align: middle;\">beta_param</th>\n",
       "<th style=\"text-align: center; vertical-align: middle;\">c</th>\n",
       "<th style=\"text-align: center; vertical-align: middle;\">depth_ratio</th>\n",
       "<th style=\"text-align: center; vertical-align: middle;\">a_num</th>\n",
       "<th style=\"text-align: center; vertical-align: middle;\">b_gum</th>\n",
       "<th style=\"text-align: center; vertical-align: middle;\">c_hum</th>\n",
       "<th style=\"text-align: center; vertical-align: middle;\">x_param</th>\n",
       "<th style=\"text-align: center; vertical-align: middle;\">y_param</th>\n",
       "<th style=\"text-align: center; vertical-align: middle;\">z_param</th>\n",
       "<th style=\"text-align: center; vertical-align: middle;\">a_num</th>\n",
       "<th style=\"text-align: center; vertical-align: middle;\">b_gum</th>\n",
       "<th style=\"text-align: center; vertical-align: middle;\">c_hum</th>\n",
       "<th style=\"text-align: center; vertical-align: middle;\">p_num</th>\n",
       "<th style=\"text-align: center; vertical-align: middle;\">k_num</th>\n",
       "<th style=\"text-align: center; vertical-align: middle;\">l_num</th>\n",
       "<th style=\"text-align: center; vertical-align: middle;\">a_num</th>\n",
       "<th style=\"text-align: center; vertical-align: middle;\">b_gum</th>\n",
       "<th style=\"text-align: center; vertical-align: middle;\">c_hum</th>\n",
       "<th style=\"text-align: center; vertical-align: middle;\">accuracy</th>\n",
       "<th style=\"text-align: center; vertical-align: middle;\">User</th>\n",
       "<th style=\"text-align: center; vertical-align: middle;\">Ult</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>01:42:40</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>08/14/23</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>110.65 ms</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>1</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>9</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>0</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>4</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>5</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>7</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>12</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>20</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>340</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>10</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>5</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>7</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>12</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>55</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>25</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>26</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>5</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>7</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>12</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>716</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>Malzahar</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>Yes</font></td>\n",
       "</tr>\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "mlq.show_logs(last_k=5)                     # higlight color assumes dark theme and can be changed"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Server"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also view the table in a server using `mlq.run_server`. There further filtering can be easily done to show specific rows of the table."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " * Serving Flask app 'mlpath.mldir_cli.web.app'\n",
      " * Debug mode: off\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[31m\u001b[1mWARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.\u001b[0m\n",
      " * Running on http://127.0.0.1:5000\n",
      "\u001b[33mPress CTRL+C to quit\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "mlq.run_server()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Deleting Runs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Deleting runs using their id is easily possible using `mlq.delete_runs`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<tr>\n",
       "<th colspan=4 style=\"text-align: center; vertical-align: middle;\">info</th>\n",
       "<th colspan=6 style=\"text-align: center; vertical-align: middle;\">Preprocessing</th>\n",
       "<th colspan=6 style=\"text-align: center; vertical-align: middle;\">FeatureExtraction</th>\n",
       "<th colspan=6 style=\"text-align: center; vertical-align: middle;\">RadialBasisNet</th>\n",
       "<th colspan=1 style=\"text-align: center; vertical-align: middle;\">metrics</th>\n",
       "<th colspan=2 style=\"text-align: center; vertical-align: middle;\">New Column</th>\n",
       "</tr>\n",
       "<th style=\"text-align: center; vertical-align: middle;\">time</th>\n",
       "<th style=\"text-align: center; vertical-align: middle;\">date</th>\n",
       "<th style=\"text-align: center; vertical-align: middle;\">duration</th>\n",
       "<th style=\"text-align: center; vertical-align: middle;\">id</th>\n",
       "<th style=\"text-align: center; vertical-align: middle;\">beta_param</th>\n",
       "<th style=\"text-align: center; vertical-align: middle;\">c</th>\n",
       "<th style=\"text-align: center; vertical-align: middle;\">depth_ratio</th>\n",
       "<th style=\"text-align: center; vertical-align: middle;\">a_num</th>\n",
       "<th style=\"text-align: center; vertical-align: middle;\">b_gum</th>\n",
       "<th style=\"text-align: center; vertical-align: middle;\">c_hum</th>\n",
       "<th style=\"text-align: center; vertical-align: middle;\">x_param</th>\n",
       "<th style=\"text-align: center; vertical-align: middle;\">y_param</th>\n",
       "<th style=\"text-align: center; vertical-align: middle;\">z_param</th>\n",
       "<th style=\"text-align: center; vertical-align: middle;\">a_num</th>\n",
       "<th style=\"text-align: center; vertical-align: middle;\">b_gum</th>\n",
       "<th style=\"text-align: center; vertical-align: middle;\">c_hum</th>\n",
       "<th style=\"text-align: center; vertical-align: middle;\">p_num</th>\n",
       "<th style=\"text-align: center; vertical-align: middle;\">k_num</th>\n",
       "<th style=\"text-align: center; vertical-align: middle;\">l_num</th>\n",
       "<th style=\"text-align: center; vertical-align: middle;\">a_num</th>\n",
       "<th style=\"text-align: center; vertical-align: middle;\">b_gum</th>\n",
       "<th style=\"text-align: center; vertical-align: middle;\">c_hum</th>\n",
       "<th style=\"text-align: center; vertical-align: middle;\">accuracy</th>\n",
       "<th style=\"text-align: center; vertical-align: middle;\">User</th>\n",
       "<th style=\"text-align: center; vertical-align: middle;\">Ult</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>19:48:05</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>08/13/23</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>245.20 ms</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>1</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>5</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>0</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>4</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>5</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>7</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>12</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>10</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>1340</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>20</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>5</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>7</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>12</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>10</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>30</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>121</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>5</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>7</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>12</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>721</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>Malzahar</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>Yes</font></td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>19:49:06</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>08/13/23</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>82.53 ms</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>4</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=yellow>9</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>0</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>4</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>5</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>7</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>12</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=yellow>20</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=yellow>340</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=yellow>10</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>5</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>7</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>12</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=yellow>55</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=yellow>20</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=yellow>21</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>5</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>7</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>12</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=yellow>712</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>Malzahar</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>Yes</font></td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>19:49:13</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>08/13/23</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>69.35 ms</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>5</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>9</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>0</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>4</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>5</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>7</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>12</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>20</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>340</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>10</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>5</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>7</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>12</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>55</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=yellow>25</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=yellow>26</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>5</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>7</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>12</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=yellow>716</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>Malzahar</font></td>\n",
       "<td style=\"text-align: center; vertical-align: middle;\"> <font color=>Yes</font></td>\n",
       "</tr>\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "mlq.delete_runs([2,3])\n",
    "mlq.show_logs(last_k=9)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Converting to DF"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The table can be easily converted to a pandas DataFrame to later be used for plotting or other analysis. The key step here is to use `mlq.get_flat_dict()` to get the equivalent flat dictionary."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>time</th>\n",
       "      <th>date</th>\n",
       "      <th>duration</th>\n",
       "      <th>id</th>\n",
       "      <th>beta_param</th>\n",
       "      <th>c</th>\n",
       "      <th>depth_ratio</th>\n",
       "      <th>a_num</th>\n",
       "      <th>b_gum</th>\n",
       "      <th>c_hum</th>\n",
       "      <th>x_param</th>\n",
       "      <th>y_param</th>\n",
       "      <th>z_param</th>\n",
       "      <th>p_num</th>\n",
       "      <th>k_num</th>\n",
       "      <th>l_num</th>\n",
       "      <th>accuracy</th>\n",
       "      <th>User</th>\n",
       "      <th>Ult</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>19:48:05</td>\n",
       "      <td>08/13/23</td>\n",
       "      <td>245.20 ms</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>5</td>\n",
       "      <td>7</td>\n",
       "      <td>12</td>\n",
       "      <td>10</td>\n",
       "      <td>1340</td>\n",
       "      <td>20</td>\n",
       "      <td>10</td>\n",
       "      <td>30</td>\n",
       "      <td>121</td>\n",
       "      <td>721</td>\n",
       "      <td>Malzahar</td>\n",
       "      <td>Yes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>19:49:06</td>\n",
       "      <td>08/13/23</td>\n",
       "      <td>82.53 ms</td>\n",
       "      <td>4</td>\n",
       "      <td>9</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>5</td>\n",
       "      <td>7</td>\n",
       "      <td>12</td>\n",
       "      <td>20</td>\n",
       "      <td>340</td>\n",
       "      <td>10</td>\n",
       "      <td>55</td>\n",
       "      <td>20</td>\n",
       "      <td>21</td>\n",
       "      <td>712</td>\n",
       "      <td>Malzahar</td>\n",
       "      <td>Yes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>19:49:13</td>\n",
       "      <td>08/13/23</td>\n",
       "      <td>69.35 ms</td>\n",
       "      <td>5</td>\n",
       "      <td>9</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>5</td>\n",
       "      <td>7</td>\n",
       "      <td>12</td>\n",
       "      <td>20</td>\n",
       "      <td>340</td>\n",
       "      <td>10</td>\n",
       "      <td>55</td>\n",
       "      <td>25</td>\n",
       "      <td>26</td>\n",
       "      <td>716</td>\n",
       "      <td>Malzahar</td>\n",
       "      <td>Yes</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       time      date   duration  id beta_param  c depth_ratio a_num b_gum  \\\n",
       "0  19:48:05  08/13/23  245.20 ms   1          5  0           4     5     7   \n",
       "1  19:49:06  08/13/23   82.53 ms   4          9  0           4     5     7   \n",
       "2  19:49:13  08/13/23   69.35 ms   5          9  0           4     5     7   \n",
       "\n",
       "  c_hum x_param y_param z_param p_num k_num l_num accuracy      User  Ult  \n",
       "0    12      10    1340      20    10    30   121      721  Malzahar  Yes  \n",
       "1    12      20     340      10    55    20    21      712  Malzahar  Yes  \n",
       "2    12      20     340      10    55    25    26      716  Malzahar  Yes  "
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "\n",
    "my_dict = mlq.get_flat_dict()\n",
    "df = pd.DataFrame.from_dict(my_dict)\n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[NbConvertApp] Converting notebook Full-Example.ipynb to rst\n",
      "[NbConvertApp] Writing 21876 bytes to Full-Example.rst\n"
     ]
    }
   ],
   "source": [
    "!jupyter nbconvert --to rst Full-Example.ipynb"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "base",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.16"
  },
  "orig_nbformat": 4,
  "vscode": {
   "interpreter": {
    "hash": "40d3a090f54c6569ab1632332b64b2c03c39dcf918b08424e98f38b5ae0af88f"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}