saiku-demo-notebook.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "70cf2ebe-2c0b-44bc-b99d-692bf5c71a5b",
   "metadata": {},
   "source": [
    "# Saiku (細工) - The AI Agent Project Setup and Execution Guide\n",
    "\n",
    "## Introduction\n",
    "\n",
    "Saiku, symbolizing \"detailed or delicate work\" in Japanese, is a venture aimed at creating a robust and intelligent AI agent. Following the PEAS framework, this project is engineered to automate various tasks efficiently. Although initiated in Tunisia, Saiku transcends geographical bounds, representing a global collaboration towards precision, innovation, and advanced technology.\n",
    "\n",
    "In this guide, we'll walk through the steps required to set up and run the Saiku project locally on your machine using Google Colab."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ead5d6f8",
   "metadata": {},
   "source": [
    "### Step 0: Setting Up Node.js Environment in Colab\n",
    "\n",
    "---\n",
    "\n",
    "#### Subsection 1: Install PixieDust and PixieDust_Node\n",
    "\n",
    "PixieDust is an extension for Jupyter Notebooks to ease data science and analytics. PixieDust_Node is an add-on to PixieDust that lets you run Node.js code cells within your notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "ca2c83f1",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: pixiedust in /opt/homebrew/lib/python3.11/site-packages (1.1.19)\n",
      "Requirement already satisfied: pixiedust_node in /opt/homebrew/lib/python3.11/site-packages (0.2.5)\n",
      "Requirement already satisfied: geojson in /opt/homebrew/lib/python3.11/site-packages (from pixiedust) (3.0.1)\n",
      "Requirement already satisfied: astunparse in /opt/homebrew/lib/python3.11/site-packages (from pixiedust) (1.6.3)\n",
      "Requirement already satisfied: markdown in /opt/homebrew/lib/python3.11/site-packages (from pixiedust) (3.4.4)\n",
      "Requirement already satisfied: colour in /opt/homebrew/lib/python3.11/site-packages (from pixiedust) (0.1.5)\n",
      "Requirement already satisfied: requests in /opt/homebrew/lib/python3.11/site-packages (from pixiedust) (2.31.0)\n",
      "Requirement already satisfied: matplotlib in /opt/homebrew/lib/python3.11/site-packages (from pixiedust) (3.8.0)\n",
      "Requirement already satisfied: pandas in /opt/homebrew/lib/python3.11/site-packages (from pixiedust) (2.1.1)\n",
      "Requirement already satisfied: ipython in /opt/homebrew/lib/python3.11/site-packages (from pixiedust_node) (8.12.0)\n",
      "Requirement already satisfied: wheel<1.0,>=0.23.0 in /opt/homebrew/lib/python3.11/site-packages (from astunparse->pixiedust) (0.41.2)\n",
      "Requirement already satisfied: six<2.0,>=1.6.1 in /opt/homebrew/lib/python3.11/site-packages (from astunparse->pixiedust) (1.16.0)\n",
      "Requirement already satisfied: backcall in /opt/homebrew/lib/python3.11/site-packages (from ipython->pixiedust_node) (0.2.0)\n",
      "Requirement already satisfied: decorator in /opt/homebrew/lib/python3.11/site-packages (from ipython->pixiedust_node) (5.1.1)\n",
      "Requirement already satisfied: jedi>=0.16 in /opt/homebrew/lib/python3.11/site-packages (from ipython->pixiedust_node) (0.18.2)\n",
      "Requirement already satisfied: matplotlib-inline in /opt/homebrew/lib/python3.11/site-packages (from ipython->pixiedust_node) (0.1.6)\n",
      "Requirement already satisfied: pickleshare in /opt/homebrew/lib/python3.11/site-packages (from ipython->pixiedust_node) (0.7.5)\n",
      "Requirement already satisfied: prompt-toolkit!=3.0.37,<3.1.0,>=3.0.30 in /opt/homebrew/lib/python3.11/site-packages (from ipython->pixiedust_node) (3.0.38)\n",
      "Requirement already satisfied: pygments>=2.4.0 in /opt/homebrew/lib/python3.11/site-packages (from ipython->pixiedust_node) (2.15.1)\n",
      "Requirement already satisfied: stack-data in /opt/homebrew/lib/python3.11/site-packages (from ipython->pixiedust_node) (0.6.2)\n",
      "Requirement already satisfied: traitlets>=5 in /opt/homebrew/lib/python3.11/site-packages (from ipython->pixiedust_node) (5.9.0)\n",
      "Requirement already satisfied: pexpect>4.3 in /opt/homebrew/lib/python3.11/site-packages (from ipython->pixiedust_node) (4.8.0)\n",
      "Requirement already satisfied: appnope in /opt/homebrew/lib/python3.11/site-packages (from ipython->pixiedust_node) (0.1.3)\n",
      "Requirement already satisfied: contourpy>=1.0.1 in /opt/homebrew/lib/python3.11/site-packages (from matplotlib->pixiedust) (1.1.1)\n",
      "Requirement already satisfied: cycler>=0.10 in /opt/homebrew/lib/python3.11/site-packages (from matplotlib->pixiedust) (0.11.0)\n",
      "Requirement already satisfied: fonttools>=4.22.0 in /opt/homebrew/lib/python3.11/site-packages (from matplotlib->pixiedust) (4.42.1)\n",
      "Requirement already satisfied: kiwisolver>=1.0.1 in /opt/homebrew/lib/python3.11/site-packages (from matplotlib->pixiedust) (1.4.5)\n",
      "Requirement already satisfied: numpy<2,>=1.21 in /opt/homebrew/lib/python3.11/site-packages (from matplotlib->pixiedust) (1.24.2)\n",
      "Requirement already satisfied: packaging>=20.0 in /opt/homebrew/lib/python3.11/site-packages (from matplotlib->pixiedust) (23.1)\n",
      "Requirement already satisfied: pillow>=6.2.0 in /opt/homebrew/lib/python3.11/site-packages (from matplotlib->pixiedust) (9.5.0)\n",
      "Requirement already satisfied: pyparsing>=2.3.1 in /opt/homebrew/lib/python3.11/site-packages (from matplotlib->pixiedust) (3.0.9)\n",
      "Requirement already satisfied: python-dateutil>=2.7 in /opt/homebrew/lib/python3.11/site-packages (from matplotlib->pixiedust) (2.8.2)\n",
      "Requirement already satisfied: pytz>=2020.1 in /opt/homebrew/lib/python3.11/site-packages (from pandas->pixiedust) (2023.3.post1)\n",
      "Requirement already satisfied: tzdata>=2022.1 in /opt/homebrew/lib/python3.11/site-packages (from pandas->pixiedust) (2023.3)\n",
      "Requirement already satisfied: charset-normalizer<4,>=2 in /opt/homebrew/lib/python3.11/site-packages (from requests->pixiedust) (3.1.0)\n",
      "Requirement already satisfied: idna<4,>=2.5 in /opt/homebrew/lib/python3.11/site-packages (from requests->pixiedust) (3.4)\n",
      "Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/homebrew/lib/python3.11/site-packages (from requests->pixiedust) (1.26.15)\n",
      "Requirement already satisfied: certifi>=2017.4.17 in /opt/homebrew/lib/python3.11/site-packages (from requests->pixiedust) (2022.12.7)\n",
      "Requirement already satisfied: parso<0.9.0,>=0.8.0 in /opt/homebrew/lib/python3.11/site-packages (from jedi>=0.16->ipython->pixiedust_node) (0.8.3)\n",
      "Requirement already satisfied: ptyprocess>=0.5 in /opt/homebrew/lib/python3.11/site-packages (from pexpect>4.3->ipython->pixiedust_node) (0.7.0)\n",
      "Requirement already satisfied: wcwidth in /opt/homebrew/lib/python3.11/site-packages (from prompt-toolkit!=3.0.37,<3.1.0,>=3.0.30->ipython->pixiedust_node) (0.2.6)\n",
      "Requirement already satisfied: executing>=1.2.0 in /opt/homebrew/lib/python3.11/site-packages (from stack-data->ipython->pixiedust_node) (1.2.0)\n",
      "Requirement already satisfied: asttokens>=2.1.0 in /opt/homebrew/lib/python3.11/site-packages (from stack-data->ipython->pixiedust_node) (2.2.1)\n",
      "Requirement already satisfied: pure-eval in /opt/homebrew/lib/python3.11/site-packages (from stack-data->ipython->pixiedust_node) (0.2.2)\n"
     ]
    }
   ],
   "source": [
    "%pip install pixiedust pixiedust_node"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "055e2140",
   "metadata": {},
   "source": [
    "#### Subsection 2: Load PixieDust_Node\n",
    "\n",
    "Load the `pixiedust_node` module to enable Node.js support in this notebook.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5eec6456",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pixiedust_node"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f88957e3",
   "metadata": {},
   "source": [
    "#### Subsection 3: Install n Globally\n",
    "\n",
    "`n` is a Node.js version management utility. Installing it globally allows us to easily switch between different Node.js versions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6c4e13ca",
   "metadata": {},
   "outputs": [],
   "source": [
    "%npm install -g n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0839ce7d",
   "metadata": {},
   "source": [
    "#### Subsection 4: Install a Stable Version of Node.js\n",
    "\n",
    "Use `n` to install a stable version of Node.js. This will help ensure compatibility and stability while running your project."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "aacae9b4",
   "metadata": {},
   "outputs": [],
   "source": [
    "%n stable"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a1b12c27",
   "metadata": {},
   "source": [
    "#### Subsection 5: Verify Node.js Installation\n",
    "\n",
    "Verify the installation by checking the Node.js version."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e8236baa",
   "metadata": {},
   "outputs": [],
   "source": [
    "%%node\n",
    "console.log(process.version)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0ba2c6a4",
   "metadata": {},
   "source": [
    "These steps create a foundation by setting up the Node.js environment in your Colab notebook, ensuring you have the necessary tools and the right Node.js version to run your project"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0a14e4f4-1c49-44fd-9ba1-dda43f80be0a",
   "metadata": {},
   "source": [
    "### Step 1: Cloning the Repository\n",
    "\n",
    "Clone the Saiku repository from GitHub to your local machine."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "4c6d7e1f-f075-410f-85fb-89911b2d228e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Cloning into 'saiku'...\n",
      "remote: Enumerating objects: 117, done.\u001b[K\n",
      "remote: Counting objects: 100% (117/117), done.\u001b[K\n",
      "remote: Compressing objects: 100% (80/80), done.\u001b[K\n",
      "remote: Total 117 (delta 47), reused 100 (delta 30), pack-reused 0\u001b[K\n",
      "Receiving objects: 100% (117/117), 109.41 KiB | 756.00 KiB/s, done.\n",
      "Resolving deltas: 100% (47/47), done.\n"
     ]
    }
   ],
   "source": [
    "!git clone https://github.com/nooqta/saiku.git"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fd43965c-2b89-49c9-bced-50b36d430e76",
   "metadata": {},
   "source": [
    "### Step 2: Navigate to the Project Folder\n",
    "\n",
    "Change the current working directory to the `saiku` directory, which contains the project files."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "14ce7e2b-9432-451b-8ca9-ec9d0a7dc93c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/Users/macbookpro/Developer/saiku/saiku\n"
     ]
    }
   ],
   "source": [
    "%cd saiku"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "acbc0374-644e-45f6-b9b9-cc015578b2ae",
   "metadata": {},
   "source": [
    "### Step 3: Installing Dependencies\n",
    "\n",
    "Install all the necessary dependencies listed in the project's `package.json` file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "b8cbb3d5-d216-443a-ac4b-114f0568c697",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[37;40mnpm\u001b[0m \u001b[0m\u001b[30;43mWARN\u001b[0m \u001b[0m\u001b[35mEBADENGINE\u001b[0m Unsupported engine {\n",
      "\u001b[0m\u001b[37;40mnpm\u001b[0m \u001b[0m\u001b[30;43mWARN\u001b[0m \u001b[0m\u001b[35mEBADENGINE\u001b[0m   package: 'got@5.7.1',\n",
      "\u001b[0m\u001b[37;40mnpm\u001b[0m \u001b[0m\u001b[30;43mWARN\u001b[0m \u001b[0m\u001b[35mEBADENGINE\u001b[0m   required: { node: '>=0.10.0 <7' },\n",
      "\u001b[0m\u001b[37;40mnpm\u001b[0m \u001b[0m\u001b[30;43mWARN\u001b[0m \u001b[0m\u001b[35mEBADENGINE\u001b[0m   current: { node: 'v18.16.0', npm: '9.5.1' }\n",
      "\u001b[0m\u001b[37;40mnpm\u001b[0m \u001b[0m\u001b[30;43mWARN\u001b[0m \u001b[0m\u001b[35mEBADENGINE\u001b[0m }\n",
      "\u001b[K\u001b[?25h\u001b[37;40mnpm\u001b[0m \u001b[0m\u001b[30;43mWARN\u001b[0m \u001b[0m\u001b[35mdeprecated\u001b[0m @types/puppeteer@7.0.4: This is a stub types definition. puppeteer provides its own type definitions, so you do not need this installed.\n",
      "\u001b[K\u001b[?25h\u001b[37;40mnpm\u001b[0m \u001b[0m\u001b[30;43mWARN\u001b[0m \u001b[0m\u001b[35mdeprecated\u001b[0m har-validator@5.1.5: this library is no longer supported\u001b[Knod\u001b[0m\u001b[K\n",
      "\u001b[K\u001b[?25h\u001b[37;40mnpm\u001b[0m \u001b[0m\u001b[30;43mWARN\u001b[0m \u001b[0m\u001b[35mdeprecated\u001b[0m j@0.4.3: The underlying libraries have been unified.  See < https://cdn.sheetjs.com/j/ >.\n",
      "\u001b[K\u001b[?25h\u001b[37;40mnpm\u001b[0m \u001b[0m\u001b[30;43mWARN\u001b[0m \u001b[0m\u001b[35mdeprecated\u001b[0m consolidate@0.15.1: Please upgrade to consolidate v1.0.0+ as it has been modernized with several long-awaited fixes implemented. Maintenance is supported by Forward Email at https://forwardemail.net ; follow/watch https://github.com/ladjs/consolidate for updates and release changelog\n",
      "\u001b[K\u001b[?25h\u001b[37;40mnpm\u001b[0m \u001b[0m\u001b[30;43mWARN\u001b[0m \u001b[0m\u001b[35mdeprecated\u001b[0m xmldom@0.1.27: Deprecated due to CVE-2021-21366 resolved in 0.5.0m\u001b[K\n",
      "\u001b[0m\u001b[37;40mnpm\u001b[0m \u001b[0m\u001b[30;43mWARN\u001b[0m \u001b[0m\u001b[35mdeprecated\u001b[0m babyparse@0.2.1: Deprecated in favour of PapaParse, which now supports Node.js\n",
      "\u001b[K\u001b[?25h\u001b[37;40mnpm\u001b[0m \u001b[0m\u001b[30;43mWARN\u001b[0m \u001b[0m\u001b[35mdeprecated\u001b[0m uuid@3.4.0: Please upgrade  to version 7 or higher.  Older versions may use Math.random() in certain circumstances, which is known to be problematic.  See https://v8.dev/blog/math-random for details.\n",
      "\u001b[K\u001b[?25h\u001b[37;40mnpm\u001b[0m \u001b[0m\u001b[30;43mWARN\u001b[0m \u001b[0m\u001b[35mdeprecated\u001b[0m request@2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142\n",
      "\u001b[0m\u001b[37;40mnpm\u001b[0m \u001b[0m\u001b[30;43mWARN\u001b[0m \u001b[0m\u001b[35mdeprecated\u001b[0m harb@0.0.7: harb has been merged into xlsx\n",
      "\u001b[K\u001b[?25h\u001b[37;40mnpm\u001b[0m \u001b[0m\u001b[30;43mWARN\u001b[0m \u001b[0m\u001b[35mdeprecated\u001b[0m xlsjs@0.7.6: xlsjs has been merged into xlsx[0m\u001b[K Compl\u001b[0m\u001b[Kd i\u001b[0m\u001b[K\n",
      "\u001b[K\u001b[?25h\u001b[37;40mnpm\u001b[0m \u001b[0m\u001b[30;43mWARN\u001b[0m \u001b[0m\u001b[35mdeprecated\u001b[0m xlsx@0.7.12: this version is no longer supported.  More info at https://cdn.sheetjs.com/xlsx/\n",
      "\u001b[K\u001b[?25hm##################\u001b[0m) ⠧ reify:typescript: \u001b[32;40mtiming\u001b[0m \u001b[35mreifyNode:node_modules/pdf-pars\u001b[0m\u001b[Km\u001b[K is\u001b[0m\u001b[K\n",
      "added 719 packages, and audited 720 packages in 7s\n",
      "\n",
      "82 packages are looking for funding\n",
      "  run `npm fund` for details\n",
      "\n",
      "\u001b[31m\u001b[1m20\u001b[22m\u001b[39m vulnerabilities (8 \u001b[33m\u001b[1mmoderate\u001b[22m\u001b[39m, 11 \u001b[31m\u001b[1mhigh\u001b[22m\u001b[39m, 1 \u001b[35m\u001b[1mcritical\u001b[22m\u001b[39m)\n",
      "\n",
      "To address issues that do not require attention, run:\n",
      "  npm audit fix\n",
      "\n",
      "To address all issues possible (including breaking changes), run:\n",
      "  npm audit fix --force\n",
      "\n",
      "Some issues need review, and may require choosing\n",
      "a different dependency.\n",
      "\n",
      "Run `npm audit` for details.\n"
     ]
    }
   ],
   "source": [
    "!npm install"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a165c38b-c5b4-4570-b87f-bede6395d4a3",
   "metadata": {},
   "source": [
    "### Step 4: Setting Up Environment Variables\n",
    "\n",
    "Copy the example environment file and manually edit the `.env` file to include your specific information for environment variables like `OPENAI_API_KEY`, `OPENAI_MODEL`, etc."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "de2828f7-c3b4-406f-8c47-f40aa3b7c51d",
   "metadata": {},
   "outputs": [],
   "source": [
    "!cp .env.example .env"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "71bda4e6-1f5b-47ee-a56d-8cf9d8f92ce6",
   "metadata": {},
   "outputs": [],
   "source": [
    "%%bash\n",
    "echo \"OPENAI_API_KEY=your-api-key-here\" >> .env\n",
    "echo \"OPENAI_MODEL=model-name-here\" >> .env\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cf2d76dd-42b8-4ea3-a867-a793aadc22b1",
   "metadata": {},
   "source": [
    "### Step 5: Building the Project Locally (Optional)\n",
    "\n",
    "Trigger the build process to compile the TypeScript code into JavaScript, making it executable."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f4c0f4f8-6a24-4a97-91b5-81015d974818",
   "metadata": {},
   "outputs": [],
   "source": [
    "!npm start"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d6a0286d-56ba-4041-9923-7f5cd0c579a9",
   "metadata": {},
   "source": [
    "### Step 8: Running with Command Line Options (Optional)\n",
    "\n",
    "Demonstrate how to start the Saiku project with specific command line options. In this example, the `-exec` flag allows code execution without prompting the user."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "dc23c922-b93a-4b78-a12c-04d7c7cb9808",
   "metadata": {},
   "outputs": [],
   "source": [
    "!npm start -- -exec"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "57be6daf-d282-4852-aa50-0e63d7d2eb49",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "This guide provides a structured pathway to set up, build, and run the Saiku project locally using Google Colab. Ensure to save your Colab notebook frequently to preserve your work. For any further assistance or information regarding the project, feel free to refer to the [Saiku GitHub repository](https://github.com/nooqta/saiku).\n",
    "\n",
    "--- \n",
    "\n",
    "By following these steps in sequence, you'll have a local instance of Saiku up and running on your machine, ready for interaction or further development."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}