notebooks/LangChain.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "368dccec-1d58-4167-a50d-01c56d98c24e",
   "metadata": {},
   "source": [
    "<img src=\"../images/coefficient-aidl.png\" width=1200>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "db52fb18-e4c3-4d38-86d6-cf10abeca5e8",
   "metadata": {},
   "source": [
    "# Build Your Own Private ChatGPT Super-Assistant Using Streamlit, LangChain, Chroma & Llama 2\n",
    "## LangChain Demo\n",
    "**Questions?** contact@coefficient.ai / [@CoefficientData](https://twitter.com/CoefficientData)\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "af9dfd7f-dcc6-418c-bc6b-02c7771e07fa",
   "metadata": {},
   "source": [
    "## 0. Imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d73809ac-30df-4182-8d69-e49d9ccc60b0",
   "metadata": {},
   "outputs": [],
   "source": [
    "import random\n",
    "\n",
    "from dotenv import load_dotenv\n",
    "\n",
    "from utils import scrape_page\n",
    "\n",
    "import warnings\n",
    "\n",
    "warnings.simplefilter(\"ignore\", UserWarning)\n",
    "\n",
    "load_dotenv();"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c8a2f7f0-8d61-4feb-b4c7-6cfff3c7cf9b",
   "metadata": {},
   "source": [
    "## 1. LangChain Basics"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "76396764-2c53-4653-a085-6ad0c5a784e7",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_openai import OpenAI"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e0c3a224-ad9e-4fc9-abef-1a731af16952",
   "metadata": {},
   "outputs": [],
   "source": [
    "llm = OpenAI(model=\"gpt-3.5-turbo-instruct\", temperature=0.9)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "99a511fe-afa5-417b-869a-cebda93113f5",
   "metadata": {},
   "outputs": [],
   "source": [
    "food1 = llm.predict(\"Give me a random food ingredient.\").strip().lower()\n",
    "food2 = llm.predict(\"Give me a random food ingredient.\").strip().lower()\n",
    "colour = llm.predict(\"Give me a random colour.\").strip().lower()\n",
    "\n",
    "prompt = f\"\"\"\n",
    "Write a recipe for a {colour} cocktail featuring {food1} and {food2}.\n",
    "Please come up with an fun name for the mocktail.\n",
    "\"\"\"\n",
    "print(prompt)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a31743ec-2003-4917-a31c-a36271f2eafc",
   "metadata": {},
   "outputs": [],
   "source": [
    "cocktail = llm.predict(prompt).strip()\n",
    "print(cocktail)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dc892754-8ee8-4ab2-ac95-590cce5f23c4",
   "metadata": {},
   "source": [
    "## 2. Chatting with LangChain 🦜"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "acead2b3-ad7b-45d5-bc27-e5b0fefdf0e6",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_openai import OpenAI\n",
    "\n",
    "llm = OpenAI(temperature=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c5b849c9-213d-46ca-88a9-81b9d6df32ff",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain import PromptTemplate\n",
    "\n",
    "template = \"\"\"\n",
    "You are a mindreader with magical abilities.\n",
    "You are very over-dramatic and speak like a mysterious shaman.\n",
    "You will be given something to guess, such as an animal, or a famous person.\n",
    "You will ask a question, I will provide an answer, and then you will ask another question.\n",
    "Try to ask questions that narrow down what the answer might be.\n",
    "If you are very confident, you can guess what it is.\n",
    "If your guess is wrong, then you must ask another question to help narrow it down.\n",
    "Repeat this until you I tell you that you have the right answer.\n",
    "\n",
    "{history}\n",
    "Human: {human_input}\n",
    "Assistant:\"\"\"\n",
    "\n",
    "prompt = PromptTemplate(input_variables=[\"history\", \"human_input\"], template=template)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "254210fc-057c-4e23-87cd-fdeebccafa80",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.memory import ConversationBufferWindowMemory\n",
    "\n",
    "short_memory = ConversationBufferWindowMemory(k=5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f49dfd50-ca44-4dc2-b14d-70b1c75231b0",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain import LLMChain\n",
    "\n",
    "chatgpt_chain = LLMChain(\n",
    "    llm=llm,\n",
    "    prompt=prompt,\n",
    "    verbose=True,\n",
    "    memory=short_memory,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "620f59f4-5d00-4890-95f4-6a69422b60ce",
   "metadata": {},
   "outputs": [],
   "source": [
    "chatgpt_chain.predict(human_input=\"I'm thinking of an animal.\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e21ee9e5-ef8c-4481-80b7-875b68070f6b",
   "metadata": {},
   "outputs": [],
   "source": [
    "chatgpt_chain.predict(human_input=\"It is not a mammal. Ask another question.\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "08bc72d5-1e51-4d0e-a62f-63ae295627e6",
   "metadata": {},
   "outputs": [],
   "source": [
    "chatgpt_chain.predict(human_input=\"Yes its a bird. Either guess, or ask another question.\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9719a278-5f85-4ec4-a307-3f3d72fa0c5f",
   "metadata": {},
   "outputs": [],
   "source": [
    "chatgpt_chain.predict(human_input=\"Yes it's a parrot. Either guess, or ask another question.\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ad4c3814-d967-43f5-8b5f-0c247ec149c2",
   "metadata": {},
   "outputs": [],
   "source": [
    "chatgpt_chain.predict(human_input=\"Yes it's a macaw! Would you like to give it a name?\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5d73bfa9-637c-4cdf-98ed-cee7638c7c19",
   "metadata": {},
   "outputs": [],
   "source": [
    "chatgpt_chain.predict(human_input=\"How about something a little more creative and fun?\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "382f5f30-c4cd-4480-92f9-a9a602ca290a",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "558e007e-76fd-4c8d-800c-010088701237",
   "metadata": {},
   "source": [
    "## 3. Llama-2 (via llama-cpp) 🦙\n",
    "\n",
    "### LLM ➡️ LLaMA 🦙 ➡️ Vicuna 🦙 ➡️ Llama-2\n",
    "See here for a macOS install guide: https://abetlen.github.io/llama-cpp-python/macos_install/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ec0feacc-6455-45ef-a6ee-02092fb56fa0",
   "metadata": {},
   "outputs": [],
   "source": [
    "from llama_cpp import Llama"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cf9952cb-21e7-4f6a-9e36-ad17933f0971",
   "metadata": {},
   "outputs": [],
   "source": [
    "%pwd\n",
    "# Where are we?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cdb86a28-7333-41a3-a954-2f207b691515",
   "metadata": {},
   "outputs": [],
   "source": [
    "llm = Llama(model_path=\"../models/llama-2-7b-chat.Q4_K_M.gguf\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "80761117-0861-4ca1-ad01-db2febd66024",
   "metadata": {},
   "outputs": [],
   "source": [
    "output = llm(\n",
    "    \"Q: Name five UK government departments? A: \", max_tokens=64, stop=[\"Q:\", \"\\n\"], echo=True\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "87ca5287-5b6d-497d-b02d-e20e2e1beff6",
   "metadata": {},
   "outputs": [],
   "source": [
    "output"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d56b72a7-aec6-4b2d-a00a-14817d228bb3",
   "metadata": {},
   "outputs": [],
   "source": [
    "output[\"choices\"][0][\"text\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e2b29744-5eea-44a8-9852-e65290fa05a6",
   "metadata": {},
   "outputs": [],
   "source": [
    "llm(\n",
    "    \"Q: What's the least known UK government department or agency? A: \",\n",
    "    max_tokens=128,\n",
    "    stop=[\"Q:\"],\n",
    "    echo=True,\n",
    ")[\"choices\"][0][\"text\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e9d75b70-dab5-460d-a1ec-d0a2ece129c3",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1f074c53-23ab-4f09-a32a-8a928b3f4f38",
   "metadata": {},
   "source": [
    "## 4. Llama-2 + LangChain"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a4e36637-61e9-4e16-babd-d0aec19dd58e",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.llms import LlamaCpp\n",
    "from langchain import PromptTemplate, LLMChain\n",
    "from langchain.callbacks.manager import CallbackManager\n",
    "from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "de7f26a7-8205-4098-a817-974017fa3443",
   "metadata": {},
   "source": [
    "### Define a LlamaCpp llm object"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "967887fe-deab-4826-83e9-8478b695f7a9",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Callbacks support token-wise streaming\n",
    "callback_manager = CallbackManager([StreamingStdOutCallbackHandler()])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c95049a5-f8f7-4bbc-9959-31341a721210",
   "metadata": {},
   "outputs": [],
   "source": [
    "n_gpu_layers = 40  # Change this value based on your model and your GPU VRAM pool.\n",
    "n_batch = 512  # Should be between 1 and n_ctx, consider the amount of VRAM in your GPU.\n",
    "n_ctx = 512 * 2\n",
    "\n",
    "# Make sure the model path is correct for your system!\n",
    "llm = LlamaCpp(\n",
    "    n_ctx=n_ctx,\n",
    "    model_path=\"../models/llama-2-7b-chat.Q4_K_M.gguf\",\n",
    "    n_gpu_layers=n_gpu_layers,\n",
    "    n_batch=n_batch,\n",
    "    callback_manager=callback_manager,\n",
    "    verbose=True,\n",
    "    model_kwargs={\"max_new_tokens\": 64 * 4},\n",
    "    stop=[\"Human:\", \"Input:\"],\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "811658c8-860d-45cf-ba05-6ee48066567a",
   "metadata": {},
   "source": [
    "### Use the LlamaCpp llm object with LangChain's PromptTemplate"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "690ca30a-70de-45da-901c-d8234bfc0f6d",
   "metadata": {},
   "outputs": [],
   "source": [
    "template = \"\"\"Question: {question}\n",
    "\n",
    "Answer: Let's work this out in a step by step way to be sure we have the right answer.\"\"\"\n",
    "\n",
    "prompt = PromptTemplate(template=template, input_variables=[\"question\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "dedc0889-46f4-455d-9e2f-4058fd0cf232",
   "metadata": {},
   "outputs": [],
   "source": [
    "%%time\n",
    "llm_chain = LLMChain(prompt=prompt, llm=llm)\n",
    "question = \"What is a vicuna? (Clue, the animal!)\"\n",
    "\n",
    "print(llm_chain.run(question))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3f967839-abd4-474f-a45d-46e51a014818",
   "metadata": {},
   "source": [
    "### Use a ConversationBufferWindowMemory"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ac4fa27e-6246-4fe3-b0ec-90674088a28c",
   "metadata": {},
   "outputs": [],
   "source": [
    "# This is the exact same code we saw before using the OpenAI LLM.\n",
    "# LangChain gives us code consistency across LLM models.\n",
    "\n",
    "chatgpt_chain = LLMChain(\n",
    "    llm=llm,\n",
    "    prompt=prompt,\n",
    "    verbose=True,\n",
    "    memory=ConversationBufferWindowMemory(k=5),\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fe448a8f-5760-4fc9-a5a0-af35308252d8",
   "metadata": {},
   "outputs": [],
   "source": [
    "chatgpt_chain.predict(\n",
    "    question=\"What NFL team won the Super Bowl in the year Justin Bieber was born?\"\n",
    ");"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e9ec07ef-4b73-4dd8-b18f-ec614becef65",
   "metadata": {},
   "source": [
    "### Mind-reading comparison: Llama2 (7B) vs OpenAI"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "490688d6-3e90-429d-9fa9-d7d0f4aa2122",
   "metadata": {},
   "outputs": [],
   "source": [
    "template = \"\"\"\n",
    "You are a mindreader with magical abilities.\n",
    "You are very over-dramatic and speak like a mysterious shaman.\n",
    "You will be given something to guess, such as an animal, or a famous person.\n",
    "You will ask a question, I will provide an answer, and then you will ask another question.\n",
    "Try to ask questions that narrow down what the answer might be.\n",
    "If you are very confident, you can guess what it is.\n",
    "If your guess is wrong, then you must ask another question to help narrow it down.\n",
    "Repeat this until you I tell you that you have the right answer.\n",
    "\n",
    "{history}\n",
    "Human: {human_input}\n",
    "Assistant:\"\"\"\n",
    "\n",
    "prompt = PromptTemplate(input_variables=[\"history\", \"human_input\"], template=template)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8b5f501e-c6fe-45b1-be10-73bed4b3047f",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Let's turn verbose off, we don't need llama_print_timings\n",
    "chatgpt_chain = LLMChain(\n",
    "    llm=llm,\n",
    "    prompt=prompt,\n",
    "    verbose=False,\n",
    "    memory=ConversationBufferWindowMemory(k=10),\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "dc8f234e-3e10-4988-971e-8b6415929cd9",
   "metadata": {},
   "outputs": [],
   "source": [
    "chatgpt_chain.predict(human_input=\"I'm thinking of an animal.\");"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7eb18396-9cd9-401c-88c8-ca165681089b",
   "metadata": {},
   "outputs": [],
   "source": [
    "chatgpt_chain.predict(human_input=\"No, its an animal.\");"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "20e95071-d7db-4e90-b721-0c76a482b35d",
   "metadata": {},
   "outputs": [],
   "source": [
    "chatgpt_chain.predict(human_input=\"No\");"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "11499b05-0a43-4c1c-96d2-7095099d4371",
   "metadata": {},
   "outputs": [],
   "source": [
    "chatgpt_chain.predict(human_input=\"Yes\");"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "272390d0-0a35-4f34-94d6-3b95e4ce7f08",
   "metadata": {},
   "outputs": [],
   "source": [
    "chatgpt_chain.predict(human_input=\"Yes, it's a bird. Can you guess the bird?\");"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f695a1af-7275-40e2-a727-6a86a9abe7b4",
   "metadata": {},
   "outputs": [],
   "source": [
    "chatgpt_chain.predict(human_input=\"Yes. Try asking a question that would narrow it down?\");"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7f664d09-6358-4360-b8c5-5cb010457c7a",
   "metadata": {},
   "outputs": [],
   "source": [
    "chatgpt_chain.predict(human_input=\"Yes. Can you guess what kind of bird it is?\");"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4a2d61c2-0880-47f9-aa6b-e7568441c280",
   "metadata": {},
   "outputs": [],
   "source": [
    "chatgpt_chain.predict(human_input=\"Maybe. Perhaps you can ask where it lives?\");"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fbd707b3-2a69-4a15-a986-5febeea65c10",
   "metadata": {},
   "outputs": [],
   "source": [
    "chatgpt_chain.predict(human_input=\"Yes, it lives in the jungle.\");"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0f5d22e1-6528-4c4b-bd77-f8e47387def9",
   "metadata": {},
   "outputs": [],
   "source": [
    "chatgpt_chain.predict(human_input=\"Yes! Would you like to give it a name?\");"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f2fdac5e-5da6-448f-8f9b-0236fa38e099",
   "metadata": {},
   "outputs": [],
   "source": [
    "chatgpt_chain.predict(human_input=\"Can you suggest something a little more creative and fun?\");"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "86b25439-6393-4755-94f3-71e8e645519c",
   "metadata": {},
   "outputs": [],
   "source": [
    "chatgpt_chain.predict(human_input=\"I love it. Can you summarise this conversation using emoji?\");"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f8646efa-78bd-4ef8-baee-31aff6165fa7",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f3b2a2e3-c1a4-41ab-9541-3f04d8cd4bb7",
   "metadata": {},
   "source": [
    "## 5. Exercise: Adapt your OpenAI chatbot to use LangChain + Llama-2\n",
    "\n",
    "> In the last notebook, you created a chatbot \"application\" using the `openai` library. You should be able to adapt this to use LangChain + Llama-2 with a little effort."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c27e74d5-fde3-4e0a-b4bd-e61b4d41079a",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python (chatefficient)",
   "language": "python",
   "name": "chatefficient"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}