hub-examples/retrieval-qa-chain/retrieval-qa.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "6f4fd289-fd25-40b8-a503-f12c170d8b81",
   "metadata": {},
   "source": [
    "# RetrievalQA Chain\n",
    "[![Open In Collab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/langchain-ai/langsmith-cookbook/blob/main/hub-examples/retrieval-qa-chain/retrieval-qa.ipynb)\n",
    "\n",
    "Developing a production-grade LLM application requires many refinements, but tracking multiple versions of prompts, models, and other components can be cumbersome. The [LangChain Hub](https://smith.langchain.com/hub) offers a centralized registry to manage and version your LLM artifacts efficiently. It even lets you interact with these artifacts directly in the browser to facilitate easier collaboration with non-technical team members.\n",
    "\n",
    "[![Playground](./img/playground.png)](https://smith.langchain.com/hub/rlm/rag-prompt/playground)\n",
    "\n",
    "In its initial release (08/05/2023), the hub is limited to prompt management, but we plan to add support for other artifacts soon.\n",
    "\n",
    "In this walkthrough, you will get started using the hub to manage prompts for a retrieval QA chain. You will go through the following steps:\n",
    "\n",
    "1. Load prompt from Hub\n",
    "2. Initialize Chain\n",
    "3. Run Chain\n",
    "4. Commit any new changes to the hub\n",
    "\n",
    "## Prerequsites\n",
    "\n",
    "#### a. Set up your LangSmith account\n",
    "\n",
    "While you can access public prompts without an account, pushing new prompts to the hub requires a LangSmith account. Create your account at https://smith.langchain.com and log in.\n",
    "\n",
    "Next, navigate to the [hub home](https://smith.langchain.com/hub). If you haven't already created a \"handle\", you will be prompted to do so. Your prompts and other artifacts will be saved within the namespace '<handle>/prompt-name', so choose one that you are comfortable sharing.\n",
    "\n",
    "#### b. Configure environment\n",
    "\n",
    "To use the hub, you'll want to use a recent version of LangChain and the `langchainhub` package. Install them with the following command:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "bfa759ba-d354-4e75-a158-bd1481ad82c1",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip available: \u001b[0m\u001b[31;49m22.3.1\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m23.2.1\u001b[0m\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n",
      "Note: you may need to restart the kernel to use updated packages.\n"
     ]
    }
   ],
   "source": [
    "%pip install -U langchain langchainhub --quiet"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f8fa970d-c3b8-4288-a6f9-a946bcb75a43",
   "metadata": {},
   "source": [
    "Finally, generate an API Key from your \"personal\" organization by navigating to the [LangSmith](https://smith.langchain.com) dashboard, and then set it in the cell below.\n",
    "\n",
    "**Note:** Currently (08/04/2023), only API keys from your 'personal' organization are supported! If you see a '403' error at any point in this walkthrough, please confirm you've set a valid API key."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "973cda39-bfed-4172-910d-395c8596ed89",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "# Update with your API URL if using a hosted instance of Langsmith.\n",
    "os.environ[\"LANGCHAIN_ENDPOINT\"] = \"https://api.smith.langchain.com\"\n",
    "os.environ[\"LANGCHAIN_API_KEY\"] = \"YOUR API KEY\"  # Update with your API key\n",
    "# Update with your API URL if using a hosted instance of Langsmith.\n",
    "os.environ[\"LANGCHAIN_HUB_API_URL\"] = \"https://api.hub.langchain.com\"\n",
    "os.environ[\"LANGCHAIN_HUB_API_KEY\"] = \"YOUR API KEY\"  # Update with your Hub API key"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b6aea7ba-7e86-4958-9314-cfc563f3c08c",
   "metadata": {},
   "source": [
    "## 1. Load prompt\n",
    "\n",
    "Now it's time to load the prompt from the hub. We will use the `latest` version of [this retrieval QA prompt](https://smith.langchain.com/hub/rlm/rag-prompt) and later initialize the chain with it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "88785ea3-ae64-49e1-b496-3970f5e6eb42",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# RAG prompt\n",
    "from langchain import hub\n",
    "\n",
    "# Loads the latest version\n",
    "prompt = hub.pull(\"rlm/rag-prompt\", api_url=\"https://api.hub.langchain.com\")\n",
    "\n",
    "# To load a specific version, specify the version hash\n",
    "# prompt = hub.pull(\"rlm/rag-prompt:50442af1\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d8095257-bbbb-4146-bbc3-9371e163ee45",
   "metadata": {},
   "source": [
    "## 2. Create the QA chain\n",
    "\n",
    "Now that we've selected our prompt, initialize the chain.\n",
    " For this example, we will create a basic [RetrievalQA](https://api.python.langchain.com/en/latest/chains/langchain.chains.retrieval_qa.base.RetrievalQA.html?highlight=retrievalqa#langchain.chains.retrieval_qa.base.RetrievalQA) over a vectorstore retriever. \n",
    "\n",
    "Loading the data requires some amount of boilerplate, which we will run below.  While the specifics aren't important to this tutorial, you can learn more about Q&A in LangChain by visiting the [docs](https://python.langchain.com/docs/use_cases/question_answering/)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "da0abdda-9766-4b08-8285-1cf803431656",
   "metadata": {},
   "outputs": [],
   "source": [
    "# %pip install chromadb --quiet"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "274eec26-4b92-445f-9130-fda195566cee",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Load docs\n",
    "from langchain.document_loaders import WebBaseLoader\n",
    "\n",
    "loader = WebBaseLoader(\"https://lilianweng.github.io/posts/2023-06-23-agent/\")\n",
    "data = loader.load()\n",
    "\n",
    "# Split\n",
    "from langchain.text_splitter import RecursiveCharacterTextSplitter\n",
    "\n",
    "text_splitter = RecursiveCharacterTextSplitter(chunk_size=500, chunk_overlap=0)\n",
    "all_splits = text_splitter.split_documents(data)\n",
    "\n",
    "# Store splits\n",
    "from langchain.embeddings import OpenAIEmbeddings\n",
    "from langchain.vectorstores import Chroma\n",
    "\n",
    "vectorstore = Chroma.from_documents(documents=all_splits, embedding=OpenAIEmbeddings())\n",
    "\n",
    "# LLM\n",
    "from langchain.chat_models import ChatOpenAI\n",
    "\n",
    "llm = ChatOpenAI(model_name=\"gpt-3.5-turbo\", temperature=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e9299312-ce29-4963-81f7-27efecbbd49c",
   "metadata": {},
   "source": [
    "**Initialize the chain**. With the data added to the vectorstore, we can initialize the chain. We will\n",
    "pass the prompt in via the `chain_type_kwargs` argument."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "4df53934-6ad8-4729-a9e9-1730834d2f05",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# RetrievalQA\n",
    "from langchain.chains import RetrievalQA\n",
    "\n",
    "qa_chain = RetrievalQA.from_chain_type(\n",
    "    llm, retriever=vectorstore.as_retriever(), chain_type_kwargs={\"prompt\": prompt}\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "51aa87b5-3622-496b-b728-9522bf4eea1c",
   "metadata": {},
   "source": [
    "## 3. Run Chain\n",
    "\n",
    "Now that the chain is initialized, you can run it just like you would normally."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "5334b09b-a1a5-4c1f-ba96-ac6dff12e63e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'The approaches to task decomposition include using LLM with simple prompting, task-specific instructions, and human inputs.'"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "question = \"What are the approaches to Task Decomposition?\"\n",
    "result = qa_chain({\"query\": question})\n",
    "result[\"result\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b04b3599-b0e2-4f51-a015-c0e227e30c32",
   "metadata": {},
   "source": [
    "## 4. (Optional) Commit any new changes to the hub\n",
    "\n",
    "After debugging, evaluating, or monitoring your chain in some deployment, you may want to make some changes to the prompt. You can do so by adding this prompt under your handle's namespace.\n",
    "\n",
    "**Note:** If you receive a '403' forbidden error, you may need to set your `LANGCHAIN_HUB_API_KEY` to a personal API key."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "c392f2c3-9d00-4c41-983a-1f7961ee3f15",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'https://smith.langchain.com/hub/infra-guy/rag-prompt/597661da'"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "handle = \"YOUR HUB HANDLE\"  # Replace with your handle!\n",
    "hub.push(f\"{handle}/rag-prompt\", prompt)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0380d2d3-6f93-480b-af5b-64205692a129",
   "metadata": {},
   "source": [
    "Now you can view your prompt in the hub. It should look something like this:\n",
    "\n",
    "[![Initial push](./img/initial_push.png)](https://smith.langchain.com/hub/wfh/rag-prompt)\n",
    "\n",
    "Let's say you've tried this prompt out and have derived a better one for your use case.\n",
    "You can push the updated prompt to the same key to \"commit\" a new version of the prompt.\n",
    "\n",
    "For instance, let's add a system message to the prompt:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "f114d583-5bd3-4929-8c78-8fe06a1bd19e",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# You may try making other changes and saving them in a new commit.\n",
    "from langchain import schema\n",
    "\n",
    "prompt.messages.insert(\n",
    "    0,\n",
    "    schema.SystemMessage(\n",
    "        content=\"You are a precise, autoregressive question-answering system.\"\n",
    "    ),\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "18076aed-32ef-4d33-a3f9-63c5a38ed5f3",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'https://smith.langchain.com/hub/infra-guy/rag-prompt/e0569108'"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Pushing to the same prompt \"repo\" will create a new commit\n",
    "hub.push(f\"{handle}/rag-prompt\", prompt)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ff16cae9-02d1-44dd-8ede-c95001eb49b3",
   "metadata": {},
   "source": [
    "Now the newest version of the prompt is saved as the `latest` version. It should look something like this:\n",
    "\n",
    "[![Updated Prompt](./img/updated.png)](https://smith.langchain.com/hub/wfh/rag-prompt)\n",
    "\n",
    "You can view all saved versions by navigating to the \"commits\" tab.\n",
    "\n",
    "[![Commits](./img/commits.png)](https://smith.langchain.com/hub/wfh/rag-prompt?tab=1)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7bc27733-b28d-4901-984b-c1c8422fcae6",
   "metadata": {},
   "source": [
    "## Conclusion\n",
    "\n",
    "In this tutorial, you learned how to use the [hub](https://smith.langchain.com/hub?page=1) to manage prompts for a retrieval QA chain. The hub is a centralized location to manage, version, and share your prompts (and later, other artifacts).\n",
    "\n",
    "For more information, check out the [docs](https://docs.smith.langchain.com/hub/quickstart) or reach out to support@langchain.dev."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}