tracing-examples/rest/rest.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "4ddd44f6-80a3-4c3c-b467-1a2a2cc05d62",
   "metadata": {
    "tags": []
   },
   "source": [
    "# Tracing via the REST API\n",
    "[![Open In Collab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/langchain-ai/langsmith-cookbook/blob/main/tracing-examples/rest/rest.ipynb)\n",
    "\n",
    "It's likely that your production LLM application is written in a language other than Python or JavaScript. In this case, you can use the REST API to log runs and take advantage of LangSmith's tracing and monitoring functionality. The OpenAPI spec for posting runs can be found [here](https://github.com/langchain-ai/langsmith-sdk/blob/main/openapi/openapi.yaml).\n",
    "\n",
    "LangSmith tracing is built on \"runs\", which are analogous to traces and spans in OpenTelemetry. The basics of logging a run to LangSmith looks like:\n",
    "\n",
    "- Submit a POST request to \"https://api.smith.langchain.com/runs\"\n",
    "- JSON body of the request must have a name, run_type, inputs, and any other desired information.\n",
    "- An \"x-api-key\" header must be provided to authenticate, using a valid API key created in the LangSmith app\n",
    "\n",
    "In the following walkthrough, you will do the following:\n",
    "- Log a generic chain run to LangSmith via the REST API\n",
    "- Add additional tags, metadata, and other information to the run\n",
    "- Log LLM (chat and completion model) runs to calculate tokens and render chat messages\n",
    "- Log nested runs by setting the parent run id\n",
    "\n",
    "Let's get started!\n",
    "\n",
    "## Prerequisites\n",
    "\n",
    "Before moving on, make sure to create an account in LangSmith, and create an API key. Once that is done, you may continue. There are no dependencies apart from Python for this tutorial."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "6df839f2-e2b6-4b94-86b5-22cad8769084",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "# Update with your API URL if using a hosted instance of Langsmith.\n",
    "os.environ[\"LANGCHAIN_ENDPOINT\"] = \"https://api.smith.langchain.com\"\n",
    "os.environ[\"LANGCHAIN_API_KEY\"] = \"YOUR API KEY\"  # Update with your API key\n",
    "project_name = \"YOUR PROJECT NAME\"  # Update with your project name"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1e425aa3-6564-43c3-8036-53969ff48566",
   "metadata": {},
   "source": [
    "## Logging a Run\n",
    "\n",
    "Below is a minimal example of how to create a run using the REST API. Since each run represents the start and end of a function call (or other unit of work), we typically log the run in two calls:\n",
    "- First create the run by submitting a POST request at the beginning of the function call\n",
    "- Then update the run via a PATCH request at the end.\n",
    "\n",
    "This ensures the runs appear in a timely manner, even for long-running operations.\n",
    "\n",
    "The following example demonstrates this approach."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "f5aff79e-8f03-4f3b-a347-032ce74bcd6e",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Response [200]>"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import datetime\n",
    "import uuid\n",
    "\n",
    "import requests\n",
    "\n",
    "_LANGSMITH_API_KEY = os.environ[\"LANGCHAIN_API_KEY\"]\n",
    "\n",
    "run_id = str(uuid.uuid4())\n",
    "\n",
    "res = requests.post(\n",
    "    \"https://api.smith.langchain.com/runs\",\n",
    "    json={\n",
    "        \"id\": run_id,\n",
    "        \"name\": \"MyFirstRun\",\n",
    "        \"run_type\": \"chain\",\n",
    "        \"start_time\": datetime.datetime.utcnow().isoformat(),\n",
    "        \"inputs\": {\"text\": \"Foo\"},\n",
    "    },\n",
    "    headers={\"x-api-key\": _LANGSMITH_API_KEY},\n",
    ")\n",
    "\n",
    "# ... do some work ...\n",
    "\n",
    "requests.patch(\n",
    "    f\"https://api.smith.langchain.com/runs/{run_id}\",\n",
    "    json={\n",
    "        \"outputs\": {\"my_output\": \"Bar\"},\n",
    "        \"end_time\": datetime.datetime.utcnow().isoformat(),\n",
    "    },\n",
    "    headers={\"x-api-key\": _LANGSMITH_API_KEY},\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1521e322-8585-4d3a-acea-65eaa1a5334f",
   "metadata": {},
   "source": [
    "Here we manually define the run's ID to reuse in the PATCH request. It is passed through the json body when creating the run and as a path parameter when updating. We have also authenticated to LangSmith by passing our API key in the \"x-api-key\" header. \n",
    "\n",
    "You can find the logged run by navigating to the [projects page](https://smith.langchain.com/projects) in LangSmith and clicking on your default project, there you will see a barebones \"chain\" run with the name \"MyFirstRun\". It should look something like the following:\n",
    "\n",
    "<a href=\"https://smith.langchain.com/public/8fbe61f2-6d23-47f4-b6d7-6946322aad48/r\"><img src=\"./img/minimal.png\" alt=\"minimal trace example\" style=\"width:75%\"></a>\n",
    "\n",
    "Not much information is included, since we haven't added any child runs, tags, or metadata yet. It is marked as a \"success\" since we patched the end time without errors.\n",
    "\n",
    "In addition to the values included above, you can also provide any of the following information in the POST request:\n",
    "\n",
    "```\n",
    "{\n",
    "  \"name\": \"string\",\n",
    "  \"inputs\": {},\n",
    "  \"run_type\": \"string\",\n",
    "  \"start_time\": \"2019-08-24T14:15:22Z\", # UTC timestamp in ISO format\n",
    "  \"end_time\": \"2019-08-24T14:15:22Z\", # UTC timestamp in ISO format\n",
    "  \"extra\": {},\n",
    "  \"error\": \"string\",\n",
    "  \"outputs\": {},\n",
    "  \"parent_run_id\": \"f8faf8c1-9778-49a4-9004-628cdb0047e5\",\n",
    "  \"events\": [\n",
    "    {}\n",
    "  ],\n",
    "  \"tags\": [\n",
    "    \"string\"\n",
    "  ],\n",
    "  \"id\": \"497f6eca-6276-4993-bfeb-53cbbbba6f08\",\n",
    "  \"session_id\": \"1ffd059c-17ea-40a8-8aef-70fd0307db82\",\n",
    "  \"session_name\": \"string\", # This is the name of the PROJECT. \"default\" if not specified. Sessions are the old name for projects.\n",
    "  \"reference_example_id\": \"9fb06aaa-105f-4c87-845f-47d62ffd7ee6\"\n",
    "}\n",
    "```\n",
    "\n",
    "This can also be found in the [API documentation](https://web.smith.langchain.com/redoc#tag/run/operation/create_run_runs_post). \n",
    "\n",
    "Lets look at a more complex chain example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "2b41f863-ba8e-41e5-908c-f74f770e00b1",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import platform\n",
    "\n",
    "run_id = str(uuid.uuid4())\n",
    "\n",
    "requests.post(\n",
    "    \"https://api.smith.langchain.com/runs\",\n",
    "    json={\n",
    "        \"id\": run_id,\n",
    "        \"name\": \"MySecondRun\",\n",
    "        \"run_type\": \"chain\",\n",
    "        \"inputs\": {\"text\": \"Foo\"},\n",
    "        \"start_time\": datetime.datetime.utcnow().isoformat(),\n",
    "        \"session_name\": project_name,\n",
    "        \"tags\": [\"langsmith\", \"rest\", \"my-example\"],\n",
    "        \"extra\": {\n",
    "            \"metadata\": {\"my_key\": \"My value\"},\n",
    "            \"runtime\": {\n",
    "                \"platform\": platform.platform(),\n",
    "            },\n",
    "        },\n",
    "    },\n",
    "    headers={\"x-api-key\": _LANGSMITH_API_KEY},\n",
    ")\n",
    "# ... do some work ...\n",
    "events = []\n",
    "# Events like new tokens and retries can be added\n",
    "events.append({\"event_name\": \"retry\", \"reason\": \"never gonna give you up\"})\n",
    "events.append({\"event_name\": \"new_token\", \"value\": \"foo\"})\n",
    "\n",
    "res = requests.patch(\n",
    "    f\"https://api.smith.langchain.com/runs/{run_id}\",\n",
    "    json={\n",
    "        \"end_time\": datetime.datetime.utcnow().isoformat(),\n",
    "        \"outputs\": {\"generated\": \"Bar\"},\n",
    "        \"events\": events,\n",
    "    },\n",
    "    headers={\"x-api-key\": _LANGSMITH_API_KEY},\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bc6cdc54-9604-47f5-886c-8c0dfc10ef1b",
   "metadata": {
    "tags": []
   },
   "source": [
    "In this example, we have logged a chain run to a new project called \"My REST Project\" by passing the \"session_name\" field. This will create a new project if a project of that name does not already exist.\n",
    "\n",
    "We also tagged the run with the `langsmith`, `rest`, and `my-example` tags, and added metadata and runtime information to the run. \n",
    "\n",
    "Finally, we added a retry event and a new token event to the run. Events can be used to log additional minor information about what occurred during a run (such as streaming and retry events), when that information doesn't merit an entire child run and is not the final output of the run.\n",
    "\n",
    "Below is an example screenshot of what the logged trace from the example above looks like. The new run now has inputs and outputs, a latency calculation, and tags.\n",
    "\n",
    "[![populated trace example](./img/populated.png)](https://smith.langchain.com/public/8535ae34-1137-4562-b0ed-5ef91bf421dc/r)\n",
    "\n",
    "To see the logged metadata and other runtime information you saved in the trace above, you can navigate to the \"metadata\" tab:\n",
    "\n",
    "[![trace metadata](./img/populated_metadata.png)](https://smith.langchain.com/public/8535ae34-1137-4562-b0ed-5ef91bf421dc/r?tab=2)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "32f9e644-bf1a-4b5d-802b-6eae3b55196c",
   "metadata": {
    "tags": []
   },
   "source": [
    "## Logging LLM Runs\n",
    "\n",
    "The \"chain\" runs above are versatile and can represent any function in your application. However, some of LangSmith's functionality is only available for LLM runs. Correctly formatted runs with the `run_type` of \"llm\" let you:\n",
    "\n",
    "- Track token usage\n",
    "- Render \"prettier\" chat or completion message formats for better readability.\n",
    "\n",
    "LangSmith supports OpenAI's llm message schema, so you can directly log the inputs and outputs of your call to any openai-compatible API without having to convert it to a new format.\n",
    "\n",
    "We will show examples below.\n",
    "\n",
    "#### Logging LLM Chat Messages\n",
    "\n",
    "To log messages in the \"chat\" model format (role and message dictionaries), LangSmith expects the following format:\n",
    "\n",
    "- Provide `messages: [{\"role\": string, \"content\": string}]` as a key-value pair in the inputs\n",
    "- Provide `choices: [{\"message\": {\"role\": string, \"content\": string}]` as a key-value pair in the outputs.\n",
    "\n",
    "For function calling, you can also pass a `functions=[...]` key-value pair in the inputs, and include a `function_call: {\"name\": string, \"arguments\": {}}` key-value pair in the response message choice.\n",
    "\n",
    "Additional parameters, such as the temperature, model, etc. ought to be passed in as inputs and will be registered as \"invocation_params\" by LangSmith. The following example shows how to log a chat model run with functions inputs:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "167e66e6-e819-4c5d-b24e-d5f5236be86f",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Response [200]>"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "run_id = str(uuid.uuid4())\n",
    "requests.post(\n",
    "    \"https://api.smith.langchain.com/runs\",\n",
    "    json={\n",
    "        \"id\": run_id,\n",
    "        \"name\": \"MyChatModelRun\",\n",
    "        \"run_type\": \"llm\",\n",
    "        \"inputs\": {\n",
    "            \"messages\": [{\"role\": \"user\", \"content\": \"What's the weather in SF like?\"}],\n",
    "            # Optional\n",
    "            \"model\": \"text-davinci-003\",\n",
    "            \"functions\": [\n",
    "                {\n",
    "                    \"name\": \"get_current_weather\",\n",
    "                    \"description\": \"Get the current weather in a given location\",\n",
    "                    \"parameters\": {\n",
    "                        \"type\": \"object\",\n",
    "                        \"properties\": {\n",
    "                            \"location\": {\n",
    "                                \"type\": \"string\",\n",
    "                                \"description\": \"The city and state, e.g. San Francisco, CA\",\n",
    "                            },\n",
    "                            \"unit\": {\n",
    "                                \"type\": \"string\",\n",
    "                                \"enum\": [\"celsius\", \"fahrenheit\"],\n",
    "                            },\n",
    "                        },\n",
    "                        \"required\": [\"location\"],\n",
    "                    },\n",
    "                }\n",
    "            ],\n",
    "            # You can add other invocation paramers as k-v pairs\n",
    "            \"temperature\": 0.0,\n",
    "        },\n",
    "        \"start_time\": datetime.datetime.utcnow().isoformat(),\n",
    "        \"session_name\": project_name,\n",
    "    },\n",
    "    headers={\"x-api-key\": _LANGSMITH_API_KEY},\n",
    ")\n",
    "\n",
    "requests.patch(\n",
    "    f\"https://api.smith.langchain.com/runs/{run_id}\",\n",
    "    json={\n",
    "        \"end_time\": datetime.datetime.utcnow().isoformat(),\n",
    "        \"outputs\": {\n",
    "            \"choices\": [\n",
    "                {\n",
    "                    \"index\": 0,\n",
    "                    \"message\": {\n",
    "                        \"role\": \"assistant\",\n",
    "                        # Content is whatever string response the\n",
    "                        # model generates\n",
    "                        \"content\": \"Mostly cloudy.\",\n",
    "                        # Function call is the function invocation and arguments\n",
    "                        # as a string\n",
    "                        \"function_call\": {\n",
    "                            \"name\": \"get_current_weather\",\n",
    "                            \"arguments\": '{\\n\"location\": \"San Francisco, CA\"\\n}',\n",
    "                        },\n",
    "                    },\n",
    "                    \"finish_reason\": \"function_call\",\n",
    "                }\n",
    "            ],\n",
    "        },\n",
    "    },\n",
    "    headers={\"x-api-key\": _LANGSMITH_API_KEY},\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3e0a3663-bedf-4416-b565-720bfa56c300",
   "metadata": {},
   "source": [
    "When viewed in LangSmith, the run will look something like the one below, with the human, AI, and other chat messages all given their own cards, and with the token counts visible on the right.\n",
    "\n",
    "[![chat example](./img/chat_example.png)](https://smith.langchain.com/public/20a2d2cb-649c-4edc-873f-2f087937a01b/r)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bc8ca5d0-6beb-4882-b74b-e378b6839bf4",
   "metadata": {},
   "source": [
    "#### Logging \"Completions\" Models\n",
    "\n",
    "To log in the \"completions\" format (string in, string out), LangSmith expects the following format:\n",
    "- Name the run \"openai.Completion.create\" or \"openai.Completion.acreate\"\n",
    "- Provide `prompt: string` as a key-value pair in the inputs\n",
    "- Provide `choices: [{\"text\": string}]` key-value pair in the outputs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "67761b57-9c0c-441b-92b5-935b059bdaff",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Response [200]>"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "run_id = str(uuid.uuid4())\n",
    "requests.post(\n",
    "    \"https://api.smith.langchain.com/runs\",\n",
    "    json={\n",
    "        \"id\": run_id,\n",
    "        \"name\": \"MyLLMRun\",\n",
    "        \"run_type\": \"llm\",\n",
    "        \"inputs\": {\n",
    "            \"prompt\": \"Hi there!\",\n",
    "            # Optional: model or engine name, and other invocation params\n",
    "            \"engine\": \"text-davinci-003\",\n",
    "            \"temperature\": 0.0,\n",
    "        },\n",
    "        \"start_time\": datetime.datetime.utcnow().isoformat(),\n",
    "        \"session_name\": project_name,\n",
    "    },\n",
    "    headers={\"x-api-key\": _LANGSMITH_API_KEY},\n",
    ")\n",
    "\n",
    "requests.patch(\n",
    "    f\"https://api.smith.langchain.com/runs/{run_id}\",\n",
    "    json={\n",
    "        \"end_time\": datetime.datetime.utcnow().isoformat(),\n",
    "        \"outputs\": {\n",
    "            \"choices\": [\n",
    "                {\n",
    "                    \"text\": \"\\nMy name is Polly and I'm excited to talk to you!\",\n",
    "                    \"index\": 0,\n",
    "                    \"logprobs\": None,\n",
    "                    \"finish_reason\": \"stop\",\n",
    "                },\n",
    "            ]\n",
    "        },\n",
    "    },\n",
    "    headers={\"x-api-key\": _LANGSMITH_API_KEY},\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9dc3b049-9192-4ff0-bb16-ecc527a91f7d",
   "metadata": {},
   "source": [
    "The completion model output looks like the example below. Once again, the token counts are indicated on the right, and the completion output is highlighted in green following the origina prompt.\n",
    "\n",
    "[![completion example](./img/completion_example.png)](https://smith.langchain.com/public/6b604b4d-b4af-4c77-bc12-d99c6c6742fd/r)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4bd688a7-e4c9-450e-943e-70f8f1ed45ca",
   "metadata": {},
   "source": [
    "## Nesting Runs\n",
    "\n",
    "The above examples work great for linear logs, but it's likely that your application involves some amount of nested execution. It's a lot easier to debug a complex chain if the logs themselves contain the required associations. There are currently two bits of complexity in doing so. We plan to relax the execution order requirement at some point in the future:\n",
    "\n",
    "- You must include a `parent_run_id` in your JSON body.\n",
    "- You must track the `execution_order` of the child run for it to be rendered correctly in the trace.\n",
    "\n",
    "Below is an updated example of how to do this. We will create a new `RunLogger` class that manages the execution order state for us."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "04e3cb65-94be-404a-8bab-45c7bd573052",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import uuid\n",
    "from typing import Optional\n",
    "\n",
    "\n",
    "class RunLogger:\n",
    "    def post_run(\n",
    "        self, data: dict, name: str, run_id: str, parent_run_id: Optional[str] = None\n",
    "    ) -> None:\n",
    "        requests.post(\n",
    "            \"https://api.smith.langchain.com/runs\",\n",
    "            json={\n",
    "                \"id\": run_id,\n",
    "                \"name\": name,\n",
    "                \"run_type\": \"chain\",\n",
    "                \"parent_run_id\": parent_run_id,\n",
    "                \"inputs\": data,\n",
    "                \"start_time\": datetime.datetime.utcnow().isoformat(),\n",
    "                \"session_name\": project_name,\n",
    "            },\n",
    "            headers={\"x-api-key\": _LANGSMITH_API_KEY},\n",
    "        )\n",
    "\n",
    "    def patch_run(\n",
    "        self, run_id: str, output: Optional[dict] = None, error: Optional[str] = None\n",
    "    ) -> None:\n",
    "        requests.patch(\n",
    "            f\"https://api.smith.langchain.com/runs/{run_id}\",\n",
    "            json={\n",
    "                \"error\": error,\n",
    "                \"outputs\": output,\n",
    "                \"end_time\": datetime.datetime.utcnow().isoformat(),\n",
    "            },\n",
    "            headers={\"x-api-key\": _LANGSMITH_API_KEY},\n",
    "        )"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cada92cd-470f-4b79-a923-7da895147c88",
   "metadata": {},
   "source": [
    "To demonstrate how this works, we will create a simple fibonacci function and log each call as a \"chain\" run."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "72331136-fca1-479c-81fc-ec98c9d9b1bd",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "logger = RunLogger()\n",
    "\n",
    "\n",
    "def fibonacci(n: int, depth: int = 0, parent_run_id: Optional[str] = None) -> int:\n",
    "    run_id = str(uuid.uuid4())\n",
    "    logger.post_run(\n",
    "        {\"n\": n}, f\"fibonacci_recursive\", run_id, parent_run_id=parent_run_id\n",
    "    )\n",
    "    try:\n",
    "        if n <= 1:\n",
    "            result = n\n",
    "        else:\n",
    "            result = fibonacci(n - 1, depth + 1, parent_run_id=run_id) + fibonacci(\n",
    "                n - 2, depth + 1, parent_run_id=run_id\n",
    "            )\n",
    "        logger.patch_run(run_id, output={\"result\": result})\n",
    "        return result\n",
    "    except Exception as e:\n",
    "        logger.patch_run(run_id, error=str(e))\n",
    "        raise"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "e087cb60-1e12-48d5-8788-7cdab59993db",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "5"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fibonacci(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "973bdee1-78a5-4eb8-abce-4ad7b9880b00",
   "metadata": {
    "tags": []
   },
   "source": [
    "This should generate a trace similar to the one shown below:\n",
    "\n",
    "[![fibonacci](./img/fibonacci.png)](https://smith.langchain.com/public/6b604b4d-b4af-4c77-bc12-d99c6c6742fd/r)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "4c713dff-7df2-4822-895a-1fd872bfc471",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "def fibonacci(n: int, depth: int = 0, parent_run_id: Optional[str] = None) -> int:\n",
    "    run_id = str(uuid.uuid4())\n",
    "    logger.post_run(\n",
    "        {\"n\": n}, f\"fibonacci_recursive\", run_id, parent_run_id=parent_run_id\n",
    "    )\n",
    "    try:\n",
    "        if n < 0:\n",
    "            raise ValueError(\"NEGATIVE NUMER NOT ALLOWED\")\n",
    "        if n <= 1:\n",
    "            result = n\n",
    "        else:\n",
    "            result = fibonacci(n - 1, depth + 1, parent_run_id=run_id) + fibonacci(\n",
    "                n - 2, depth + 1, parent_run_id=run_id\n",
    "            )\n",
    "        logger.patch_run(run_id, output={\"result\": result})\n",
    "        return result\n",
    "    except Exception as e:\n",
    "        logger.patch_run(run_id, error=str(e))\n",
    "        raise\n",
    "\n",
    "\n",
    "# We will show what the trace looks like with an error\n",
    "try:\n",
    "    fibonacci(2.3)\n",
    "except:\n",
    "    pass"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5ada1f06-6222-4f44-9707-7ed41a95131e",
   "metadata": {
    "tags": []
   },
   "source": [
    "The resulting run should look something like the following. The errors are propagated up the call hierarchy so you can easily see where in the execution the chain failed.\n",
    "\n",
    "[![Fibonacci with error](./img/recursive_error.png)](https://smith.langchain.com/public/a3065970-1da6-472e-862b-eb9d97fd9607/r)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "22caa026-ef76-4540-b4eb-e48ce1d64325",
   "metadata": {},
   "source": [
    "## Single Requests\n",
    "\n",
    "If you want to reduce the number of requests you make to LangSmith, you can log runs in a single request. Just be sure to include the outputs or error and fix the end_time all in the post request. \n",
    "\n",
    "Below is an example that logs the completion LLM run from above in a single call. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "c0a0c88f-8264-4329-a522-6164b47decc5",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Response [200]>"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "requests.post(\n",
    "    \"https://api.smith.langchain.com/runs\",\n",
    "    json={\n",
    "        \"name\": \"MyLLMRun\",\n",
    "        \"run_type\": \"llm\",\n",
    "        \"inputs\": {\n",
    "            \"prompt\": \"Hi there!\",\n",
    "            # Optional: model or engine name, and other invocation params\n",
    "            \"engine\": \"text-davinci-003\",\n",
    "            \"temperature\": 0.0,\n",
    "        },\n",
    "        \"outputs\": {\n",
    "            \"choices\": [\n",
    "                {\n",
    "                    \"text\": \"\\nMy name is Polly and I'm excited to talk to you!\",\n",
    "                    \"index\": 0,\n",
    "                    \"logprobs\": None,\n",
    "                    \"finish_reason\": \"stop\",\n",
    "                },\n",
    "            ]\n",
    "        },\n",
    "        \"session_name\": project_name,\n",
    "        \"start_time\": datetime.datetime.utcnow().isoformat(),\n",
    "        \"end_time\": datetime.datetime.utcnow().isoformat(),\n",
    "    },\n",
    "    headers={\"x-api-key\": _LANGSMITH_API_KEY},\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "05e3dbd9-9fbb-403a-8f38-8e553bafe654",
   "metadata": {},
   "source": [
    "## Conclusion\n",
    "\n",
    "In this walkthrough, you used the REST API to log chain and LLM runs to LangSmith and reviewed the resulting traces. This is currently the only way to log runs to LangSmith if you aren't using a language supported by one of the LangSmith SDK's (python and JS/TS).\n",
    "\n",
    "You then created a helper class to log nested runs (similar to OTel spans) to take advantage of LangSmith's full trace tree debugging."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}