angelina: Apply suggestions from code review (large size notebook)

Co-authored-by: angelinalg <[email protected]>
Signed-off-by: Aydin Abiar <[email protected]>

Author: Aydin-ab

doc/source/serve/tutorials/deployment-serve-llm/large-size-llm/notebook.ipynb

@@ -7,27 +7,27 @@
    "source": [
     "# Deploy a large size LLM\n",
     "\n",
-    "A large size LLM typically runs on multiple nodes with multiple GPUs, prioritizing peak quality and capability: stronger reasoning, broader knowledge, longer context windows, more robust generalization. It’s the right choice when state-of-the-art results are required and higher latency, complexity, and cost are acceptable trade-offs.\n",
+    "A large LLM typically runs on multiple nodes with multiple GPUs, prioritizing peak quality and capability: stronger reasoning, broader knowledge, longer context windows, more robust generalization. When higher latency, complexity, and cost are acceptable trade-offs because you require state-of-the-art results.\n",
     "\n",
-    "This tutorial deploys a large size LLM like DeepSeek-R1 (685 B parameters) using Ray Serve LLM. For smaller models, see [Deploying a small size LLM](https://docs.ray.io/en/latest/serve/tutorials/deployment-serve-llm/small-size-llm/README.html) or [Deploying a medium size LLM](https://docs.ray.io/en/latest/serve/tutorials/deployment-serve-llm/medium-size-llm/README.html).\n",
+    "This tutorial deploys DeepSeek-R1, a large LLM with 685 B parameters, using Ray Serve LLM. For smaller models, see [Deploying a small-sized LLM](https://docs.ray.io/en/latest/serve/tutorials/deployment-serve-llm/small-size-llm/README.html) or [Deploying a medium-sized LLM](https://docs.ray.io/en/latest/serve/tutorials/deployment-serve-llm/medium-size-llm/README.html).\n",
     "\n",
     "---\n",
     "\n",
-    "## Challenges of large-scale deployment\n",
+    "## Challenges of large-scale deployments\n",
     "\n",
-    "Deploying a 685 B-parameter model like DeepSeek-R1 presents significant technical challenges. At this scale, the model can't fit on a single GPU or even a single node. It must be distributed across multiple GPUs and nodes using *tensor parallelism* (splitting tensors within each layer) and *pipeline parallelism* (spreading layers across devices).  \n",
+    "Deploying a 685 B-parameter model like DeepSeek-R1 presents significant technical challenges. At this scale, the model can't fit on a single GPU or even a single node. You must distribute it across multiple GPUs and nodes using *tensor parallelism* (splitting tensors within each layer) and *pipeline parallelism* (spreading layers across devices).  \n",
     "\n",
     "Deploying a model of this scale normally requires you to manually launch and coordinate multiple nodes, unless you use a managed platform like [Anyscale](https://www.anyscale.com/), which automates cluster scaling and node orchestration. See [Deploy to production with Anyscale Services](#deploy-to-production-with-anyscale-services) for more details.\n",
     "\n",
     "---\n",
     "\n",
     "## Configure Ray Serve LLM\n",
     "\n",
-    "A large size LLM is typically deployed across multiple nodes with multiple GPUs. To fully utilize the hardware, set `pipeline_parallel_size` to the number of nodes and `tensor_parallel_size` to the number of GPUs per node, which distributes the model’s weights evenly.\n",
+    "A large-sized LLM is typically deployed across multiple nodes with multiple GPUs. To fully utilize the hardware, set `pipeline_parallel_size` to the number of nodes and `tensor_parallel_size` to the number of GPUs per node, which distributes the model’s weights evenly.\n",
     "\n",
     "Ray Serve LLM provides multiple [Python APIs](https://docs.ray.io/en/latest/serve/api/index.html#llm-api) for defining your application. Use [`build_openai_app`](https://docs.ray.io/en/latest/serve/api/doc/ray.serve.llm.build_openai_app.html#ray.serve.llm.build_openai_app) to build a full application from your [`LLMConfig`](https://docs.ray.io/en/latest/serve/api/doc/ray.serve.llm.LLMConfig.html#ray.serve.llm.LLMConfig) object.\n",
     "\n",
-    "**Optional:** Since Deepseek-R1 is a reasoning model, we use vLLM’s built-in reasoning parser to correctly separate its reasoning content from the final response. See [Deploying a reasoning LLM: Parse reasoning outputs](https://docs.ray.io/en/latest/serve/tutorials/deployment-serve-llm/reasoning-llm/README.html#parse-reasoning-outputs)."
+    "**Optional:** Because Deepseek-R1 is a reasoning model, this tutorial uses vLLM’s built-in reasoning parser to correctly separate its reasoning content from the final response. See [Deploying a reasoning LLM: Parse reasoning outputs](https://docs.ray.io/en/latest/serve/tutorials/deployment-serve-llm/reasoning-llm/README.html#parse-reasoning-outputs)."
    ]
   },
   {
@@ -51,7 +51,7 @@
     "            min_replicas=1, max_replicas=1,\n",
     "        )\n",
     "    ),\n",
-    "    ### Uncomment if your model is gated and need your Huggingface Token to access it\n",
+    "    ### Uncomment if your model is gated and needs your Hugging Face token to access it.\n",
     "    #runtime_env=dict(\n",
     "    #    env_vars={\n",
     "    #        \"HF_TOKEN\": os.environ.get(\"HF_TOKEN\")\n",
@@ -74,7 +74,7 @@
    "id": "6b2231a5",
    "metadata": {},
    "source": [
-    "**Note:** Before moving to a production setup, migrate to using a [Serve config file](https://docs.ray.io/en/latest/serve/production-guide/config.html) to make your deployment version-controlled, reproducible, and easier to maintain for CI/CD pipelines. See [Serving LLMs: Production Guide](https://docs.ray.io/en/latest/serve/llm/serving-llms.html#production-deployment) for an example.\n",
+    "**Note:** Before moving to a production setup, migrate to a [Serve config file](https://docs.ray.io/en/latest/serve/production-guide/config.html) to make your deployment version-controlled, reproducible, and easier to maintain for CI/CD pipelines. See [Serving LLMs: Production Guide](https://docs.ray.io/en/latest/serve/llm/serving-llms.html#production-deployment) for an example.\n",
     "\n",
     "---\n",
     "\n",
@@ -223,9 +223,9 @@
     "\n",
     "---\n",
     "\n",
-    "## Deploy to production with Anyscale Services\n",
+    "## Deploy to production with Anyscale services\n",
     "\n",
-    "For production deployment, use Anyscale Services to deploy the Ray Serve app to a dedicated cluster without modifying the code. Anyscale provides scalability, fault tolerance, and load balancing, keeping the service resilient against node failures, high traffic, and rolling updates, while also automating multi-node setup and autoscaling for large models like DeepSeek-R1.\n",
+    "For production deployment, use Anyscale services to deploy the Ray Serve app to a dedicated cluster without modifying the code. Anyscale provides scalability, fault tolerance, and load balancing, keeping the service resilient against node failures, high traffic, and rolling updates, while also automating multi-node setup and autoscaling for large models like DeepSeek-R1.\n",
     "\n",
     "**Beware**: this is an expensive deployment. At the time of writing, the deployment cost is around \\$110 USD per hour in the `us-west-2` AWS region using on-demand instances. Because this node has a high amount of inter-node traffic, and cross-zone traffic is expensive (around \\$0.02 per GB), it's recommended to *disable cross-zone autoscaling*. This demo is pre-configured with cross-zone autoscaling disabled for your convenience.\n",
     "\n",
@@ -239,9 +239,9 @@
     "\n",
     "### Launch the service\n",
     "\n",
-    "Anyscale provides out-of-the-box images (`anyscale/ray-llm`) which come pre-loaded with Ray Serve LLM, vLLM, and all required GPU/runtime dependencies. This makes it easy to get started without building a custom image.\n",
+    "Anyscale provides out-of-the-box images (`anyscale/ray-llm`), which come pre-loaded with Ray Serve LLM, vLLM, and all required GPU/runtime dependencies. This makes it easy to get started without building a custom image.\n",
     "\n",
-    "Create your Anyscale Service configuration in a new `service.yaml` file:\n",
+    "Create your Anyscale service configuration in a new `service.yaml` file:\n",
     "```yaml\n",
     "#service.yaml\n",
     "name: deploy-deepseek-r1\n",
@@ -274,7 +274,7 @@
     "- import_path: serve_deepseek_r1:app\n",
     "```\n",
     "\n",
-    "Deploy your Service"
+    "Deploy your service"
    ]
   },
   {
@@ -295,7 +295,7 @@
    "id": "18226fd7",
    "metadata": {},
    "source": [
-    "**Note:** If your model is gated, make sure to pass your HuggingFace Token to the Service with `--env HF_TOKEN=<YOUR_HUGGINGFACE_TOKEN>`\n",
+    "**Note:** If your model is gated, make sure to pass your Hugging Face token to the service with `--env HF_TOKEN=<YOUR_HUGGINGFACE_TOKEN>`\n",
     "\n",
     "**Custom Dockerfile**  \n",
     "You can customize the container by building your own Dockerfile. In your Anyscale Service config, reference the Dockerfile with `containerfile` (instead of `image_uri`):\n",
@@ -319,22 +319,22 @@
     "```console\n",
     "(anyscale +3.9s) curl -H \"Authorization: Bearer <YOUR-TOKEN>\" <YOUR-ENDPOINT>\n",
     "```\n",
-    "You can also retrieve both from the service page in the Anyscale Console. Click the **Query** button at the top. See [Send requests](#send-requests) for example requests, but make sure to use the correct endpoint and authentication token.  \n",
+    "You can also retrieve both from the service page in the Anyscale console. Click the **Query** button at the top. See [Send requests](#send-requests) for example requests, but make sure to use the correct endpoint and authentication token.  \n",
     "\n",
     "---\n",
     "\n",
     "### Access the Serve LLM dashboard\n",
     "\n",
-    "See [Enable LLM monitoring](#enable-llm-monitoring) for instructions on enabling LLM-specific logging. To open the Ray Serve LLM Dashboard from an Anyscale Service:\n",
+    "See [Enable LLM monitoring](#enable-llm-monitoring) for instructions on enabling LLM-specific logging. To open the Ray Serve LLM dashboard from an Anyscale service:\n",
     "1. In the Anyscale console, go to your **Service** or **Workspace**\n",
     "2. Navigate to the **Metrics** tab\n",
-    "3. Expand **View in Grafana** and click **Serve LLM Dashboard**\n",
+    "3. Click **View in Grafana** and click **Serve LLM Dashboard**\n",
     "\n",
     "---\n",
     "\n",
     "### Shutdown \n",
     " \n",
-    "Shutdown your Anyscale Service:"
+    "Shutdown your Anyscale service:"
    ]
   },
   {
@@ -358,7 +358,7 @@
     "\n",
     "## Enable LLM monitoring\n",
     "\n",
-    "The *Serve LLM Dashboard* offers deep visibility into model performance, latency, and system behavior, including:\n",
+    "The *Serve LLM dashboard* offers deep visibility into model performance, latency, and system behavior, including:\n",
     "\n",
     "* Token throughput (tokens/sec)\n",
     "* Latency metrics: Time To First Token (TTFT), Time Per Output Token (TPOT)\n",
@@ -390,7 +390,7 @@
     "INFO 07-30 11:56:04 [kv_cache_utils.py:637] Maximum concurrency for 32,768 tokens per request: 29.06x\n",
     "```\n",
     "\n",
-    "Here are a few ways to improve concurrency depending on your model and hardware:  \n",
+    "The following are a few ways to improve concurrency depending on your model and hardware:  \n",
     "\n",
     "**Reduce `max_model_len`**  \n",
     "Lowering `max_model_len` reduces the memory needed for KV cache.\n",
@@ -399,14 +399,14 @@
     "* `max_model_len = 32,768` → concurrency ≈ 29\n",
     "* `max_model_len = 16,384` → concurrency ≈ 58\n",
     "\n",
-    "**Use Distilled or Quantized Models**  \n",
+    "**Use distilled or quantized models**  \n",
     "Quantizing or distilling your model reduces its memory footprint, freeing up space for more KV cache and enabling more concurrent requests. For example, see [`deepseek-ai/DeepSeek-R1-Distill-Llama-70B`](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Llama-70B) for a distilled version of DeepSeek-R1.\n",
     "\n",
     "\n",
     "**Upgrade to GPUs with more memory**  \n",
     "Some GPUs provide significantly more room for KV cache and allow for higher concurrency out of the box.\n",
     "\n",
-    "**Scale with more Replicas**  \n",
+    "**Scale with more replicas**  \n",
     "In addition to tuning per-GPU concurrency, you can scale *horizontally* by increasing the number of replicas in your config.  \n",
     "Each replica runs on its own GPU, so raising the replica count increases the total number of concurrent requests your service can handle, especially under sustained or bursty traffic.\n",
     "```yaml\n",
@@ -416,24 +416,24 @@
     "    max_replicas: 4\n",
     "```\n",
     "\n",
-    "*For more details on tuning strategies, hardware guidance, and serving configurations, see the [GPU Selection Guide for LLM Serving](https://docs.anyscale.com/overview) and [Tuning vLLM and Ray Serve Parameters for LLM Deployment](https://docs.anyscale.com/overview).*\n",
+    "*For more details on tuning strategies, hardware guidance, and serving configurations, see [Choose a GPU for LLM serving](https://docs.anyscale.com/llm/serving/gpu-guidance) and [Tune parameters for LLMs on Anyscale services](https://docs.anyscale.com/llm/serving/parameter-tuning).*\n",
     "\n",
     "---\n",
     "\n",
     "## Troubleshooting\n",
     "\n",
-    "**HuggingFace Auth Errors**  \n",
+    "**Hugging Face auth errors**  \n",
     "Some models, such as Llama-3.1, are gated and require prior authorization from the organization. See your model’s documentation for instructions on obtaining access.\n",
     "\n",
-    "**Out-Of-Memory Errors**  \n",
+    "**Out-Of-Memory errors**  \n",
     "Out‑of‑memory (OOM) errors are one of the most common failure modes when deploying LLMs, especially as model sizes, and context length increase.  \n",
-    "See this [Troubleshooting Guide](https://docs.anyscale.com/overview) for common errors and how to fix them.\n",
+    "See [Troubleshooting Guide](https://docs.anyscale.com/overview) for common errors and how to fix them.\n",
     "\n",
     "---\n",
     "\n",
     "## Summary\n",
     "\n",
-    "In this tutorial, you deployed a large size LLM with Ray Serve LLM, from development to production. You learned how to configure Ray Serve LLM, deploy your service on your Ray cluster, and how to send requests. You also learned how to monitor your app and common troubleshooting issues."
+    "In this tutorial, you deployed a large-sized LLM with Ray Serve LLM, from development to production. You learned how to configure Ray Serve LLM, deploy your service on your Ray cluster, and how to send requests. You also learned how to monitor your app and troubleshoot common issues."
    ]
   }
  ],