Skip to content

Commit dd9e991

Browse files
docs: adding Helm chart docs to Cosmos3 Nim cookbook (NVIDIA#373)
## Summary - populate the helm chart docs with the guide on running NIM on kubernetes clusters - added & updated references to helm chart guide in Readme, Skill, deployment guide ## Validation - Tested with an Opus 5.5 that succesfully deployed Cosmos3 NIM on local mini kube cluster using this docs
1 parent c3e4354 commit dd9e991

4 files changed

Lines changed: 264 additions & 9 deletions

File tree

‎cookbooks/cosmos3/nim/.agents/skills/cosmos3-nim-user/SKILL.md‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -58,11 +58,13 @@ Guide the customer through the canonical pages in this order:
5858
Transfer eligibility.
5959
3. `deployment.md` for the exact image, authentication, pre-download profile
6060
preflight, cache, runtime selector, launch, readiness, logs, and cleanup.
61+
For Kubernetes, use `helm.md` for the chart, secrets, values file, and
62+
verification.
6163
4. `configuration.md` only when a documented non-default setting is needed.
6264
5. The existing-endpoint preflight above before the first inference request.
6365

64-
Use only the exact image and commands in `deployment.md`; never substitute
65-
`latest`. After the image pull, run the documented pre-download profile
66+
Use only the exact image and commands in `deployment.md` or, for Kubernetes,
67+
`helm.md`; never substitute `latest`. After the image pull, run the documented pre-download profile
6668
preflight with the intended selectors and GPUs. For a Reasoner on DGX Spark/GB10
6769
or Jetson AGX Thor, every preflight and launch command shown to the customer,
6870
starting with the first one, must include

‎cookbooks/cosmos3/nim/README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -159,7 +159,7 @@ pages and examples as their source of truth and default to read-only assistance.
159159
- [Deployment](deployment.md)
160160
- [Configuration](configuration.md)
161161
- [Support matrix](support-matrix.md)
162-
- [Helm deployment status](helm.md)
162+
- [Helm deployment](helm.md)
163163
- [Bring your own checkpoint](bring-your-own-checkpoint.md)
164164

165165
### Use the APIs

‎cookbooks/cosmos3/nim/deployment.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -79,8 +79,8 @@ requires an explicit precision. Nano-DROID currently has BF16 profiles only.
7979
## Before you deploy
8080

8181
Verify the host against [Prerequisites](prerequisites.md) and choose a
82-
compatible configuration from the [Support matrix](support-matrix.md). For the
83-
current availability of Helm guidance, see [Helm deployment](helm.md).
82+
compatible configuration from the [Support matrix](support-matrix.md). For a
83+
Kubernetes deployment, follow [Deploy with Helm](helm.md).
8484

8585
## Authenticate to NGC
8686

‎cookbooks/cosmos3/nim/helm.md‎

Lines changed: 257 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,261 @@
11
<!-- SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
22
SPDX-License-Identifier: OpenMDW-1.1 -->
33

4-
# Cosmos3 Certified NIM Helm deployment
4+
# Deploy the Cosmos3 Certified NIM with Helm
55

6-
Helm deployment guidance is not included in this cookbook yet. Do not adapt a
7-
chart or values file from another NIM. Use the documented
8-
[Docker deployment](deployment.md) for the current Cosmos3 Certified NIM image.
6+
Use this page to deploy the Cosmos3 Certified NIM on Kubernetes with the shared
7+
`nim-wfm` Helm chart. Model selection, hardware floors, and verification are
8+
the same as in [Deployment](deployment.md).
9+
10+
## Prerequisites
11+
12+
- A Kubernetes cluster with GPU nodes that match a row in the
13+
[Support matrix](support-matrix.md), with the
14+
[NVIDIA GPU Operator](https://docs.nvidia.com/datacenter/cloud-native/gpu-operator/latest/)
15+
installed.
16+
- Helm and `kubectl` access to the target namespace. The commands below use the
17+
default namespace.
18+
- A storage class for the persistent model cache.
19+
- `NGC_API_KEY` exported as described in
20+
[Network and NGC access](prerequisites.md#network-and-ngc-access).
21+
22+
## Create secrets
23+
24+
The chart needs an image pull secret and a generic secret with a key named
25+
`NGC_API_KEY`:
26+
27+
```bash
28+
kubectl create secret docker-registry ngc-secret \
29+
--docker-server=nvcr.io \
30+
--docker-username='$oauthtoken' \
31+
--docker-password="$NGC_API_KEY"
32+
kubectl create secret generic ngc-api \
33+
--from-literal=NGC_API_KEY="$NGC_API_KEY"
34+
```
35+
36+
## Fetch the chart
37+
38+
This page uses `nim-wfm` chart version 1.3.0:
39+
40+
```bash
41+
export NIM_WFM_VERSION='1.3.0'
42+
helm fetch "https://helm.ngc.nvidia.com/nim/charts/nim-wfm-${NIM_WFM_VERSION}.tgz" \
43+
--username='$oauthtoken' \
44+
--password="$NGC_API_KEY"
45+
```
46+
47+
The chart README lists every value supported by that version:
48+
49+
```bash
50+
helm show readme "nim-wfm-${NIM_WFM_VERSION}.tgz"
51+
helm show values "nim-wfm-${NIM_WFM_VERSION}.tgz"
52+
```
53+
54+
## Configure values
55+
56+
Save the following as `cosmos3-values.yaml`. It deploys the Nano Generator,
57+
optimized for latency, on one GPU:
58+
59+
```yaml
60+
image:
61+
repository: nvcr.io/nim/nvidia/cosmos3
62+
tag: "2.0.0"
63+
imagePullSecrets:
64+
- name: ngc-secret
65+
model:
66+
ngcAPISecret: ngc-api
67+
nimCache: /opt/nim/.cache
68+
persistence:
69+
enabled: true
70+
size: 220Gi
71+
sharedMemory:
72+
sizeLimit: 16Gi
73+
resources:
74+
limits:
75+
nvidia.com/gpu: 1
76+
env:
77+
- name: NIM_MODEL_TYPE
78+
value: generator
79+
- name: NIM_MODEL_VARIANT
80+
value: nano
81+
- name: NIM_PERF_PROFILE
82+
value: latency
83+
```
84+
85+
Adjust it for the chosen configuration:
86+
87+
- **Model and GPUs:** Change `NIM_MODEL_VARIANT`, `NIM_PERF_PROFILE`, and
88+
`nvidia.com/gpu` together using the
89+
[Generator configurations](support-matrix.md#generator-configurations). The
90+
GPU count does not select a GPU type; use `nodeSelector` or `affinity` to
91+
target eligible GPU nodes, and `tolerations` if those nodes are tainted. On
92+
nodes with MIG enabled, request the MIG resource instead of
93+
`nvidia.com/gpu`, for example `nvidia.com/mig-4g.24gb: 2`.
94+
- **Cache size:** `220Gi` is a starting size for large model files. Set `persistence.storageClass` if the
95+
cluster has no default storage class.
96+
- **Memory:** A pod memory limit counts as system memory during profile
97+
selection. Keep any limit at or above the host RAM value of the selected row.
98+
- **Shared memory:** The chart mounts a memory-backed `emptyDir` at `/dev/shm`
99+
by default. `sharedMemory.sizeLimit: 16Gi` matches the 16 GiB that the Docker
100+
launch allocates.
101+
- **Pod user:** The chart runs the pod as UID and GID 1000 with
102+
`fsGroup: 1000` (`podSecurityContext`). The cache volume must be writable by
103+
that user; storage that ignores `fsGroup`, such as some NFS exports and
104+
`hostPath` directories, needs matching ownership.
105+
106+
Set these through chart values rather than `env`:
107+
108+
| Chart value | Replaces |
109+
| --- | --- |
110+
| `model.nimCache` | `NIM_CACHE_PATH` |
111+
| `model.ngcAPISecret` | `NGC_API_KEY` |
112+
| `model.apiPort` | `NIM_HTTP_API_PORT` |
113+
| `model.jsonLogging` | `NIM_LOGGING_JSONL` |
114+
| `model.logLevel` | `NIM_LOG_LEVEL` |
115+
116+
Add other [Configuration](configuration.md) variables to `env`.
117+
118+
To check profile compatibility before a cold download, run the Docker
119+
[pre-download profile preflight](deployment.md#run-the-pre-download-profile-preflight)
120+
on a host with the same GPUs and selectors.
121+
122+
### Reasoner
123+
124+
For Reasoner, replace `env` with the Reasoner selectors and choose the GPU
125+
count from the [Reasoner configurations](support-matrix.md#reasoner-configurations).
126+
Reasoner does not use `NIM_PERF_PROFILE`:
127+
128+
```yaml
129+
env:
130+
- name: NIM_MODEL_TYPE
131+
value: reasoner
132+
- name: NIM_MODEL_VARIANT
133+
value: nano
134+
```
135+
136+
On DGX Spark/GB10 or Jetson AGX Thor nodes, also set
137+
`NIM_GPU_MEMORY_UTILIZATION` to `"0.80"` for image-only workloads or `"0.70"`
138+
for video or mixed-media workloads; see
139+
[Set the Reasoner memory share](deployment.md#set-the-reasoner-memory-share-on-unified-memory-systems).
140+
141+
### OpenTelemetry
142+
143+
To export traces and metrics over OTLP, add these entries to `env`. The
144+
endpoint assumes an OpenTelemetry Collector DaemonSet that listens on host
145+
port 4318; change `OTEL_EXPORTER_OTLP_ENDPOINT` for other collector layouts:
146+
147+
```yaml
148+
- name: NIM_ENABLE_OTEL
149+
value: "1"
150+
- name: OTEL_SERVICE_NAME
151+
value: cosmos3
152+
- name: OTEL_TRACES_EXPORTER
153+
value: otlp
154+
- name: OTEL_METRICS_EXPORTER
155+
value: otlp
156+
- name: HOST_IP
157+
valueFrom:
158+
fieldRef:
159+
fieldPath: status.hostIP
160+
- name: OTEL_EXPORTER_OTLP_ENDPOINT
161+
value: "http://$(HOST_IP):4318"
162+
```
163+
164+
`HOST_IP` must come before `OTEL_EXPORTER_OTLP_ENDPOINT` so that Kubernetes
165+
can expand it.
166+
167+
### Other chart features
168+
169+
The chart README documents these optional features; all are off by default:
170+
171+
- **Prometheus metrics:** Set `metrics.serviceMonitor.enabled: true` to create
172+
a Prometheus Operator `ServiceMonitor` that scrapes `/v1/metrics` on the
173+
`http-api` port. See [Metrics](operations.md#metrics).
174+
- **Ingress:** `ingress.enabled`, `ingress.className`, `ingress.hosts`, and
175+
`ingress.tls`. The NIM does not authenticate requests, so protect any
176+
ingress outside the cluster.
177+
- **Autoscaling:** `autoscaling.enabled` with `minReplicas`, `maxReplicas`, and
178+
`metrics`. CPU and memory metrics are of limited use for scaling a NIM; use
179+
custom metrics, for example through `prometheus-adapter`. Each new replica
180+
must download or mount the model before it becomes ready; see
181+
[Storage and scaling](#storage-and-scaling).
182+
183+
## Install
184+
185+
Review the rendered manifests, then install the release:
186+
187+
```bash
188+
helm template cosmos3 "nim-wfm-${NIM_WFM_VERSION}.tgz" -f cosmos3-values.yaml
189+
helm install cosmos3 "nim-wfm-${NIM_WFM_VERSION}.tgz" -f cosmos3-values.yaml
190+
kubectl get pods -w
191+
```
192+
193+
Cold download, materialization, and model load can take much longer than pod
194+
startup. Send requests only after the pod reports `Ready`. Kubernetes restarts
195+
the container if it is not ready within `startupProbe.initialDelaySeconds` plus
196+
`startupProbe.failureThreshold` × `startupProbe.periodSeconds`; raise
197+
`failureThreshold` for slow first downloads.
198+
199+
## Verify
200+
201+
The chart creates a `ClusterIP` Service with no ingress, and the NIM does not
202+
handle authentication. Port-forward the Service for testing:
203+
204+
```bash
205+
kubectl port-forward service/cosmos3-nim-wfm 8000:http-api
206+
```
207+
208+
In another terminal:
209+
210+
```bash
211+
export NIM_URL=http://localhost:8000
212+
curl -fsS "$NIM_URL/v1/health/ready"
213+
curl -fsS "$NIM_URL/v1/metadata" | python3 -m json.tool
214+
```
215+
216+
Confirm that metadata reports `generator` and `/v1/infer`, or `reasoner` and
217+
`/v1/chat/completions`. Then run a first request from
218+
[Use an existing NIM endpoint](README.md#use-an-existing-nim-endpoint).
219+
220+
## Storage and scaling
221+
222+
The chart supports these cache options; use only one:
223+
224+
| Option | Values | Notes |
225+
| --- | --- | --- |
226+
| StatefulSet volume claims | `persistence.enabled: true` with the default `statefulSet.enabled: true` | Default. Each replica gets its own claim and downloads the model before it becomes ready |
227+
| Shared volume claim | `persistence.enabled: true`, `statefulSet.enabled: false`, `persistence.accessMode: ReadWriteMany` | One cache shared by all replicas; needs a `ReadWriteMany` storage class |
228+
| Existing claim | `persistence.existingClaim: <claim-name>` | Reuse a pre-filled cache. Run one replica unless the claim is `ReadWriteMany` |
229+
| Direct NFS | `nfs.enabled: true`, `nfs.server`, `nfs.path` | Mount options cannot be set per pod; an NFS-backed claim is usually a better choice |
230+
| `hostPath` | `hostPath.enabled: true`, `hostPath.path` | Ties pods to one node and has security implications |
231+
232+
Set `replicaCount` for a fixed number of replicas. With the default
233+
StatefulSet, you can pre-populate caches by scaling up to the maximum replica
234+
count once and then scaling down; the retained claims let later scale-ups
235+
start without a new download.
236+
237+
With `statefulSet.enabled: false` and a `ReadWriteOnce` volume, scaling or
238+
rolling upgrades beyond one pod are likely to fail. Use a `ReadWriteMany`
239+
storage class, manually cloned `ReadOnlyMany` volumes of a pre-filled cache,
240+
or direct NFS. Without persistence, the cache is an `emptyDir` and every pod
241+
start downloads the model again.
242+
243+
## Troubleshooting
244+
245+
| Symptom | Check |
246+
| --- | --- |
247+
| Pod stays `Pending` | Run `kubectl describe pod <pod>` and check `Events` for insufficient GPUs, untolerated taints, or an unbound volume claim |
248+
| Pod restarts while preparing the model workspace | The startup probe ran out before the download finished; increase `startupProbe.failureThreshold` |
249+
| Scaling or upgrade fails with `statefulSet.enabled: false` | The cache volume is `ReadWriteOnce`; see [Storage and scaling](#storage-and-scaling) |
250+
| Profile selection fails | Compare GPU type, count, free memory, and pod memory limit with the [Support matrix](support-matrix.md); see [Troubleshooting](operations.md#troubleshooting) |
251+
252+
## Remove the deployment
253+
254+
```bash
255+
helm uninstall cosmos3
256+
kubectl get pvc
257+
```
258+
259+
StatefulSet volume claims are retained by default and keep the model cache.
260+
Delete them and the `ngc-secret` and `ngc-api` secrets only when they are no
261+
longer needed.

0 commit comments

Comments
 (0)