Skip to content

Commit 9b080bf

Browse files
authored
Add Python image upload docs (#48)
1 parent 2f3d00e commit 9b080bf

File tree

3 files changed

+104
-75
lines changed

3 files changed

+104
-75
lines changed

1_python/1_llm-prediction/_image-input.md

Lines changed: 0 additions & 73 deletions
This file was deleted.
Lines changed: 102 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,102 @@
1+
---
2+
title: Image Input
3+
description: API for passing images as input to the model
4+
index: 2
5+
---
6+
7+
*Required Python SDK version*: **1.1.0**
8+
9+
Some models, known as VLMs (Vision-Language Models), can accept images as input. You can pass images to the model using the `.respond()` method.
10+
11+
### Prerequisite: Get a VLM (Vision-Language Model)
12+
13+
If you don't yet have a VLM, you can download a model like `qwen2-vl-2b-instruct` using the following command:
14+
15+
```bash
16+
lms get qwen2-vl-2b-instruct
17+
```
18+
19+
## 1. Instantiate the Model
20+
21+
Connect to LM Studio and obtain a handle to the VLM (Vision-Language Model) you want to use.
22+
23+
```lms_code_snippet
24+
variants:
25+
"Python (convenience API)":
26+
language: python
27+
code: |
28+
import lmstudio as lms
29+
model = lms.llm("qwen2-vl-2b-instruct")
30+
31+
"Python (scoped resource API)":
32+
language: python
33+
code: |
34+
import lmstudio as lms
35+
with lms.Client() as client:
36+
model = client.llm.model("qwen2-vl-2b-instruct")
37+
38+
```
39+
40+
## 2. Prepare the Image
41+
42+
Use the `prepare_image()` function or `files` namespace method to
43+
get a handle to the image that can subsequently be passed to the model.
44+
45+
```lms_code_snippet
46+
variants:
47+
"Python (convenience API)":
48+
language: python
49+
code: |
50+
import lmstudio as lms
51+
image_path = "/path/to/image.jpg"; // Replace with the path to your image
52+
image_handle = lms.prepare_image(image_path)
53+
54+
"Python (scoped resource API)":
55+
language: python
56+
code: |
57+
import lmstudio as lms
58+
with lms.Client() as client:
59+
image_path = "/path/to/image.jpg"; // Replace with the path to your image
60+
image_handle = client.files.prepare_image(image_path)
61+
62+
```
63+
64+
If you only have the raw data of the image, you can supply the raw data directly as a bytes
65+
object without having to write it to disk first. Due to this feature, binary filesystem
66+
paths are *not* supported (as they will be handled as malformed image data rather than as
67+
filesystem paths).
68+
69+
Binary IO objects are also accepted as local file inputs.
70+
71+
The LM Studio server supports JPEG, PNG, and WebP image formats.
72+
73+
## 3. Pass the Image to the Model in `.respond()`
74+
75+
Generate a prediction by passing the image to the model in the `.respond()` method.
76+
77+
```lms_code_snippet
78+
variants:
79+
"Python (convenience API)":
80+
language: python
81+
code: |
82+
import lmstudio as lms
83+
image_path = "/path/to/image.jpg"; // Replace with the path to your image
84+
image_handle = lms.prepare_image(image_path)
85+
model = lms.llm("qwen2-vl-2b-instruct")
86+
chat = lms.Chat()
87+
chat.add_user_message("Describe this image please", images=[image_handle])
88+
prediction = model.respond(chat)
89+
90+
"Python (scoped resource API)":
91+
language: python
92+
code: |
93+
import lmstudio as lms
94+
with lms.Client() as client:
95+
image_path = "/path/to/image.jpg"; // Replace with the path to your image
96+
image_handle = client.files.prepare_image(image_path)
97+
model = client.llm.model("qwen2-vl-2b-instruct")
98+
chat = lms.Chat()
99+
chat.add_user_message("Describe this image please", images=[image_handle])
100+
prediction = model.respond(chat)
101+
102+
```

2_typescript/2_llm-prediction/image-input.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,7 @@ Connect to LM Studio and obtain a handle to the VLM (Vision-Language Model) you
3131

3232
## 2. Prepare the Image
3333

34-
Use the `client.files.prepareImage()` method to get a handle to the image you can pass to the model.
34+
Use the `client.files.prepareImage()` method to get a handle to the image that can be subsequently passed to the model.
3535

3636
```lms_code_snippet
3737
variants:
@@ -54,7 +54,7 @@ If you only have the image in the form of a base64 string, you can use the `clie
5454
const image = await client.files.prepareImageBase64(imageBase64);
5555
```
5656

57-
We support JPEG, PNG, and WebP image formats.
57+
The LM Studio server supports JPEG, PNG, and WebP image formats.
5858

5959
## 3. Pass the Image to the Model in `.respond()`
6060

0 commit comments

Comments
 (0)