-
Notifications
You must be signed in to change notification settings - Fork 128
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* added new tool docs * added example tutorials * fixed link * fixed link * fixed link * spelling mistakes
- Loading branch information
1 parent
c601195
commit 6875b6a
Showing
4 changed files
with
107 additions
and
20 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,29 @@ | ||
# Template Matching Custom Tool | ||
|
||
This demo shows you how to create a custom tool for template matching that your Vision | ||
Agent can then use to help you answer questions. To get started, you can install the | ||
requirements by running: | ||
|
||
```bash | ||
pip install -r requirements.txt | ||
``` | ||
|
||
You can then run the custom tool by running: | ||
|
||
```bash | ||
python run_custom_tool.py | ||
``` | ||
|
||
Tool choice can be difficult for the agent to get, so sometimes it helps to explicitly | ||
call out which tool you want to use. For example: | ||
|
||
```python | ||
import vision_agent as va | ||
|
||
agent = va.agent.VisionAgent(verbose=True) | ||
agent( | ||
"Can you use the 'template_match_' tool to find the location of pid_template.png in pid.png?", | ||
image="pid.png", | ||
reference_data={"image": "pid_template.png"}, | ||
) | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,31 @@ | ||
# Generate Masks for DINOv | ||
|
||
This application allows you to generate masks to use for the DINOv tool. To get started | ||
install the requirements by running: | ||
|
||
```bash | ||
pip install -r requirements.txt | ||
``` | ||
|
||
Then you can run the streamlit app by running: | ||
|
||
```bash | ||
streamlit run app.py | ||
``` | ||
|
||
From here you can upload an image, paint a mask over the image, and then save the mask. | ||
This can be used as input for the DINOv tool. | ||
|
||
```python | ||
import vision_agent as va | ||
|
||
data = { | ||
"prompt": [{"mask": "baggage.png", "image": "baggage_mask.png"}], | ||
"image": "baggage2.png", | ||
} | ||
tool = va.tools.DINOv() | ||
output = res(**data) | ||
image = va.image_utils.overlay_masks("baggage2.png", output) | ||
image = va.image_utils.overlay_bboxes(image, output) | ||
image.show() | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters