-
Notifications
You must be signed in to change notification settings - Fork 128
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
a7484ca
commit 97d7fd6
Showing
3 changed files
with
102 additions
and
17 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,29 @@ | ||
# Template Matching Custom Tool | ||
|
||
This demo shows you how to create a custom tool for template matching that your Vision | ||
Agent can then use to help you answer questions. To get started, you can install the | ||
requirements by running: | ||
|
||
```bash | ||
pip install -r requirements.txt | ||
``` | ||
|
||
You can then run the custom tool by running: | ||
|
||
```bash | ||
python run_custom_tool.py | ||
``` | ||
|
||
Tool choice can be difficult for the agent to get, so sometimes it helps to explicitly | ||
call out which tool you want to use. For example: | ||
|
||
```python | ||
import vision_agent as va | ||
|
||
agent = va.agent.VisionAgent(verbose=True) | ||
agent( | ||
"Can you use the 'template_match_' tool to find the location of pid_template.png in pid.png?", | ||
image="pid.png", | ||
reference_data={"image": "pid_template.png"}, | ||
) | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,31 @@ | ||
# Generate Masks for DINOv | ||
|
||
This application allows you to generate masks to use for the DINOv tool. To get started | ||
install the requirements by running: | ||
|
||
```bash | ||
pip install -r requirements.txt | ||
``` | ||
|
||
Then you can run the streamlit app by running: | ||
|
||
```bash | ||
streamlit run app.py | ||
``` | ||
|
||
From here you can upload an image, paint a mask over the image, and then save the mask. | ||
This can be used as input for the DINOv tool. | ||
|
||
```python | ||
import vision_agent as va | ||
|
||
data = { | ||
"prompt": [{"mask": "baggage.png", "image": "baggage_mask.png"}], | ||
"image": "baggage2.png", | ||
} | ||
tool = va.tools.DINOv() | ||
output = res(**data) | ||
image = va.image_utils.overlay_masks("baggage2.png", output) | ||
image = va.image_utils.overlay_bboxes(image, output) | ||
image.show() | ||
``` |