Skip to content

Commit

Permalink
updated readme
Browse files Browse the repository at this point in the history
  • Loading branch information
dillonalaird committed Apr 22, 2024
1 parent 03c2480 commit 9effc1f
Showing 1 changed file with 27 additions and 2 deletions.
29 changes: 27 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ pip install vision-agent
```

Ensure you have an OpenAI API key and set it as an environment variable (if you are
using Azure OpenAI please see the additional setup section):
using Azure OpenAI please see the Azure setup section):

```bash
export OPENAI_API_KEY="your-api-key"
Expand Down Expand Up @@ -96,6 +96,31 @@ you. For example:
}]
```

#### Custom Tools
You can also add your own custom tools for your vision agent to use:

```python
>>> from vision_agent.tools import Tool, register_tool
>>> @register_tool
>>> class NumItems(Tool):
>>> name = "num_items_"
>>> description = "Returns the number of items in a list."
>>> usage = {
>>> "required_parameters": [{"name": "prompt", "type": "list"}],
>>> "examples": [
>>> {
>>> "scenario": "How many items are in this list? ['a', 'b', 'c']",
>>> "parameters": {"prompt": "['a', 'b', 'c']"},
>>> }
>>> ],
>>> }
>>> def __call__(self, prompt: str) -> int:
>>> return len(prompt)
```
This will register it with the list of tools Vision Agent has access to. It will be able
to pick it based on the tool description and use it based on the usage provided.

#### Tool List
| Tool | Description |
| --- | --- |
| CLIP | CLIP is a tool that can classify or tag any image given a set of input classes or tags. |
Expand All @@ -112,7 +137,7 @@ you. For example:

It also has a basic set of calculate tools such as add, subtract, multiply and divide.

### Additional Setup
### Azure Setup
If you want to use Azure OpenAI models, you can set the environment variable:

```bash
Expand Down

0 comments on commit 9effc1f

Please sign in to comment.