Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor gemini and openai #8945

Closed
wants to merge 19 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
128 changes: 113 additions & 15 deletions mindsdb/integrations/handlers/google_gemini_handler/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,35 +6,133 @@ Google Generative AI is a library that provides access to powerful language mode

*Note:* Ensure you have the necessary API key for accessing the Google gen AI library. You can get your API key at https://makersuite.google.com/.

>> This Handler requires python>=3.9 to work
>> (Default model_name is **gemini-pro** )

# Example Usage

Create a ML Engine with the new `google_gemini` engine.
#### Create Gemini ML Engine
```sql
CREATE ML_ENGINE g
FROM gemini
USING
api_key = 'AI-i5-c001';
```


#### Create Gemini Pro Model (Prompt-Template)
```sql
CREATE MODEL gem_p
PREDICT answer
USING
engine = 'g',
prompt_template = 'Product Description: {{description}}. Question: {{question}}. Answer:',
model_name = 'gemini-pro';
```

```sql
SELECT answer
FROM gem_p
WHERE description = "
What is Rabbit R1?
The Rabbit R1 is a pocket-sized AI device that promises a simpler and more intuitive way to interact with technology. Instead of being app-driven, the device relies on an AI model called LAMB (large action model) to understand your instructions and complete tasks autonomously.
The device has a bright orange body, and is small and lightweight with a touchscreen, scroll wheel, and a talk button. There is also a rotating camera that functions as eyes of the device.

The Rabbit R1 runs on its own operating system, called the Rabbit OS, that eliminates the need for app stores and downloads, requiring only natural language voice input to navigate. The initial version supports integration with the likes of Uber, Spotify, and Amazon, with the AI able to train and learn using other apps in the future.
"
AND question = 'What are some key feature bullet points of this product?';
```



#### Create Gemini Contextual Model (Column-based)
```sql
CREATE MODEL gemini_c
PREDICT answer
USING
engine = 'g',
question_column = 'question',
context_column = 'context',
model_name="gemini-pro";
```
```sql
SELECT answer
FROM gem_qc
WHERE context = "Ashoka the Great was an Indian emperor of the Maurya Dynasty who ruled from 268 to 232 BCE. He is regarded as one of India's greatest emperors, known for his extensive empire, his efforts to spread Buddhism, and his commitment to non-violence and peaceful coexistence."
AND question = 'Ashoka was from which dynasty?';
```


#### Vision Mode Query
```sql
CREATE ML_ENGINE gemini_engine
FROM google_gemini
CREATE MODEL gem_v
PREDICT answer
USING
google_gemini_api_key = 'google_gemini_api_key';
engine = 'g',
mode = 'vision',
img_url = 'url',
ctx_column = 'context';
```

```sql
CREATE MODEL mindsdb.Google_Gemini_test
SELECT *
FROM gem_v
WHERE url = 'https://images.unsplash.com/photo-1589762738975-a6773160c7d7?q=80&w=1374&auto=format&fit=crop&ixlib=rb-4.0.3&ixid=M3wxMjA3fDB8MHxwaG90by1wYWdlfHx8fGVufDB8fHx8fA%3D%3D'
AND context='Is this man a superhuman?';
```

#### Embedding Mode
```sql
CREATE MODEL gem_e
PREDICT answer
USING
column = 'question',
engine = 'gemini_engine',
google_gemini_api_key = 'google_gemini_api_key',
model = 'gemini-pro'
engine = 'g',
mode = 'embedding',
model_name = 'models/embedding-001', --- default for embedding mode
question_column = 'question',
context_column = 'context',
title_column = 'title'; --OPTIONAL
```
>> (**title_column** is optional and default model_name for embedding mode is **models/embedding-001'**)

```sql
SELECT question, answer
FROM mindsdb.Google_Gemini_test
WHERE question = 'What is the meaning of life?';
SELECT question, answer
FROM gem_e
WHERE question = 'How many moons are there in the solar system?'
USING
type='document';
```
>>(Use ** type='document'** for storing vectors for later searching and **type='query'** for searching in already created vectors.)

####json-struct Mode
```sql
CREATE MODEL product_extract_json
PREDICT json
USING
engine = 'g',
json_struct = {
'product_name': 'name',
'product_category': 'category',
'product_price': 'price'
},
input_text = 'description';
```
```sql
SELECT t.question, m.answer
FROM mindsdb.Google_Gemini_test as m
JOIN files.question_table as t;
SELECT json
FROM product_extract_json
WHERE description =
"
What is Rabbit R1?
The Rabbit R1 is a pocket-sized AI device that promises a simpler and more intuitive way to interact with technology. Instead of being app-driven, the device relies on an AI model called LAMB (large action model) to understand your instructions and complete tasks autonomously.
The device has a bright orange body, and is small and lightweight with a touchscreen, scroll wheel, and a talk button. There is also a rotating camera that functions as eyes of the device.IT provide all of this just for 300 dollar.

The Rabbit R1 runs on its own operating system, called the Rabbit OS, that eliminates the need for app stores and downloads, requiring only natural language voice input to navigate. The initial version supports integration with the likes of Uber, Spotify, and Amazon, with the AI able to train and learn using other apps in the future.
";
```
**Output**
![image](https://github.com/mindsdb/mindsdb/assets/75653580/aad51d3f-4458-4bcc-b4a3-07983496d2fe)

#### Describe Gemini Pro Model Metadata
```sql
DESCRIBE MODEL `MODEL_NAME`.metadata;
```