Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add structured outputs. #7

Merged
merged 3 commits into from
Feb 9, 2025
Merged

Add structured outputs. #7

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
bae45d1
updates
sayakpaul Feb 6, 2025
8ec75ed
updates
sayakpaul Feb 8, 2025
bf71bbe
updates
sayakpaul Feb 8, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions configs/responses.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
from pydantic import BaseModel

class SummaryResponses(BaseModel):
previous_summary: str
updated_summary: str
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

since this is the "response", I think we don't need previous_summary and updated_summary. Instead, just summary.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How do we keep track of the previous summary in a better manner otherwise?

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LLM only generates summary. LLM does not generate previous summary (having previous_summary in response means that we ask LLM to generate previous summary). it is something to be given to the LLM as input (we don't give it as input currently though).

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure I understand this bit:

it is something to be given to the LLM as input (we don't give it as input currently though).

We're already doing it here no:

AdaptSum/main.py

Line 69 in 8ec75ed

previous_summary=state['summary'],

Or am I missing something?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@deep-diver

21 changes: 16 additions & 5 deletions main.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,9 @@
import argparse
import gradio as gr
from difflib import Differ
from functools import partial
from string import Template
from utils import load_prompt, setup_gemini_client
from configs.responses import SummaryResponses

def parse_args():
parser = argparse.ArgumentParser()
Expand Down Expand Up @@ -54,7 +54,7 @@ def echo(message, history, state):

response = client.models.generate_content(
model="gemini-1.5-flash",
contents=state['messages']
contents=state['messages'],
)
model_response = response.text

Expand All @@ -69,7 +69,10 @@ def echo(message, history, state):
previous_summary=state['summary'],
latest_conversation=str({"user": message['text'], "assistant": model_response})
)
]
],
config={'response_mime_type': 'application/json',
'response_schema': SummaryResponses,
},
)

if state['summary'] != "":
Expand All @@ -78,8 +81,16 @@ def echo(message, history, state):
prev_summary = ""

d = Differ()
state['summary'] = response.text
state['summary_history'].append(response.text)
state['summary'] = (
response.parsed.updated_summary
if getattr(response.parsed, "updated_summary", None) is not None
else response.text
)
state['summary_history'].append(
response.parsed.updated_summary
if getattr(response.parsed, "updated_summary", None) is not None
else response.text
)
state['summary_diff_history'].append(
[
(token[2:], token[0] if token[0] != " " else None)
Expand Down