Skip to content

Commit

Permalink
exclude citation from openai request
Browse files Browse the repository at this point in the history
  • Loading branch information
Pavan Kumar committed Aug 21, 2024
1 parent 4448980 commit 69e9d0d
Show file tree
Hide file tree
Showing 3 changed files with 3 additions and 3 deletions.
2 changes: 1 addition & 1 deletion ResearchAssistant/App/app.py
Original file line number Diff line number Diff line change
Expand Up @@ -152,7 +152,7 @@ def prepare_body_headers_with_data(request):

messages = [{"role": "system", "content": AZURE_OPENAI_SYSTEM_MESSAGE}]
for message in request_messages:
if message:
if message and message.get("role") != "tool":
messages.append({"role": message["role"], "content": message["content"]})

body = {
Expand Down
2 changes: 1 addition & 1 deletion ResearchAssistant/Deployment/bicep/main.bicep
Original file line number Diff line number Diff line change
Expand Up @@ -170,7 +170,7 @@ module appserviceModule 'deploy_app_service.bicep' = {
AzureOpenAIModelName:'gpt-4o'
AzureOpenAITemperature:'0'
AzureOpenAITopP:'1'
AzureOpenAIMaxTokens:'1000'

This comment has been minimized.

Copy link
@Vinay-Microsoft

Vinay-Microsoft Aug 22, 2024

@Roopan-Microsoft @Pavan-Microsoft What could be the implications of this change? I think we should consult James for this change just a suggestion

This comment has been minimized.

Copy link
@Pavan-Microsoft

Pavan-Microsoft Aug 26, 2024

@Roopan-Microsoft @Pavan-Microsoft What could be the implications of this change? I think we should consult James for this change just a suggestion

After upgrading model to gpt-4o we started getting ratelimit/cognitive search related error.
Based on suggetion from Nalini, We've excluded citations from the OpenAI request body and also made the following changes:
AI Search SKU: Upgraded from Basic to Standard
Rate Limit (Tokens per minute): Increased from 30k to 60k
AZURE_OPENAI_MAX_TOKENS: Adjusted from 1000 to 800

These maxtoken changes will reduce the response length, since we're using this response as the request body for the next API call This was identified as one of the root causes of the rate limit issue. We will discuss this further with Nalini once she's back.

AzureOpenAIMaxTokens:'800'
AzureOpenAIStopSequence:''
AzureOpenAISystemMessage:'''You are a research grant writer assistant chatbot whose primary goal is to help users find information from research articles or grants in a given search index. Provide concise replies that are polite and professional. Answer questions truthfully based on available information. Do not answer questions that are not related to Research Articles or Grants and respond with "I am sorry, I don’t have this information in the knowledge repository. Please ask another question.".
Do not answer questions about what information you have available.
Expand Down
2 changes: 1 addition & 1 deletion ResearchAssistant/Deployment/bicep/main.json
Original file line number Diff line number Diff line change
Expand Up @@ -1444,7 +1444,7 @@
"value": "1"
},
"AzureOpenAIMaxTokens": {
"value": "1000"
"value": "800"
},
"AzureOpenAIStopSequence": {
"value": ""
Expand Down

0 comments on commit 69e9d0d

Please sign in to comment.