Skip to content

The withStreamUsage() of the OpenAiChatOptions is not supported in AzureOpenAiChatOptions. #1724

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
hy4470 opened this issue Nov 12, 2024 · 4 comments

Comments

@hy4470
Copy link

hy4470 commented Nov 12, 2024

When using Azure OpenAI, the 'usage' metadata is always returned as empty in the stream().

I understand that 'usage' is supported in the latest OpenAIServiceVersion, V2024_08_01_PREVIEW. Therefore, the AzureOpenAiChatOptions should also support the withStreamUsage() method to align with this.

@garethjevans
Copy link
Contributor

Related Issue: Azure/azure-sdk-for-java#40998

@andresssantos
Copy link

Can we implement this feature now that the issue in the Azure SDK has been resolved?

@markpollack
Copy link
Member

@andresssantos yes, it seems like this should be possible if you want to take a look, that would be appreciated.

@andresssantos
Copy link

@markpollack, opened PR #2858.

This is my first PR for Spring projects, please feel free to review and share any feedback or suggestions for improvement. I’m open to making adjustments as needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

6 participants