-
Notifications
You must be signed in to change notification settings - Fork 64
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support gpt-4-vision-preview
#247
Comments
As far as I can tell we're limited by the library we use for API communication, which does not yet support vision. Although I'm very interested and will check what we can do as soon as the library adds support. |
I open up a request. |
I think we might consider dropping the third party SDK and switch to the official node package from openai: https://github.com/openai/openai-node#readme which seems to support vision |
Going for the official node library seems like the best option for long-term sustainability and rapid adoption of new features |
Yes but we would then be responsible for handling context, which is fine if someone is willing to write the code. |
https://platform.openai.com/docs/guides/vision
It seems like uploading base64-encoded images may be a generic viable strategy for passing images through the API.
Alternatively/for speed, and from unencrypted rooms, it may instead be possible/desirable to pass an image URL by transforming the image
mxc
url to anhttps
url via theimage_url
key.The text was updated successfully, but these errors were encountered: