-
Notifications
You must be signed in to change notification settings - Fork 133
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Garbled characters when using OpenAI models to embed Chinese #42
Comments
Hmm weird that the title is showing up just right. Might be a problem with indexing? But not sure why E5 works. 如果你可以分享一下有问题的文件的话,我可以尝试一下复现 (I can try to replicate the problem if you can share the file) |
I managed to confirm the issue. The problem seems to be in Imo openai is not that much superior than e5 when it comes to search in Chinese, so not sure if this should be fixed now or after there's a plan for v0.2---this will be up to @freedmand |
Thanks for your reply. Looking forward to V0.2 improving this issue. |
Garbled characters when using OpenAI models to embed Chinese.
But using --transformer-model intfloat/multilingual-e5-base it's normal.
endoing utf-8
What's the promble?
The text was updated successfully, but these errors were encountered: