-
Notifications
You must be signed in to change notification settings - Fork 988
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Feature request] High-level API support for DRY and XTC samplers #1813
Comments
This patch adds the DRY and XTC samplers:
|
@zpin feel free to open a PR ! |
Done: #1843 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Is your feature request related to a problem? Please describe.
Recently llama.cpp added support for the DRY and XTC samplers which can help reduce repetition and increase creativity without losing coherence. It would be wonderful if users of llama-cpp-python could take advantage of these advanced samplers.
Describe the solution you'd like
Ideally the high-level API would expose parameters so that the end user / developer may use XTC and DRY, in the same way that we can currently use temperature, top-p, min-p, etc. Functions like
Llama.create_completion()
would be updated with these new parameters.Additional context
I would be happy to help in any way I can with the implementation of these samplers, but I'm not sure where to start. @abetlen If there is anything I can do to help get this supported as quickly as possible, please point me in the right direction.
Thank you!
The text was updated successfully, but these errors were encountered: