Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(mlx_lm): support batch input in generate() #948

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Commits on Aug 29, 2024

  1. feat: support batch input in generate()

    The `prompt` argument can now be either a `str` or `list[str]`.
    
    The change to `generate()` is backwards-compatible.
    
    The changes to `generate_step()`, `top_p_sampling()`, and
    `min_p_sampling()` are backwards-incompatible in order to unify shapes;
    this could be changed by adding a few if-statements, if preferred.
    llllvvuu committed Aug 29, 2024
    Configuration menu
    Copy the full SHA
    280b378 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    2caa832 View commit details
    Browse the repository at this point in the history

Commits on Oct 9, 2024

  1. Configuration menu
    Copy the full SHA
    8fb82fe View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    9ee726c View commit details
    Browse the repository at this point in the history