Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

llama : save and restore kv cache for single seq id #6341

Merged
merged 34 commits into from Apr 8, 2024
Merged

llama : save and restore kv cache for single seq id #6341

merged 34 commits into from Apr 8, 2024

Commits on Mar 27, 2024

  1. llama : save and restore kv cache for single seq id

    @kaetemi
    kaetemi committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    662aaea View commit details
    Browse the repository at this point in the history

  2. remove trailing whitespace

    @kaetemi
    kaetemi committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    5462817 View commit details
    Browse the repository at this point in the history

  3. respond error in case there's no space in the kv cache

    @kaetemi
    kaetemi committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    ab1c46a View commit details
    Browse the repository at this point in the history

  4. add kv seq save restore to test case

    @kaetemi
    kaetemi committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    02a1840 View commit details
    Browse the repository at this point in the history

  5. add --slot-save-path arg to enable save restore and restrict save loc… 

    …ation
    @kaetemi
    kaetemi committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    b8e8fac View commit details
    Browse the repository at this point in the history

  6. Returning 0 for some cases, instead of asserting.

    @martindevans
    martindevans committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    b182f8f View commit details
    Browse the repository at this point in the history

  7. cleanup error cases

    @kaetemi
    kaetemi committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    a2b48b9 View commit details
    Browse the repository at this point in the history

Commits on Mar 28, 2024

  1. rename sequence state functions

    @kaetemi
    kaetemi committed Mar 28, 2024
    Configuration menu
    Copy the full SHA
    c4443d7 View commit details
    Browse the repository at this point in the history

  2. rename state get set functions

    @kaetemi
    kaetemi committed Mar 28, 2024
    Configuration menu
    Copy the full SHA
    4d5356b View commit details
    Browse the repository at this point in the history

Commits on Mar 29, 2024

  1. add previous function names back in with DEPRECATED notice

    @kaetemi
    kaetemi committed Mar 29, 2024
    Configuration menu
    Copy the full SHA
    bbcbf47 View commit details
    Browse the repository at this point in the history

  2. update doc

    @kaetemi
    kaetemi committed Mar 29, 2024
    Configuration menu
    Copy the full SHA
    8b5ae29 View commit details
    Browse the repository at this point in the history

  3. adjust endpoints to preferred style

    @kaetemi
    kaetemi committed Mar 29, 2024
    Configuration menu
    Copy the full SHA
    a71ec3d View commit details
    Browse the repository at this point in the history

  4. fix restoring zero cell count

    @kaetemi
    kaetemi committed Mar 29, 2024
    Configuration menu
    Copy the full SHA
    bf1d493 View commit details
    Browse the repository at this point in the history

  5. handle seq rm return value

    @kaetemi
    kaetemi committed Mar 29, 2024
    Configuration menu
    Copy the full SHA
    8ab1a17 View commit details
    Browse the repository at this point in the history

  6. unused param

    @kaetemi
    kaetemi committed Mar 29, 2024
    Configuration menu
    Copy the full SHA
    0d22136 View commit details
    Browse the repository at this point in the history

  7. keep in the size check

    @kaetemi
    kaetemi committed Mar 29, 2024
    Configuration menu
    Copy the full SHA
    29f18c2 View commit details
    Browse the repository at this point in the history

  8. fix return types

    @kaetemi
    kaetemi committed Mar 29, 2024
    Configuration menu
    Copy the full SHA
    f2e41b3 View commit details
    Browse the repository at this point in the history

  9. add server test case for slot save restore

    @kaetemi
    kaetemi committed Mar 29, 2024
    Configuration menu
    Copy the full SHA
    92c4681 View commit details
    Browse the repository at this point in the history

  10. cleanup

    @kaetemi
    kaetemi committed Mar 29, 2024
    Configuration menu
    Copy the full SHA
    60f685f View commit details
    Browse the repository at this point in the history

Commits on Mar 30, 2024

  1. add cake

    @kaetemi
    kaetemi committed Mar 30, 2024
    Configuration menu
    Copy the full SHA
    d38eef4 View commit details
    Browse the repository at this point in the history

  2. cleanup style

    @kaetemi
    kaetemi committed Mar 30, 2024
    Configuration menu
    Copy the full SHA
    ea717f7 View commit details
    Browse the repository at this point in the history

  3. add special

    @kaetemi
    kaetemi committed Mar 30, 2024
    Configuration menu
    Copy the full SHA
    b509b8b View commit details
    Browse the repository at this point in the history

  4. removing a whole sequence never fails

    @kaetemi
    kaetemi committed Mar 30, 2024
    Configuration menu
    Copy the full SHA
    129b6ff View commit details
    Browse the repository at this point in the history

  5. move sequence state file functionality from server to llama to match … 

    …session api and add version tags
    @kaetemi
    kaetemi committed Mar 30, 2024
    Configuration menu
    Copy the full SHA
    8af7211 View commit details
    Browse the repository at this point in the history

Commits on Apr 1, 2024

  1. catch exceptions on save as well

    @kaetemi
    kaetemi committed Apr 1, 2024
    Configuration menu
    Copy the full SHA
    3d6fa5b View commit details
    Browse the repository at this point in the history

  2. error log messages

    @kaetemi
    kaetemi committed Apr 1, 2024
    Configuration menu
    Copy the full SHA
    b3f6da3 View commit details
    Browse the repository at this point in the history

  3. check types for stricter restore

    @kaetemi
    kaetemi committed Apr 1, 2024
    Configuration menu
    Copy the full SHA
    be714a0 View commit details
    Browse the repository at this point in the history

  4. update server doc

    @kaetemi
    kaetemi committed Apr 1, 2024
    Configuration menu
    Copy the full SHA
    0ccfbf2 View commit details
    Browse the repository at this point in the history

Commits on Apr 4, 2024

  1. readme : update API changes date

    @ggerganov
    ggerganov committed Apr 4, 2024
    Configuration menu
    Copy the full SHA
    205c44c View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into feature/save-restore-seq

    @kaetemi
    kaetemi committed Apr 4, 2024
    Configuration menu
    Copy the full SHA
    d9fd0d7 View commit details
    Browse the repository at this point in the history

Commits on Apr 5, 2024

  1. strict filename validation

    @kaetemi
    kaetemi committed Apr 5, 2024
    Configuration menu
    Copy the full SHA
    f2a4777 View commit details
    Browse the repository at this point in the history

  2. move include, reject bom as well

    @kaetemi
    kaetemi committed Apr 5, 2024
    Configuration menu
    Copy the full SHA
    4a4f399 View commit details
    Browse the repository at this point in the history

  3. also reject empty filename

    @kaetemi
    kaetemi committed Apr 5, 2024
    Configuration menu
    Copy the full SHA
    2fbf0c3 View commit details
    Browse the repository at this point in the history

  4. reject whitespace and trailing dot

    @kaetemi
    kaetemi committed Apr 5, 2024
    Configuration menu
    Copy the full SHA
    bf94e9f View commit details
    Browse the repository at this point in the history