Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Mixtral scaling: Reduce perplexity from 4.294 to 4.269 #301

Open
wants to merge 10 commits into
base: main
Choose a base branch
from
Open

Mixtral scaling: Reduce perplexity from 4.294 to 4.269 #301

wants to merge 10 commits into from

Commits on Jan 3, 2024

  1. Mixtral Scaling [WIP]

    @casper-hansen
    casper-hansen committed Jan 3, 2024
    Configuration menu
    Copy the full SHA
    4d83875 View commit details
    Browse the repository at this point in the history

  2. Mixtral individual expert scaling

    @casper-hansen
    casper-hansen committed Jan 3, 2024
    Configuration menu
    Copy the full SHA
    ff37720 View commit details
    Browse the repository at this point in the history

Commits on Jan 4, 2024

  1. Various fixes

    @casper-hansen
    casper-hansen committed Jan 4, 2024
    Configuration menu
    Copy the full SHA
    0a6a6d0 View commit details
    Browse the repository at this point in the history

  2. Refactor quantization

    @casper-hansen
    casper-hansen committed Jan 4, 2024
    Configuration menu
    Copy the full SHA
    aec278e View commit details
    Browse the repository at this point in the history

  3. Minor fixes and type hinting

    @casper-hansen
    casper-hansen committed Jan 4, 2024
    Configuration menu
    Copy the full SHA
    3a581a5 View commit details
    Browse the repository at this point in the history

Commits on Jan 9, 2024

  1. Lower perplexity

    @casper-hansen
    casper-hansen committed Jan 9, 2024
    Configuration menu
    Copy the full SHA
    43f21c9 View commit details
    Browse the repository at this point in the history

  2. Minor fixes, code cleaning

    @casper-hansen
    casper-hansen committed Jan 9, 2024
    Configuration menu
    Copy the full SHA
    112de41 View commit details
    Browse the repository at this point in the history

Commits on Jan 21, 2024

  1. Backward compatibility

    @casper-hansen
    casper-hansen committed Jan 21, 2024
    Configuration menu
    Copy the full SHA
    7525fb2 View commit details
    Browse the repository at this point in the history

  2. Rework modules_to_not_convert

    @casper-hansen
    casper-hansen committed Jan 21, 2024
    Configuration menu
    Copy the full SHA
    6c065ab View commit details
    Browse the repository at this point in the history

  3. Merge branch 'main' into mixtral_scaling

    @casper-hansen
    casper-hansen committed Jan 21, 2024
    Configuration menu
    Copy the full SHA
    02ab075 View commit details
    Browse the repository at this point in the history