Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Verify minicpm 2b dpo #738

Closed

Conversation

pohsengl
Copy link

@pohsengl pohsengl commented Aug 3, 2024

This should close #261, I have updated the model list, and added tests taking reference from issue description at #259.

@Wovchena, Please let me know if there is any changes needed. Thanks!

@ilya-lavrenov ilya-lavrenov added this to the 2024.4 milestone Aug 5, 2024
@@ -349,6 +349,55 @@ jobs:
&& timeout 50s ./build/samples/cpp/beam_search_causal_lm/beam_search_causal_lm ./notus-7b-v1/ 69
| diff <(timeout 50s ./samples/python/beam_search_causal_lm/beam_search_causal_lm.py ./notus-7b-v1/ 69) -

cpp-beam_search_causal_lm-MiniCPM-2B-dpo:
runs-on: ubuntu-20.04-16-cores
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
runs-on: ubuntu-20.04-16-cores
runs-on: ubuntu-20.04

Try a smaller runner. If it fails, switch to ubuntu-20.04-4-cores

source ./ov/setupvars.sh
python -m pip install --upgrade-strategy eager -r ./samples/requirements.txt --pre --extra-index-url https://storage.openvinotoolkit.org/simple/wheels/nightly
python -m pip install ./thirdparty/openvino_tokenizers/[transformers] --pre --extra-index-url https://storage.openvinotoolkit.org/simple/wheels/nightly
optimum-cli export openvino --trust-remote-code --weight-format fp16 --model openbmb/MiniCPM-2B-dpo-fp16 MiniCPM-2B-dpo-fp16
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The model doesn't have a license. https://huggingface.co/openbmb/MiniCPM-2B-dpo-fp16/tree/main. We can't test it until it gets a compatible license. Please, submit an issue to https://huggingface.co/openbmb/MiniCPM-2B-dpo-fp16/ asking to add the license. I didn't think about it earlier. Sorry. I updated #259 to reflect the license question.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There was

  1. Extend the nightly_models list with the given model and run nightly tests for that model: https://github.com/openvinotoolkit/openvino.genai/tree/master/tests/python_tests#customise-tests-run. Report if there are failing tests and comment out the model in the nightly_models list. Add this change to your pull request (PR).

recently added. Please do that after MiniCPM-2B-dpo-fp16 gets a compatible license.

@andrei-kochin andrei-kochin removed this from the 2024.4 milestone Sep 9, 2024
@mlukasze
Copy link

hey @pohsengl will you find a time to update a PR, please?

@ilya-lavrenov ilya-lavrenov added the category: GHA CI based on Github actions label Oct 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
category: GHA CI based on Github actions
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Good First Issue]: Verify mini-cpm-2b-dpo with GenAI text_generation
5 participants