-
Notifications
You must be signed in to change notification settings - Fork 37
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Questions on creating instruction data #13
Comments
Hey, thanks for the thorough investigation!
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Thanks for the great work!
I have a few questions regarding data creation of xP3 after following the guide here to create instruction data on the
code
language subset.I noticed the total samples of the public processed data (from here) on the
code
split is 2707724. However, my resulting data following the above github guide is much more than that (approximately >3M samples). I wonder if there were any additional post-processing to get the final instruction data for tuning?Following the above github guide, I noticed there was no prompt for this particular dataset State Changes. I got this warning when running the creation code:
Tried instantiating `DatasetTemplates` for Fraser/python-state-changes, but no prompts found. Please ignore this warning if you are creating new prompts for this dataset.
Is this dataset not assigned with any prompt (similar to how HumanEval was treated). Or is the below version of PromptSource I used is not correct:
git clone -b tr13 https://github.com/Muennighoff/promptsource.git & install cd promptsource; pip install -e .
The text was updated successfully, but these errors were encountered: