-
Notifications
You must be signed in to change notification settings - Fork 168
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Parsing the output of a model #45
Comments
Hi, our current approach involves directing the model to specifically output the desired labels ('positive' or 'negative'), which are then enclosed within a unique pattern (e.g., '<<<>>>'). This format allows for easy extraction using regular expressions. |
im getting issues regarding qqp dataset evaluation for diff attacks. it gives me dataset_name attribute error when I run attacks |
Hi, I have tested qqp dataset with Flan-T5 model using StressTest attack, it works for me. Could you please test it and paste the detailed error messages here? |
Stale issue message |
The output of the model largely depends on the prompt. When performing text classification tasks such as sst2, which requires the model to give negative or positive responses, the model may provide various forms of answers, and the form of answers from different models may vary. Is there a good way to parse these results?
The text was updated successfully, but these errors were encountered: