You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We saw lawyer-llama built by using SFT on a dataset of legal QA pairs. Recently Saul-LM which was fine-tuned on Mistral achieves good performance as well.
However, I've seen Qwen-72B performing exceptionally well just with prompt tuning, would it be feasible to do continual pre-training and would it possibly improve accuracy and conversation quality?
Use cases:
Adding clauses to draft
Drafting using prompt
Legal QA , for ex: legal limit of drinking or how to file for bankrupty
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
We saw lawyer-llama built by using SFT on a dataset of legal QA pairs. Recently Saul-LM which was fine-tuned on Mistral achieves good performance as well.
However, I've seen Qwen-72B performing exceptionally well just with prompt tuning, would it be feasible to do continual pre-training and would it possibly improve accuracy and conversation quality?
Use cases:
Beta Was this translation helpful? Give feedback.
All reactions