Finetune with/without flash attention #983

Unanswered

chiroiu96 asked this question in Q&A

chiroiu96
Jan 10, 2024

I am trying to finetune LLaVA on a custom dataset. Flash attention is not supported by my GPU.

Do I necessarily need to finetune with flash attention?
Will results differ with or without flash attention?

Replies: 0 comments

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment