Welcome to the most kickass event for all you Actually Open AI aficionados out there: The Fine-Tuning Llama Hackathon! Llama 2 has gone open-ish source and CodeLlama is just around the corner. But we need you to finesse it further. Bring your peculiar data sets and let's build some killer chains and agents.
Think of this as a hackathon and a meetup combined. Learn, hack, and win!
| Speaker | Affiliation |
|---|---|
| Harrison Chase | CEO, Langchain |
| Robert Nishihara | CEO, Anyscale |
| Ben Firshman | CEO, Replicate |
| Will Bryk | CEO, Metaphor |
| Michael Royzen | CEO, Phind |
| Mark Huang | CEO, Gradient AI |
| Yuze Ma | Engineer @ Lepton AI |
| Sam L'Huillier | Finetune God @ Brev.dev |
- 9:00am: Meet & Greet
- 9:30am: Talks & Intro to Fine-tuning
- 12:00pm: Lunch & Teamforming
- 1:00pm: Hacking Begins
- 4:00pm: Presentations & Awards
🎨 Most Creative
💸 Most Expensive
🤝 Most Practical
No cash prizes, just bragging rights
- Structured Data
- YAML
- JSON
- Markdown tables
- Code
- finetune for Python, JS, React, Other code languages
- finetune for JS→ TypeScript, Python → typehints?
- finetune CodeLlama
- Text Generation
- Creative writing
- Style Replicas - imitate a particular person’s style
- Other forms of creative output
- drawing ascii art - pator
- poetry? rap? Music?
- finetune
- Newer Techniques
- Gorilla reproduction
- RETRO reproduction
- SequenceMatch reproduction
- CodeLlama reproduction
Check out the example datasets in this repo.
- astrology-dataset.jsonl
- socialmediamanager-dataset.jsonl
- please PR more!
- https://platform.openai.com/docs/guides/fine-tuning/fine-tuning-examples
- https://brev.dev/blog/fine-tuning-llama-2
- https://replicate.com/blog/fine-tune-llama-2
- https://huggingface.co/docs/transformers/training
- Talk: WandB on Training and Fine-tuning LLMs
- Notebook: fine-tune LLaMA 2 with qlora via matt shumer
- (Please PR more!)
We've got awards for the weirdest, most practical, and most creative applications of fine-tuning. But remember, the real prize is the friends finetunes we made along the way!
- Hosted by: brev.dev, ai.engineer, celo.org - Contact Nader or Swyx.
- Special Thanks: Amplify & Celo for being awesome sponsors
Be cool and respectful. Check the full code of conduct here.
Questions? Hit us up! Looking forward to finetuning some llamas with y'all!