Table of Contents
hiabGPT, which stands for "Help, I Accidentally Built GPT" was my personal project to make GPT2 from Scratch. I've used Pytorch and really accidentally built GPT.
Please chech the github tech blog I've uploaded on the bottom.
Full data is about 40,000. It's too much so I just added about 30 datum for example
I was able to make GPT much more faster if my PyTorch was not 2.0.0. PyTorch 2.0.0 sucks. It explodes with nan.
✒️ I am writing blog, 'Tech Blog, I guess?'. It's updating and progressing.
📍 This is based on Attention is all you need and OpenAI GPT2 papers.
🪴 MaskedMultiheadSelfAttention was built with torch.MultiheadAttention.
You need Android Studio and dart, flutter plugin installed. If you are using mac, you need to set up XCode and Simulator.
- Clone the repo
git clone https://github.com/DutchVandaline/GPT_from_Scratch.git
- Excecute on PyCharm
If you liked this app, please buy me a coffee! It helps me do creative stuffs.
Contributions are what make the open source community such an amazing place to learn, inspire, and create. Any contributions you make are greatly appreciated.
If you have a suggestion that would make this better, please fork the repo and create a pull request. You can also simply open an issue with the tag "enhancement". Don't forget to give the project a star! Thanks again!
- Fork the Project
- Create your Feature Branch (
git checkout -b feature/AmazingFeature) - Commit your Changes (
git commit -m 'Add some AmazingFeature') - Push to the Branch (
git push origin feature/AmazingFeature) - Open a Pull Request
Dutch Vanaline - [email protected]
Project Link: https://github.com/DutchVandaline?tab=repositories