This was an attempt to create a GPT-2 Chatbot that knows lifehacks. But it turned out to be a joke in the end.
The goal of this project is to create a question answering bot and host it online using Hugging Face. We started by scraping text-based answers from the website stackexchange.com. Specifically, we scraped the site https://lifehacks.stackexchange.com and collected 9500 questions and their answers.
We used a generic notebook that can scrape any site from stackexchange.com, but it extracts only the text. We chose lifehacks.stackexchange.com because the answers were text-based. We scraped every question and all of its answers.
Our initial attempt was to use a seq2seq model, which was originally used in a medium article for machine translation. We edited the model for our problem and data and trained it for 10 epochs. After more research, we switched to Abstractive Question Answering.
We found that it is possible to fine-tune a GPT model for question answering. We chose the small version of the GPT-2 model for ease of training. After 30k steps of training, we tried the model on Hugging Face but the answers were not accurate. We learned that the data needed to be in a specific format, with (Question:Answer) in one line. We trained the model again for 31k steps with the correct data format.
To create the bot API, we created a space on Hugging Face with two files: app.py, which contains the code that represents the GUI, and requirements.txt, which contains the packages that will be installed in our model's environment. We used Gradio to create the API.
And The Best One:
The project requires Python 3.5 or later and the following libraries:
- NumPy
- pandas
- BeautifulSoup
- gradio
- transformers