Skip to content

ShubhAgarwal-dev/languagemodels

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

17 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

languagemodels

This is the the official DevHack 4.0 Hackathon Repository. Participants list:

  • Divy Jain (210010015)
  • Vivek Pillai (210010058)
  • Karthik Hegde (210010022)
  • Shubh Agarwal (210020047)

Report Link
VIDEO GUIDES

Installation Guide

We assume Linux environment for local installation, python>=3.9 is required. Follow the given steps to replicate:

# the steps given in this block are optional, but RECOMMENDED 
sudo apt-get install python3-virtualenv
# go to languagemodels directory, the one you will get after cloning
virtualenv ./env
source ./env/bin/activate

The replication parts starts from below:

git clone https://github.com/ShubhAgarwal-dev/languagemodels.git
cd languagemodels
mkdir data
# activate the environment using source ./env/bin/activate, if made
pip install "fastapi[all]"
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu
pip install python-multipart
pip install transformers

For Summary Model:

pip install sentencepiece

For Question-Answer Pair Generator Model:

pip install nltk
python -m nltk.downloader punkt
pip install protobuf==3.15.8

For Translation Model:

pip install sentencepiece
pip install sacremoses

For Question Answering Model:

pip install openai

For PDF Text-Retriver:

pip install PyPDF2

Now to start the server

uvicorn main:app --reload

If you want to run API Server and client on different machines connected to the same network, you can get your IP using ifconfig:

uvicorn --host <YOUR_IP_HERE> main:app --reload

If running the server for the first time, it is going to take a lot of time as a lot of models will be downloaded locally. Good internet connection is recommended.

After everything is done, you can go to localhost:8000/docs or <YOUR_IP_HERE>:8000/docs for using the interface.

Testing the project

Sample Research Paper

After visiting localhost:8000/docs or <YOUR_IP_HERE>:8000/docs you can try any of the listed actions.

Generate OPENAI api key

Refer to this article and place the generated key in api_nlp.py.

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages