Semantic Search Engine for Scientific Literature

NASA DT Hackathon
By the Vizzies (GVIS)

Challenge: Develop an intelligent literature search and analysis application that reads NASA-relevant scientific articles using natural language processing and text mining, returns knowledge analysis and relevant scientific findings, and reveals knowledge gaps and multi-disciplinary research opportunities.
Time Frame: about 15 hours

Setup

BERT Model - Semantic-Search.ipynb

1. Clone the entire repository

$ git clone https://github.com/vizzies/NASA-Semantic-Search-Engine-for-Scientific-Literature.git
$ cd NASA-Semantic-Search-Engine-for-Scientific-Literature

2. Open a Google Colab Environment and import semantic-search.ipynb
3. Go into edit > notebook settings under hardware accelerator and select GPU
4. To run each cell, press SHIFT + Return
5. Under the heading BERT Sentence Transformer Semantic Search, in the queries list, add your scientific query of choice
6. Run that cell to recieve searched literature documents

Figma Mockup - Clickable Prototype

1. Create a background frame #F3F3F3 for web application
2. Add in search bar for semantic query search
3. Add title and hackathon team logo into the pseudo-navigation bar
   • create footer similarly
4. Add placeholder for advanced search (future implementation?)
5. Create data cards for returned results of publications. Metadata received on card preview includes:
   • publish dates
   • abstract character length
   • relevancy score
   • main author(s)
6. Place pagination function buttons previous and next at the bottom
   • emphasize current page navigation number at bottom of page
7. Export CSS code from each object mentioned above for Flask front-end implementation
   • select object > go to code in side menu > CRTL + c

Flask Application Setup

A microservice written in Python with the Flask micro framework. Source code located at https://github.com/vizzies/bertsearch.

Contents

1. Set Up Environment
   1. virtualenv environment
   2. conda environment
2. Initialize Flask App

Set Up Environment

virtualenv environment

1. Clone the repo

git clone https://github.com/vizzies/bertsearch.git

2. cd into the new directory

cd bertsearch

3. Initiate a new virtual environment and store in the project's env directory

python -m virtualenv env

4. Activate the new virtual environment

source env/bin/activate

5. Install dependencies in new environment

pip install -r requirements.txt

conda environment

1. Clone the repo

git clone https://github.com/vizzies/bertsearch.git

2. cd into the new directory

cd bertsearch

3. Create a new conda environment in the project's env directory

conda create python=3.8.3 --prefix ./env

4. Activate the new environment

conda activate ./env

5. Install dependencies in new environment

pip install -r requirements.txt

Initialize Flask App

3. Initialize the app database

flask db init

4. Generate the first database migration

flask db migrate

5. Run database the migration upgrade script

flask db upgrade

6. Run the flask app

flask run

7. You should now be able to make a call to your API. Paste this URL into your browser and check for a valid response

http://127.0.0.1:8080/api/v1/results

Contributors

• Herb Schilling, GRC, Computer Scientist, [email protected]
• Calvin Robinson, GRC, Data Architect, [email protected]
• Evan "Taylor" Yates, MSFC, Software Engineer, [email protected]
• Gulsum Oz, ARC, Information Tech Specialist, [email protected]
• Blake Lafuente, GRC, AST Data Systems, [email protected]
• Shruti Janardhanan, GRC, Intern, [email protected]
• Kelci Mensah, GRC, Intern, [email protected]
• Samantha "Sam" Stesch, GRC, Intern, [email protected]
• Heather Sulier, GRC, Intern, [email protected]