Skip to content

News Aggregator boilerplate built for Reddit and NewsAPI, using (Python, FastApi, PyTest, MySQL)

License

Notifications You must be signed in to change notification settings

umairqadir97/news-aggregator-with-fastapi

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

9 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Python 3.6 Issues MIT License LinkedIn


Logo

News Aggregator with FastAPI

Boilerplate news aggregation application build for Reddit and NewsAPI. Main idea is to use lightweight development frameworks like FastAPI rather than heavy weight lifting with Django/ Flask. Further information can be found in Problem Statement document.


Request Demo · Report Bug · Request Feature

Table of Contents

About The Project

Product Name Screen Shot

The application is built in accordance with Problem Statement document. All of the features are implemented. All the contraints are implemented, along with project documentation and unit-tests.


What's Not Implemented

Current version of this project does not implement cache server or job queue to minimize performance dependence on external APIs.


Built With

Getting Started

General Technique to Aggregate

  1. Get list of registered APIs
  2. Loop over APIs
  3. Make JSON object of configurations for each API
  4. Write JSON parsers for each API
  5. Combine result in uniform format to display
  6. Return results

Detailed Approach

Every Request to API should be async; put the request in job tracker, get results and push back responses

News Listing Endpoint:

api.route('/news')

def get_top_news():

  1. Get all available APIs
  2. Call each API for getting top 10 news in JSON; Listing Function for Each API
  3. Aggregate news from all APIs, discard empty responses
News Search Endpoint

api.route('/news?query=bitcoin')

def get_search_results():

  1. Get all available APIs
  2. Call each API with search query to get top 10 results in JSON; Searching Function for Each API
  3. Aggregate results from all APIs, discard empty responses
Other Helper Functions
  1. News_API json parser for listing responses; should return only required fields; ["title", "link", "source"]
  2. Reddit_API json parser for listing results
Steps to Add any New API
  1. Make a new file for new-API in src/external_api/
  2. Take src/external_api/sample_api.py as reference
  3. Prepare API Mapping object
  4. Write JSON parser for that API
  5. Register your API in API_COLLECTION in src/api_helper.py

Prerequisites

To run this project, should install project dependencies:

  1. Python3
  2. pip
  3. Intsall Python packages

Instructions to Run

  1. Clone the repo
git clone https://github.com/umairqadir97/news-aggregator-with-fastapi.git
  1. Open terminal in project folder
cd news-aggregator-with-fastapi
  1. Install python packages
pip3 install -r requirements.txt
  1. Create your configurations file src/config.py. Take src/sample_config.py as example

  2. Run server

python3 api.py
You can run tests with:
pytest

Roadmap

See the open issues for a list of proposed features (and known issues).

Contributing

Contributions are what make the open source community such an amazing place to be learn, inspire, and create. Any contributions you make are greatly appreciated.

  1. Fork the Project
  2. Create your Feature Branch (git checkout -b amazing_features)
  3. Commit your Changes (git commit -m 'Add some Amazing Features')
  4. Push to the Branch (git push origin amazing_features)
  5. Open a Pull Request

Contribution guidelines

  1. Writing more unit tests
  2. Code review
  3. Feature Enhancement

License

Distributed under the MIT License. See LICENSE for more information.

Contact

Muhammad Umair Qadir - Email

LinkedIn: LinkedIn

About

News Aggregator boilerplate built for Reddit and NewsAPI, using (Python, FastApi, PyTest, MySQL)

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages