WeatherFlow

An AirFlow project to gather data from OpenWeatherMap API at regular intervals.

Setup

AirFlow

Create a virtual environment, activate environment
Install requirements.txt
Run airflow scheduler in a terminal from project directory
Run airflow webserver in another terminal from project directory
Open localhost:8080 in browser
Go to Admin -> Variables and add values for API_KEY1, API_KEY2, MYSQL_DB, MYSQL_HOST, MYSQL_USER, MYSQL_PSWD
Enable DAG from DAGs menu

Database (MySQL)

Install MySQL community server
Create a user or use Admin user
Create tables by running SQL/create_tables.sql

Database Schema

This project follows star schema in a data warehouse