This is project 4 of Udacitys Data Engineering Nanodegree. In this project Spark is used to pull data from a s3 bucket. Temporary tables are then created, and data is uploaded to another bucket written in .parquet format.
- Add global aws config values in dl.cfg
- Execute "etl.py".