Skip to content

Latest commit

 

History

History
21 lines (16 loc) · 617 Bytes

README.md

File metadata and controls

21 lines (16 loc) · 617 Bytes

Data Pipeline

Make sure you are in this directory. Also make sure to put veritas-trial-deployment.json under the secrets/ folder in the root directory. To run the data pipeline:

make build
make run

This should bring you within the Docker container. Then inside the container, run:

python cli.py fetch
python cli.py clean
python cli.py upload

Note

  • See clean.py for the structure of the cleaned data.
  • Use --help on cli.py to get an overview of what each subcommand does.
  • This step is automated in the "Deploy pipeline" workflow with GitHub Actions.