ATL City Council Scraper Workflow #659
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
name: ATL City Council Scraper Workflow | |
on: | |
schedule: | |
- cron: '0 11 * * *' # 11am daily. Ref https://crontab.guru/examples.html | |
workflow_dispatch: | |
jobs: | |
build: | |
runs-on: ubuntu-latest | |
steps: | |
- uses: actions/checkout@v2 | |
- name: Set up Python 3.8 | |
uses: actions/setup-python@v2 | |
with: | |
python-version: 3.8 | |
- name: Install dependencies | |
run: | | |
python -m pip install --upgrade pip | |
if [ -f requirements.txt ]; then pip install -r requirements.txt; fi | |
- name: Scrape the site | |
run: | | |
python3 -m app scrape > scraped/atl-citycouncil.json | |
- name: Commit changes | |
run: | | |
git config user.name "Automated scrape commit." | |
git config user.email "[email protected]" | |
git add -A | |
timestamp=$(date -u) | |
git commit -m "Updated scraped data: ${timestamp}" || exit 0 | |
git push |