Backend Architecture

Jump to bottom Edit New page

Clayton Dunwell edited this page Aug 28, 2015 · 2 revisions

To be filled in more thoroughly...

A/B Elasticsearch boxes
Bulk download data from various APIs
Task scheduler to sync

DATA SOURCE STRATEGY

Congress API

Run unitedstates scrapers and mash legislators together with Influence Explorer

Congress Expenditures

Locally index from https://sunlightfoundation.com/tools/expenditures/

Capitol Words

Locally index a small set of most popular words. Rely on API for other words.

Political Party Time

Bulk download from http://politicalpartytime.org/www/partytime_dump_all.csv and index

Influence Explorer

Foreign Lobbying