Should we mine bioguide for biographical info?

[wilson428]

Per #571, I found that the best way to figure out whether a member retired and didn't run for a different office was to crawl every page on bioguide and identify unique phrases that matched existing lists. I'm sure it's not perfect, but so far I haven't found any false positives or negatives.

I wonder if it's worth trying to cautiously mine more information from these little squibs. For example, we were debating at work today whether more or fewer members have been previously elected to lower offices before reaching Congress than in past terms. Military service might also be possible to tease out, though at a glance it's worded differently during different wars.

Could be a fun NLP project if carefully, carefully spot-checked. Happy to provide the raw text of the squibs if anyone wants to have a look.

[dwillis]

I think this would be worthwhile, at least as a guide for updating. FWIW, we do this manually.

[wilson428]

I'll take a hack if time ever permits. Now for the hard part: Are we still a Python shop or can I use Node? Be gentle.

[JoshData]

See #304 where I came up with an over-engineered solution. :)

[wilson428]

I saw that after I wrote this! Will play around.