Define report types

[spulec]

On most of the early scrapers, there was a type field that was used quite a bit to describe the type of report (audit, testimony, etc). I think it would be useful to establish a standard list of values for this and then work on applying it to all of the scrapers.

Some types:

• audit
• investigation
• testimony
• press
• research
• performance
• evaluation
• inspection
• congress
• peer_review
• semiannual_report
• management_challenges
• fisma
• other

Benefits are two-fold: first, it would allow for projects like oversight.io to be able to apply some nice filtering. Second, it would allow for better QA of our dataset. For example, "Oh, we don't have any Peer Reviews for the SSA. We should issue some FOIA requests for those."

Remaining scrapers without a type field:

• dhs
• dod
• doj
• eeoc
• epa
• gao
• hhs
• house
• labor
• loc

[konklone]

Completely agree - this fell by the wayside, and will be very useful, especially now that we have a better understanding of what kinds of reports IGs produce.

[spulec]

Great. I'm not too concerned right now about what the actual list of types is. I'm more concerned with getting something implemented since it should be pretty easy to combine/split/rename things post-implementation. If anyone has opinions on that though, feel free to chime in.

My plan is to go through the scrapers alphabetically and try to apply types to all reports. Anyone else should feel free to jump in too. Once they are all added, I think it makes sense to make type a required field going forward.

[spulec]

I added a bunch of these, but there are 10 remaining that are a bit trickier. I've added them to the issue description.