You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
Currently, our 50-a Scraper does not properly capture officer data. First, officers are not being associated with their Units. We are collecting the unit names, but we aren't taking the step of connecting each officer to the unit(s) that they've worked for. Second, we aren't properly collecting the complaints associated with each officer. We are collecting the dispositions of the complaints, but we aren't associating complaint data with individual officers.
Describe the solution you'd like
When scraping officer data from 50-a.org, make the following adjustments:
Include a list of complaint numbers associated with the officer.
Include the Tax Number for each officer
This means an entry in the JSON output might change from this:
{"scraped_at": "2024-05-15 14:17:28", "name": "24th Precinct", "url": "https://www.50-a.org/command/24pct"}, "website_url": "https://www1.nyc.gov/site/nypd/bureaus/patrol/precincts/24th-precinct.page", "commanding_officer": "https://www.50-a.org/officer/KYGH", "address": "151 W 100th St, New York, NY 10025", "description": "The 24th Precinct is located on the Upper West Side of Manhattan and encompasses Manhattan Valley and a portion of Riverside Park. It is a residential and commercial community of multiple dwelling homes and one major housing development.", "officers": [{"url": "https://www.50-a.org/officer/WHJ5", "most_recent": 2024}, {"url": "https://www.50-a.org/officer/4JJ9", "most_recent": 2024}, {"url": "https://www.50-a.org/officer/J7Y3", "most_recent": 2023}]}
Additional context
The text was updated successfully, but these errors were encountered:
Is your feature request related to a problem? Please describe.
Currently, our 50-a Scraper does not properly capture officer data. First, officers are not being associated with their Units. We are collecting the unit names, but we aren't taking the step of connecting each officer to the unit(s) that they've worked for. Second, we aren't properly collecting the complaints associated with each officer. We are collecting the dispositions of the complaints, but we aren't associating complaint data with individual officers.
Describe the solution you'd like
When scraping officer data from 50-a.org, make the following adjustments:
This means an entry in the JSON output might change from this:
To this:
When scraping command data, make the following adjustments:
Therefore this:
{"scraped_at": "2024-05-15 14:17:28", "name": "24th Precinct", "url": "https://www.50-a.org/command/24pct"}
Will become this:
{"scraped_at": "2024-05-15 14:17:28", "name": "24th Precinct", "url": "https://www.50-a.org/command/24pct"}, "website_url": "https://www1.nyc.gov/site/nypd/bureaus/patrol/precincts/24th-precinct.page", "commanding_officer": "https://www.50-a.org/officer/KYGH", "address": "151 W 100th St, New York, NY 10025", "description": "The 24th Precinct is located on the Upper West Side of Manhattan and encompasses Manhattan Valley and a portion of Riverside Park. It is a residential and commercial community of multiple dwelling homes and one major housing development.", "officers": [{"url": "https://www.50-a.org/officer/WHJ5", "most_recent": 2024}, {"url": "https://www.50-a.org/officer/4JJ9", "most_recent": 2024}, {"url": "https://www.50-a.org/officer/J7Y3", "most_recent": 2023}]}
Additional context
The text was updated successfully, but these errors were encountered: