Handing HK, Taiwan data moving forward

[beansrowning]

• Closes Update data source for HK, Macau, and Taiwan #83

Decision points

• JHU is shut as of 3/10 (Johns Hopkins Has Ceased Live COVID-19 Data Reporting as of March 10, 2023 CSSEGISandData/COVID-19#6651)
• WHO is the only source of mainland china data as of early January (China has stopped reporting cumulative cases - alternatives being explored but thus far not identified CSSEGISandData/COVID-19#6414)
  • Though WHO only provides aggregated China data (inclusive of Taiwan, HK, Macau)
• OWID will not separately scrape for HK, Macau, Taiwan data after JHU sunset (Our World in Data will rely on data from the WHO to track confirmed COVID-19 cases and deaths owid/covid-19-data#2784)
• Prospective scraping of Macau data may be out of scope
  • It seems the only source are PDF updates, which may be standardized, but would be a larger lift to incorporate, and we'd need to warehouse
  • https://www.ssm.gov.mo/portal/news/ch/ss.aspx
• Even in the lack of Mainland China disaggregated data, there's still a need for HK and Taiwan data

Implemented changes

• Implemented get_hk_data() and get_taiwan_data() as internal functions to return case and death time series data from HK CHP and Taiwan CDC respectively
• Updated OWID source in data source lookup
• get_covid_df() now accepts a sources argument
  • "all": the default, returns JHU, WHO, Taiwan CDC, HK CHP
  • "WHO": WHO data as-is (china aggregated counts and no supplemental data)
  • "WHO+JHU": the "old" version containing WHO + JHU disaggregated data
  • "WHO+Primary": WHO data plus HK data from HK CHP, Taiwan data from Taiwan CDC, and Macau data from JHU thru Mar 10, 2023
• get_combined_table() was reworked internally to handle new data sources
• type argument for get_combined_table() was changed to accept a new type: "legacy", which returns what "Both" used to return previously (though the JHU data won't be updated and China Mainland data haven't been updated in some time)
• "Both" type for get_combined_table() now returns case and death data from get_covid_df("WHO+Primary")
• Updated docs accordingly

Possible future directions

• Data warehousing in AZDL to avoid pulling full time series from these sources each time
  • and to avoid eventuality where JHU data are removed entirely
• Work with OWID to incorporate Taiwan, HK, and Macau data into their system, as desired
• Possibly publish our own global time-series hosted on data.cdc.gov
  • Is there value in this above and beyond WHO dashboard / OWID offerings?

TODO

• Finish fixing unit tests

[beansrowning]

@jamesfuller-cdc FYI, we can chat when you're back in office Monday

[beansrowning]

Test failures

• Looks like one is due to existing issue with an update to name entry for Saint Helena