diff --git a/docs/source/user_guide/connector/query.ipynb b/docs/source/user_guide/connector/query.ipynb index a3b17da57..4df77cbf5 100644 --- a/docs/source/user_guide/connector/query.ipynb +++ b/docs/source/user_guide/connector/query.ipynb @@ -81,7 +81,15 @@ "```\n", "df = await dc.query(\"tweets\", _q=\"COVID-19\", _count=50)\n", "```\n", - "This query searches 50 results for tweets related to COVID-19 from Twitter search API." + "This query searches 50 results for tweets related to COVID-19 from Twitter search API.\n", + "\n", + "\n", + "* **Covid Tracking**\n", + "\n", + "```\n", + "df = await dc.query(\"state_historical\", state = \"mi\")\n", + "```\n", + "This query searches for Historical COVID-19 data in Michigan" ] } ], @@ -106,4 +114,4 @@ }, "nbformat": 4, "nbformat_minor": 4 -} +} \ No newline at end of file diff --git a/examples/DataConnector_CovidTracking.ipynb b/examples/DataConnector_CovidTracking.ipynb new file mode 100644 index 000000000..f39e5df8b --- /dev/null +++ b/examples/DataConnector_CovidTracking.ipynb @@ -0,0 +1,162 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Connector for Covid Tracking\n", + "\n", + "In this example, we will be going over how to use Connector with Covid Tarcking and understand how to use the Dynamic Url functionality" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Prerequisites\n", + "\n", + "Connector is a component in the DataPrep library that aims to simplify data access by providing a standard API set. The goal is to help users skip the complex API configuration. In this tutorial, we demonstrate how to use the connector component with Twitter.\n", + "\n", + "If you haven't installed DataPrep, run command `!pip install dataprep` or execute the following cell." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Run me if you'd like to install\n", + "!pip install dataprep" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Download and store the configuration files in DataPrep. \n", + "\n", + "The configuration files are used to configure the parameters and initial setup for the API. The available configuration files can be manually downloaded here: [Configuration Files](https://github.com/sfu-db/DataConnectorConfigs) or automatically downloaded at usage. \n", + "\n", + "Store the configuration file in the dataprep folder. " + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Initialize connector\n", + "\n", + "This establishes a connection with Covid Tracking Api and returns an object. Once you run the code you can use the built in functions available from connector." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from dataprep.connector import connect, info\n", + "\n", + "dc = connect('covidTracking', _concurrency = 10)\n", + "\n", + "dc" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Functionalities\n", + "\n", + "Connector has several functions you can perform to gain insight on the data downloaded from Covid Tarcking API." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Connector.query\n", + "The query method downloads the website data and displays it in a Dataframe. The parameters must meet the requirements as indicated in connector.info for the operation to run. \n", + "\n", + "When the data is received from the server, it will either be in a JSON or XML format. The connector reformats the data in pandas Dataframe for the convenience of downstream operations.\n", + "\n" + ] + }, + { + "source": [ + "### Dynamic Url Creation\n", + "The Url genration method allows the user to create a url in which url can be changed to allow on the spot modification. The parameters passed from the query must meet the requirements as indicated in url for the operation to run.\n", + "\n", + "For Example:\n", + "\n", + "We have the following url, here you can see that, we have the ability to modify the url with any state we want.\n", + "http://api.covidtracking.com/v2/states/{state}/current.json \n", + "\n", + "Lets suppose we want to retrieve data for New York. We would just inter-exchange the {state} variable with ny (Abbr. for New York)\n", + "http://api.covidtracking.com/v2/states/ny/current.json\n", + "\n", + "Now seeing this you may be asking where do we pass the data we want to replace. This can be achived from by pacing the data as a parameter in the query function.\n", + "\n", + "Example:\n", + "\n", + "await dc.query(\"states\",state = \"ny\")\n", + "\n", + "Now, let's try to get Current COVID-19 data for a state in US from Covid Data API." + ], + "cell_type": "markdown", + "metadata": {} + }, + { + "source": [], + "cell_type": "markdown", + "metadata": {} + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### Searching for Historical COVID-19 data in Michigan" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Getting covid data for Michigan\n", + "df = await dc.query(\"state_historical\",state = \"mi\")\n", + "df" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# That's all for now. \n", + "If you are interested in writing your own configuration file or modify an existing one, refer to the [Example folder]()." + ] + } + ], + "metadata": { + "kernelspec": { + "name": "python376jvsc74a57bd0c958774743908812a06d5249a6a57af991212914420a506af38b26bbd3d5952a", + "display_name": "Python 3.7.6 64-bit" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.6-final" + } + }, + "nbformat": 4, + "nbformat_minor": 4 +} \ No newline at end of file