Tutorials/03_FDAScraper.ipynb

{
 "cells": [
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "# Scrape Information from the FDA\n",
    "\n",
    "The openFDA contains information to obtain data from their JSON file (see tutorial [here](01_Fetch_Data.ipynb)). However, the JSON information is updated regularly. During testing, some information may have been removed or placed back at a later date. Additionally, at the time of writing this notebook, the JSON data contained drug data that only extended up to 2020. As a workaround, **drug_nme** includes additional methods to extract information from the FDA from several sources, which will be demoed below. "
   ]
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "### Initialize the FDAScrapper() class\n",
    "\n",
    "The FDASCrapper() class will need to be initialized. This class can take several parameters. \n",
    "- **url**: the url to a specific site. By default, it will point to the [NME Drug and New Biologic Approvals](https://www.fda.gov/drugs/nda-and-bla-approvals/new-molecular-entity-nme-drug-and-new-biologic-approvals) page. \n",
    "- **compilation_link**: A link to the page containing the [Compilation of CDER NME Drug and New Biologic Approvals](https://www.fda.gov/drugs/drug-approvals-and-databases/compilation-cder-new-molecular-entity-nme-drug-and-new-biologic-approvals). \n",
    "- **latest_link**: A link to the page containing the [Novel Drug Approvals at FDA](https://www.fda.gov/drugs/development-approval-process-drugs/novel-drug-approvals-fda).\n",
    "\n",
    "By default, these are already set to the specific web addresses (as linked above). However, if there are issues, users can try to use different sites and the url link will need to be given accordingly.  "
   ]
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-08-11T15:23:11.333610Z",
     "start_time": "2024-08-11T15:23:11.258842Z"
    }
   },
   "cell_type": "code",
   "source": [
    "import pandas as pd\n",
    "from tqdm import tqdm\n",
    "from drug_nme import FDAScraper\n",
    "scrape = FDAScraper()"
   ],
   "outputs": [],
   "execution_count": 1
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "### Get NMEs From PDF Reports\n",
    "\n",
    "The FDA curates reports of approval by [calendar year](https://www.fda.gov/drugs/nda-and-bla-approvals/new-molecular-entity-nme-drug-and-new-biologic-approvals) as PDF files. The **get.pdf_links()** method will try to extract links to the PDFs and extract the NME drug approval information. \n",
    "\n",
    "This will result in a dictionary containing the year and a link to the corresponding PDF report. This can be viewed by clicking or copy and pasting the link to the browser.  The links are also used to set the instance variable. Thus, it is not necessary to set the **get_pdf_links()** to a variable.\n",
    "\n",
    "**NOTE:** Currently, the FDA curates reports from 2015-2023. There are additional reports available for 1999-2014, however these links have been moved to an FDA Archive link. Extraction there can be difficult. A workaround will be created in the future.    "
   ]
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-08-11T15:23:11.392376Z",
     "start_time": "2024-08-11T15:23:11.334245Z"
    }
   },
   "cell_type": "code",
   "source": [
    "# set to variable to demo the output\n",
    "links = scrape.get_pdf_links()\n",
    "links"
   ],
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'2023': 'https://www.fda.gov/media/177083/download?attachment',\n",
       " '2022': 'https://www.fda.gov/media/165828/download?attachment',\n",
       " '2021': 'https://www.fda.gov/media/158152/download?attachment',\n",
       " '2020': 'https://www.fda.gov/media/147400/download?attachment',\n",
       " '2019': 'https://www.fda.gov/media/147414/download?attachment',\n",
       " '2018': 'https://www.fda.gov/media/124809/download?attachment',\n",
       " '2017': 'https://www.fda.gov/media/110746/download',\n",
       " '2016': 'https://www.fda.gov/media/102967/download',\n",
       " '2015': 'https://www.fda.gov/media/93424/download'}"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "execution_count": 2
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "### Extract Table From PDF Report\n",
    "\n",
    "The above method, **get_pdf_links()**, uses the instance variable set in the FDAScrapper class. The PDF will be accessed and NME drug information can be extracted using the **extract_table()** method. In this case, FDA CDER NME approvals will be mined from 2023. "
   ]
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-08-11T15:23:12.225633Z",
     "start_time": "2024-08-11T15:23:11.393723Z"
    }
   },
   "cell_type": "code",
   "source": [
    "year_2023 = scrape.extract_table(year='2023')\n",
    "year_2023.head()"
   ],
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0 APPLICATION NUMBER PROPRIETARY NAME ESTABLISHED NAME  \\\n",
       "1         NDA 214373        BRENZAVVY    BEXAGLIFLOZIN   \n",
       "2         NDA 216059         JAYPIRCA    PIRTOBRUTINIB   \n",
       "3         NDA 217639          ORSERDU      ELACESTRANT   \n",
       "4         NDA 216951        JESDUVROQ      DAPRODUSTAT   \n",
       "5         NDA 216403         FILSPARI       SPARSENTAN   \n",
       "\n",
       "0                                          APPLICANT REVIEW CLASSIFICATION  \\\n",
       "1                                    THERACOSBIO LLC                     S   \n",
       "2                                  LOXO ONCOLOGY INC                   P,O   \n",
       "3                          STEMLINE THERAPEUTICS INC                     P   \n",
       "4  GLAXOSMITHKLINE INTELLECTUAL PROPERTY NO 2 LTD...                     S   \n",
       "5                           TRAVERE THERAPEUTICS INC                   P,O   \n",
       "\n",
       "0 APPROVAL DATE                                         INDICATION  \n",
       "1    2023-01-20  ADJUNCT TO DIET AND EXERCISE TO IMPROVE GLYCEM...  \n",
       "2    2023-01-27  FOR THE TREATMENT OF ADULT PATIENTS WITH RELAP...  \n",
       "3    2023-01-27  FOR THE TREATMENT OF POSTMENOPAUSAL WOMEN OR A...  \n",
       "4    2023-02-01  FOR THE TREATMENT OF ANEMIA DUE TO CHRONIC KID...  \n",
       "5    2023-02-17  TO REDUCE PROTEINURIA IN ADULTS WITH PRIMARY I...  "
      ],
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>APPLICATION NUMBER</th>\n",
       "      <th>PROPRIETARY NAME</th>\n",
       "      <th>ESTABLISHED NAME</th>\n",
       "      <th>APPLICANT</th>\n",
       "      <th>REVIEW CLASSIFICATION</th>\n",
       "      <th>APPROVAL DATE</th>\n",
       "      <th>INDICATION</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>NDA 214373</td>\n",
       "      <td>BRENZAVVY</td>\n",
       "      <td>BEXAGLIFLOZIN</td>\n",
       "      <td>THERACOSBIO LLC</td>\n",
       "      <td>S</td>\n",
       "      <td>2023-01-20</td>\n",
       "      <td>ADJUNCT TO DIET AND EXERCISE TO IMPROVE GLYCEM...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>NDA 216059</td>\n",
       "      <td>JAYPIRCA</td>\n",
       "      <td>PIRTOBRUTINIB</td>\n",
       "      <td>LOXO ONCOLOGY INC</td>\n",
       "      <td>P,O</td>\n",
       "      <td>2023-01-27</td>\n",
       "      <td>FOR THE TREATMENT OF ADULT PATIENTS WITH RELAP...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>NDA 217639</td>\n",
       "      <td>ORSERDU</td>\n",
       "      <td>ELACESTRANT</td>\n",
       "      <td>STEMLINE THERAPEUTICS INC</td>\n",
       "      <td>P</td>\n",
       "      <td>2023-01-27</td>\n",
       "      <td>FOR THE TREATMENT OF POSTMENOPAUSAL WOMEN OR A...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>NDA 216951</td>\n",
       "      <td>JESDUVROQ</td>\n",
       "      <td>DAPRODUSTAT</td>\n",
       "      <td>GLAXOSMITHKLINE INTELLECTUAL PROPERTY NO 2 LTD...</td>\n",
       "      <td>S</td>\n",
       "      <td>2023-02-01</td>\n",
       "      <td>FOR THE TREATMENT OF ANEMIA DUE TO CHRONIC KID...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>NDA 216403</td>\n",
       "      <td>FILSPARI</td>\n",
       "      <td>SPARSENTAN</td>\n",
       "      <td>TRAVERE THERAPEUTICS INC</td>\n",
       "      <td>P,O</td>\n",
       "      <td>2023-02-17</td>\n",
       "      <td>TO REDUCE PROTEINURIA IN ADULTS WITH PRIMARY I...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "execution_count": 3
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "### Extract Table From Multiple Years\n",
    "The above example is great for a single year. Multiple years can be obtained using a loop. The link dictionary demoed above will be looped and the table for each key/item pair will be extracted. Then the tables will be combined to generate a single pd.DataFrame for NME across the indicated years. "
   ]
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-08-11T15:23:13.911636Z",
     "start_time": "2024-08-11T15:23:12.226518Z"
    }
   },
   "cell_type": "code",
   "source": [
    "all_df = scrape.extract_table(year='all')\n",
    "all_df"
   ],
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Extracting tables: 100%|██████████| 9/9 [00:01<00:00,  5.42it/s]\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "    APPLICATION NUMBER PROPRIETARY NAME ESTABLISHED NAME  \\\n",
       "0           NDA 214373        BRENZAVVY    BEXAGLIFLOZIN   \n",
       "1           NDA 216059         JAYPIRCA    PIRTOBRUTINIB   \n",
       "2           NDA 217639          ORSERDU      ELACESTRANT   \n",
       "3           NDA 216951        JESDUVROQ      DAPRODUSTAT   \n",
       "4           NDA 216403         FILSPARI       SPARSENTAN   \n",
       "..                 ...              ...              ...   \n",
       "300     BLA 125526/0.0           NUCALA      MEPOLIZUMAB   \n",
       "301     BLA 761036/0.0         DARZALEX      DARATUMUMAB   \n",
       "302     BLA 125547/0.0        PORTRAZZA      NECITUMUMAB   \n",
       "303     BLA 761035/0.0        EMPLICITI       ELOTUZUMAB   \n",
       "304     BLA 125561/0.0           KANUMA  SEBELIPASE ALFA   \n",
       "\n",
       "                                             APPLICANT REVIEW CLASSIFICATION  \\\n",
       "0                                      THERACOSBIO LLC                     S   \n",
       "1                                    LOXO ONCOLOGY INC                   P,O   \n",
       "2                            STEMLINE THERAPEUTICS INC                     P   \n",
       "3    GLAXOSMITHKLINE INTELLECTUAL PROPERTY NO 2 LTD...                     S   \n",
       "4                             TRAVERE THERAPEUTICS INC                   P,O   \n",
       "..                                                 ...                   ...   \n",
       "300                                GLAXOSMITHKLINE LLC                     S   \n",
       "301                              JANSSEN BIOTECH, INC.                   P,O   \n",
       "302                              ELI LILLY AND COMPANY                     S   \n",
       "303                       BRISTOL-MYERS SQUIBB COMPANY                   P,O   \n",
       "304                            SYNAGEVA BIOPHARMA CORP                   P,O   \n",
       "\n",
       "    APPROVAL DATE                                         INDICATION  \\\n",
       "0      2023-01-20  ADJUNCT TO DIET AND EXERCISE TO IMPROVE GLYCEM...   \n",
       "1      2023-01-27  FOR THE TREATMENT OF ADULT PATIENTS WITH RELAP...   \n",
       "2      2023-01-27  FOR THE TREATMENT OF POSTMENOPAUSAL WOMEN OR A...   \n",
       "3      2023-02-01  FOR THE TREATMENT OF ANEMIA DUE TO CHRONIC KID...   \n",
       "4      2023-02-17  TO REDUCE PROTEINURIA IN ADULTS WITH PRIMARY I...   \n",
       "..            ...                                                ...   \n",
       "300    2015-11-04  INDICATED FOR THE ADD-ON MAINTENANCE TREATMENT...   \n",
       "301    2015-11-16  INDICATED FOR TREATMENT OF PATIENTS WITH MULTI...   \n",
       "302    2015-11-24  IS INDICATED, IN COMBINATION WITH GEMCITABINE ...   \n",
       "303    2015-11-30  INDICATED IN COMBINATION WITH LENALIDOMIDE AND...   \n",
       "304    2015-12-08  INDICATED FOR THE TREATMENT OF PATIENTS WITH A...   \n",
       "\n",
       "    CY 2020 CDER New Molecular Entity (NME) Drug & Original BLA Calendar Year Approvals\\rAs of Decemeber 31, 2020\\rThis report reflects the data shown as it is identified in the database.\\rSelection Criteria:\\rUser Response: Start Date: 1/1/2020    End Date: 12/31/2020\\rSort Order: Approval Date  \\\n",
       "0                                                  NaN                                                                                                                                                                                                                                                     \n",
       "1                                                  NaN                                                                                                                                                                                                                                                     \n",
       "2                                                  NaN                                                                                                                                                                                                                                                     \n",
       "3                                                  NaN                                                                                                                                                                                                                                                     \n",
       "4                                                  NaN                                                                                                                                                                                                                                                     \n",
       "..                                                 ...                                                                                                                                                                                                                                                     \n",
       "300                                                NaN                                                                                                                                                                                                                                                     \n",
       "301                                                NaN                                                                                                                                                                                                                                                     \n",
       "302                                                NaN                                                                                                                                                                                                                                                     \n",
       "303                                                NaN                                                                                                                                                                                                                                                     \n",
       "304                                                NaN                                                                                                                                                                                                                                                     \n",
       "\n",
       "    Unnamed: 1_level_0 Unnamed: 2_level_0 Unnamed: 3_level_0  \\\n",
       "0                  NaN                NaN                NaN   \n",
       "1                  NaN                NaN                NaN   \n",
       "2                  NaN                NaN                NaN   \n",
       "3                  NaN                NaN                NaN   \n",
       "4                  NaN                NaN                NaN   \n",
       "..                 ...                ...                ...   \n",
       "300                NaN                NaN                NaN   \n",
       "301                NaN                NaN                NaN   \n",
       "302                NaN                NaN                NaN   \n",
       "303                NaN                NaN                NaN   \n",
       "304                NaN                NaN                NaN   \n",
       "\n",
       "    Unnamed: 4_level_0 Unnamed: 5_level_0 Unnamed: 6_level_0  \\\n",
       "0                  NaN                NaN                NaN   \n",
       "1                  NaN                NaN                NaN   \n",
       "2                  NaN                NaN                NaN   \n",
       "3                  NaN                NaN                NaN   \n",
       "4                  NaN                NaN                NaN   \n",
       "..                 ...                ...                ...   \n",
       "300                NaN                NaN                NaN   \n",
       "301                NaN                NaN                NaN   \n",
       "302                NaN                NaN                NaN   \n",
       "303                NaN                NaN                NaN   \n",
       "304                NaN                NaN                NaN   \n",
       "\n",
       "    Unnamed: 7_level_0  \\\n",
       "0                  NaN   \n",
       "1                  NaN   \n",
       "2                  NaN   \n",
       "3                  NaN   \n",
       "4                  NaN   \n",
       "..                 ...   \n",
       "300                NaN   \n",
       "301                NaN   \n",
       "302                NaN   \n",
       "303                NaN   \n",
       "304                NaN   \n",
       "\n",
       "    CY 2018 CDER New Molecular Entity (NME) Drug & Original BLA Calendar Year Approvals\\rAs of December 31, 2018\\rThis report reflects the data shown as it is identified in the database.\\rSelection Criteria:\\rUser Response: Start Date: 1/1/2018    End Date: 12/31/2018\\rSort Order: Approval Date  \n",
       "0                                                  NaN                                                                                                                                                                                                                                                   \n",
       "1                                                  NaN                                                                                                                                                                                                                                                   \n",
       "2                                                  NaN                                                                                                                                                                                                                                                   \n",
       "3                                                  NaN                                                                                                                                                                                                                                                   \n",
       "4                                                  NaN                                                                                                                                                                                                                                                   \n",
       "..                                                 ...                                                                                                                                                                                                                                                   \n",
       "300                                                NaN                                                                                                                                                                                                                                                   \n",
       "301                                                NaN                                                                                                                                                                                                                                                   \n",
       "302                                                NaN                                                                                                                                                                                                                                                   \n",
       "303                                                NaN                                                                                                                                                                                                                                                   \n",
       "304                                                NaN                                                                                                                                                                                                                                                   \n",
       "\n",
       "[305 rows x 16 columns]"
      ],
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>APPLICATION NUMBER</th>\n",
       "      <th>PROPRIETARY NAME</th>\n",
       "      <th>ESTABLISHED NAME</th>\n",
       "      <th>APPLICANT</th>\n",
       "      <th>REVIEW CLASSIFICATION</th>\n",
       "      <th>APPROVAL DATE</th>\n",
       "      <th>INDICATION</th>\n",
       "      <th>CY 2020 CDER New Molecular Entity (NME) Drug &amp; Original BLA Calendar Year Approvals\\rAs of Decemeber 31, 2020\\rThis report reflects the data shown as it is identified in the database.\\rSelection Criteria:\\rUser Response: Start Date: 1/1/2020    End Date: 12/31/2020\\rSort Order: Approval Date</th>\n",
       "      <th>Unnamed: 1_level_0</th>\n",
       "      <th>Unnamed: 2_level_0</th>\n",
       "      <th>Unnamed: 3_level_0</th>\n",
       "      <th>Unnamed: 4_level_0</th>\n",
       "      <th>Unnamed: 5_level_0</th>\n",
       "      <th>Unnamed: 6_level_0</th>\n",
       "      <th>Unnamed: 7_level_0</th>\n",
       "      <th>CY 2018 CDER New Molecular Entity (NME) Drug &amp; Original BLA Calendar Year Approvals\\rAs of December 31, 2018\\rThis report reflects the data shown as it is identified in the database.\\rSelection Criteria:\\rUser Response: Start Date: 1/1/2018    End Date: 12/31/2018\\rSort Order: Approval Date</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NDA 214373</td>\n",
       "      <td>BRENZAVVY</td>\n",
       "      <td>BEXAGLIFLOZIN</td>\n",
       "      <td>THERACOSBIO LLC</td>\n",
       "      <td>S</td>\n",
       "      <td>2023-01-20</td>\n",
       "      <td>ADJUNCT TO DIET AND EXERCISE TO IMPROVE GLYCEM...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>NDA 216059</td>\n",
       "      <td>JAYPIRCA</td>\n",
       "      <td>PIRTOBRUTINIB</td>\n",
       "      <td>LOXO ONCOLOGY INC</td>\n",
       "      <td>P,O</td>\n",
       "      <td>2023-01-27</td>\n",
       "      <td>FOR THE TREATMENT OF ADULT PATIENTS WITH RELAP...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>NDA 217639</td>\n",
       "      <td>ORSERDU</td>\n",
       "      <td>ELACESTRANT</td>\n",
       "      <td>STEMLINE THERAPEUTICS INC</td>\n",
       "      <td>P</td>\n",
       "      <td>2023-01-27</td>\n",
       "      <td>FOR THE TREATMENT OF POSTMENOPAUSAL WOMEN OR A...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>NDA 216951</td>\n",
       "      <td>JESDUVROQ</td>\n",
       "      <td>DAPRODUSTAT</td>\n",
       "      <td>GLAXOSMITHKLINE INTELLECTUAL PROPERTY NO 2 LTD...</td>\n",
       "      <td>S</td>\n",
       "      <td>2023-02-01</td>\n",
       "      <td>FOR THE TREATMENT OF ANEMIA DUE TO CHRONIC KID...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>NDA 216403</td>\n",
       "      <td>FILSPARI</td>\n",
       "      <td>SPARSENTAN</td>\n",
       "      <td>TRAVERE THERAPEUTICS INC</td>\n",
       "      <td>P,O</td>\n",
       "      <td>2023-02-17</td>\n",
       "      <td>TO REDUCE PROTEINURIA IN ADULTS WITH PRIMARY I...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>300</th>\n",
       "      <td>BLA 125526/0.0</td>\n",
       "      <td>NUCALA</td>\n",
       "      <td>MEPOLIZUMAB</td>\n",
       "      <td>GLAXOSMITHKLINE LLC</td>\n",
       "      <td>S</td>\n",
       "      <td>2015-11-04</td>\n",
       "      <td>INDICATED FOR THE ADD-ON MAINTENANCE TREATMENT...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>301</th>\n",
       "      <td>BLA 761036/0.0</td>\n",
       "      <td>DARZALEX</td>\n",
       "      <td>DARATUMUMAB</td>\n",
       "      <td>JANSSEN BIOTECH, INC.</td>\n",
       "      <td>P,O</td>\n",
       "      <td>2015-11-16</td>\n",
       "      <td>INDICATED FOR TREATMENT OF PATIENTS WITH MULTI...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>302</th>\n",
       "      <td>BLA 125547/0.0</td>\n",
       "      <td>PORTRAZZA</td>\n",
       "      <td>NECITUMUMAB</td>\n",
       "      <td>ELI LILLY AND COMPANY</td>\n",
       "      <td>S</td>\n",
       "      <td>2015-11-24</td>\n",
       "      <td>IS INDICATED, IN COMBINATION WITH GEMCITABINE ...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>303</th>\n",
       "      <td>BLA 761035/0.0</td>\n",
       "      <td>EMPLICITI</td>\n",
       "      <td>ELOTUZUMAB</td>\n",
       "      <td>BRISTOL-MYERS SQUIBB COMPANY</td>\n",
       "      <td>P,O</td>\n",
       "      <td>2015-11-30</td>\n",
       "      <td>INDICATED IN COMBINATION WITH LENALIDOMIDE AND...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>304</th>\n",
       "      <td>BLA 125561/0.0</td>\n",
       "      <td>KANUMA</td>\n",
       "      <td>SEBELIPASE ALFA</td>\n",
       "      <td>SYNAGEVA BIOPHARMA CORP</td>\n",
       "      <td>P,O</td>\n",
       "      <td>2015-12-08</td>\n",
       "      <td>INDICATED FOR THE TREATMENT OF PATIENTS WITH A...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>305 rows × 16 columns</p>\n",
       "</div>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "execution_count": 4
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "# TODO function to clean up the compilation table"
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "### Get Compilation Data\n",
    "\n",
    "The FDA contains a site containing [Compilation Dataset links](https://www.fda.gov/drugs/drug-approvals-and-databases/compilation-cder-new-molecular-entity-nme-drug-and-new-biologic-approvals). This link points to a .csv file that can be downloaded and converted into a pd.DataFrame. This is already handled by FDAScrapper.\n",
    "\n",
    "If there are updates to the Compilation Dataset page, a different URL can be tested by passing in the url to the \"url\" parameter.  \n",
    "\n",
    "As of this writing, the compilation dataset contains **NME and New Biologic Approvals from 1985-2023**"
   ]
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-08-11T15:23:14.053345Z",
     "start_time": "2024-08-11T15:23:13.912569Z"
    }
   },
   "cell_type": "code",
   "source": [
    "compilation = scrape.get_compilation()\n",
    "compilation"
   ],
   "outputs": [
    {
     "data": {
      "text/plain": [
       "     Proprietary  Name     Active Ingredient/Moiety  \\\n",
       "0               Lupron           leuprolide acetate   \n",
       "1              Seldane                  terfenadine   \n",
       "2              Ridaura                    auranofin   \n",
       "3              Marinol                   dronabinol   \n",
       "4               Fortaz                  ceftazidime   \n",
       "...                ...                          ...   \n",
       "1286          Ryzneuta  efbemalenograstim alfa-vuxw   \n",
       "1287           Ogsiveo                 nirogacestat   \n",
       "1288          Fabhalta                    iptacopan   \n",
       "1289          Filsuvez            birch triterpenes   \n",
       "1290            Wainua                  eplontersen   \n",
       "\n",
       "                                    Applicant NDA/BLA   Application Number(1)  \\\n",
       "0                         TAP Pharmaceuticals     NDA                   19010   \n",
       "1                                 Merrell-Dow     NDA                   18949   \n",
       "2                         SmithKline & French     NDA                   18689   \n",
       "3                                      Unimed     NDA                   18651   \n",
       "4                                       Glaxo     NDA                   50578   \n",
       "...                                       ...     ...                     ...   \n",
       "1286  Evive Biotechnology Singapore PTE. Ltd.     BLA                  761134   \n",
       "1287           SpringWorks Therapeutics, Inc.     NDA                  217677   \n",
       "1288     Novartis Pharmaceuticals Corporation     NDA                  218276   \n",
       "1289                Amryt Pharmaceuticals DAC     NDA                  215064   \n",
       "1290              Ionis Pharmaceuticals, Inc.     NDA                  217388   \n",
       "\n",
       "       Application Number(2)   Application Number(3) Dosage Form(1)  \\\n",
       "0                        NaN                     NaN     Injectable   \n",
       "1                        NaN                     NaN         Tablet   \n",
       "2                        NaN                     NaN        Capsule   \n",
       "3                        NaN                     NaN        Capsule   \n",
       "4                        NaN                     NaN     Injectable   \n",
       "...                      ...                     ...            ...   \n",
       "1286                     NaN                     NaN     Injectable   \n",
       "1287                     NaN                     NaN         Tablet   \n",
       "1288                     NaN                     NaN        Capsule   \n",
       "1289                     NaN                     NaN            Gel   \n",
       "1290                     NaN                     NaN     Injectable   \n",
       "\n",
       "     Route of Administration(1) Dosage Form(2)  ...  \\\n",
       "0                     Injection            NaN  ...   \n",
       "1                          Oral            NaN  ...   \n",
       "2                          Oral            NaN  ...   \n",
       "3                          Oral            NaN  ...   \n",
       "4                     Injection            NaN  ...   \n",
       "...                         ...            ...  ...   \n",
       "1286                  Injection            NaN  ...   \n",
       "1287                       Oral            NaN  ...   \n",
       "1288                       Oral            NaN  ...   \n",
       "1289                    Topical            NaN  ...   \n",
       "1290                  Injection            NaN  ...   \n",
       "\n",
       "                                        Approved Use(s)  \\\n",
       "0                                                   NaN   \n",
       "1                                                   NaN   \n",
       "2                                                   NaN   \n",
       "3                                                   NaN   \n",
       "4                                                   NaN   \n",
       "...                                                 ...   \n",
       "1286  RYZNEUTA is indicated to decrease the incidenc...   \n",
       "1287  OGSIVEO is indicated for adult patients with p...   \n",
       "1288  FABHALTA is indicated for the treatment of adu...   \n",
       "1289  FILSUVEZ topical gel is indicated for the trea...   \n",
       "1290  WAINUA is indicated for the treatment of the p...   \n",
       "\n",
       "                           Review Designation Orphan Drug Designation  \\\n",
       "0                                    Priority                      No   \n",
       "1                                    Priority                      No   \n",
       "2                                    Priority                      No   \n",
       "3                                    Priority                      No   \n",
       "4                                    Priority                      No   \n",
       "...                                       ...                     ...   \n",
       "1286                                 Standard                      No   \n",
       "1287                                 Priority                     Yes   \n",
       "1288  Priority (used priority review voucher)                     Yes   \n",
       "1289                                 Priority                     Yes   \n",
       "1290                                 Standard                     Yes   \n",
       "\n",
       "     Accelerated Approval Breakthrough Therapy Designation  \\\n",
       "0                     NaN                              NaN   \n",
       "1                     NaN                              NaN   \n",
       "2                     NaN                              NaN   \n",
       "3                     NaN                              NaN   \n",
       "4                     NaN                              NaN   \n",
       "...                   ...                              ...   \n",
       "1286                   No                               No   \n",
       "1287                   No                              Yes   \n",
       "1288                   No                              Yes   \n",
       "1289                   No                               No   \n",
       "1290                   No                               No   \n",
       "\n",
       "      Fast Track Designation Qualified Infectious Disease Product  \\\n",
       "0                        NaN                                  NaN   \n",
       "1                        NaN                                  NaN   \n",
       "2                        NaN                                  NaN   \n",
       "3                        NaN                                  NaN   \n",
       "4                        NaN                                  NaN   \n",
       "...                      ...                                  ...   \n",
       "1286                      No                                  NaN   \n",
       "1287                     Yes                                   No   \n",
       "1288                      No                                   No   \n",
       "1289                     Yes                                   No   \n",
       "1290                      No                                   No   \n",
       "\n",
       "     Issued a Priority Review Voucher Redeemed a Priority Review Voucher Notes  \n",
       "0                                 NaN                                NaN   NaN  \n",
       "1                                 NaN                                NaN   NaN  \n",
       "2                                 NaN                                NaN   NaN  \n",
       "3                                 NaN                                NaN   NaN  \n",
       "4                                 NaN                                NaN   NaN  \n",
       "...                               ...                                ...   ...  \n",
       "1286                               No                                 No   NaN  \n",
       "1287                               No                                 No   NaN  \n",
       "1288                               No                                Yes   NaN  \n",
       "1289                              RPD                                 No   NaN  \n",
       "1290                               No                                 No   NaN  \n",
       "\n",
       "[1291 rows x 27 columns]"
      ],
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Proprietary  Name</th>\n",
       "      <th>Active Ingredient/Moiety</th>\n",
       "      <th>Applicant</th>\n",
       "      <th>NDA/BLA</th>\n",
       "      <th>Application Number(1)</th>\n",
       "      <th>Application Number(2)</th>\n",
       "      <th>Application Number(3)</th>\n",
       "      <th>Dosage Form(1)</th>\n",
       "      <th>Route of Administration(1)</th>\n",
       "      <th>Dosage Form(2)</th>\n",
       "      <th>...</th>\n",
       "      <th>Approved Use(s)</th>\n",
       "      <th>Review Designation</th>\n",
       "      <th>Orphan Drug Designation</th>\n",
       "      <th>Accelerated Approval</th>\n",
       "      <th>Breakthrough Therapy Designation</th>\n",
       "      <th>Fast Track Designation</th>\n",
       "      <th>Qualified Infectious Disease Product</th>\n",
       "      <th>Issued a Priority Review Voucher</th>\n",
       "      <th>Redeemed a Priority Review Voucher</th>\n",
       "      <th>Notes</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Lupron</td>\n",
       "      <td>leuprolide acetate</td>\n",
       "      <td>TAP Pharmaceuticals</td>\n",
       "      <td>NDA</td>\n",
       "      <td>19010</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Injectable</td>\n",
       "      <td>Injection</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Priority</td>\n",
       "      <td>No</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Seldane</td>\n",
       "      <td>terfenadine</td>\n",
       "      <td>Merrell-Dow</td>\n",
       "      <td>NDA</td>\n",
       "      <td>18949</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Tablet</td>\n",
       "      <td>Oral</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Priority</td>\n",
       "      <td>No</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Ridaura</td>\n",
       "      <td>auranofin</td>\n",
       "      <td>SmithKline &amp; French</td>\n",
       "      <td>NDA</td>\n",
       "      <td>18689</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Capsule</td>\n",
       "      <td>Oral</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Priority</td>\n",
       "      <td>No</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Marinol</td>\n",
       "      <td>dronabinol</td>\n",
       "      <td>Unimed</td>\n",
       "      <td>NDA</td>\n",
       "      <td>18651</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Capsule</td>\n",
       "      <td>Oral</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Priority</td>\n",
       "      <td>No</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Fortaz</td>\n",
       "      <td>ceftazidime</td>\n",
       "      <td>Glaxo</td>\n",
       "      <td>NDA</td>\n",
       "      <td>50578</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Injectable</td>\n",
       "      <td>Injection</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Priority</td>\n",
       "      <td>No</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1286</th>\n",
       "      <td>Ryzneuta</td>\n",
       "      <td>efbemalenograstim alfa-vuxw</td>\n",
       "      <td>Evive Biotechnology Singapore PTE. Ltd.</td>\n",
       "      <td>BLA</td>\n",
       "      <td>761134</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Injectable</td>\n",
       "      <td>Injection</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>RYZNEUTA is indicated to decrease the incidenc...</td>\n",
       "      <td>Standard</td>\n",
       "      <td>No</td>\n",
       "      <td>No</td>\n",
       "      <td>No</td>\n",
       "      <td>No</td>\n",
       "      <td>NaN</td>\n",
       "      <td>No</td>\n",
       "      <td>No</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1287</th>\n",
       "      <td>Ogsiveo</td>\n",
       "      <td>nirogacestat</td>\n",
       "      <td>SpringWorks Therapeutics, Inc.</td>\n",
       "      <td>NDA</td>\n",
       "      <td>217677</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Tablet</td>\n",
       "      <td>Oral</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>OGSIVEO is indicated for adult patients with p...</td>\n",
       "      <td>Priority</td>\n",
       "      <td>Yes</td>\n",
       "      <td>No</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Yes</td>\n",
       "      <td>No</td>\n",
       "      <td>No</td>\n",
       "      <td>No</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1288</th>\n",
       "      <td>Fabhalta</td>\n",
       "      <td>iptacopan</td>\n",
       "      <td>Novartis Pharmaceuticals Corporation</td>\n",
       "      <td>NDA</td>\n",
       "      <td>218276</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Capsule</td>\n",
       "      <td>Oral</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>FABHALTA is indicated for the treatment of adu...</td>\n",
       "      <td>Priority (used priority review voucher)</td>\n",
       "      <td>Yes</td>\n",
       "      <td>No</td>\n",
       "      <td>Yes</td>\n",
       "      <td>No</td>\n",
       "      <td>No</td>\n",
       "      <td>No</td>\n",
       "      <td>Yes</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1289</th>\n",
       "      <td>Filsuvez</td>\n",
       "      <td>birch triterpenes</td>\n",
       "      <td>Amryt Pharmaceuticals DAC</td>\n",
       "      <td>NDA</td>\n",
       "      <td>215064</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Gel</td>\n",
       "      <td>Topical</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>FILSUVEZ topical gel is indicated for the trea...</td>\n",
       "      <td>Priority</td>\n",
       "      <td>Yes</td>\n",
       "      <td>No</td>\n",
       "      <td>No</td>\n",
       "      <td>Yes</td>\n",
       "      <td>No</td>\n",
       "      <td>RPD</td>\n",
       "      <td>No</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1290</th>\n",
       "      <td>Wainua</td>\n",
       "      <td>eplontersen</td>\n",
       "      <td>Ionis Pharmaceuticals, Inc.</td>\n",
       "      <td>NDA</td>\n",
       "      <td>217388</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Injectable</td>\n",
       "      <td>Injection</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>WAINUA is indicated for the treatment of the p...</td>\n",
       "      <td>Standard</td>\n",
       "      <td>Yes</td>\n",
       "      <td>No</td>\n",
       "      <td>No</td>\n",
       "      <td>No</td>\n",
       "      <td>No</td>\n",
       "      <td>No</td>\n",
       "      <td>No</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1291 rows × 27 columns</p>\n",
       "</div>"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "execution_count": 5
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "# Get Current Approvals for Current Year\n",
    "\n",
    "Finally, the FDA curates [Novel Drug Approvals](https://www.fda.gov/drugs/development-approval-process-drugs/novel-drug-approvals-fda) for a given year. Information from this page can be used to obtain the current year drug approvals.\n",
    "\n",
    "A downside to this page is that it does not properly designate a drug as New Drug Application (NDA) or Biologics License Application (BLA). This would indicate small-molecule or biologics, respectively. A rule of thumb can be taken by the active ingredient names, where drugs ending in \"mab\" and \"cept\" can indicate biologics. However, confirmation should still be performed to ensure labeling accuracy.   "
   ]
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-08-11T15:23:14.142082Z",
     "start_time": "2024-08-11T15:23:14.054103Z"
    }
   },
   "cell_type": "code",
   "source": [
    "latest = scrape.get_current_year()\n",
    "latest"
   ],
   "outputs": [
    {
     "data": {
      "text/plain": [
       "    Drug Name                  Active Ingredient Approval Date  \\\n",
       "0    Leqselvi                     deuruxolitinib     7/25/2024   \n",
       "1     Kisunla                     donanemab-azbt      7/2/2024   \n",
       "2   Ohtuvayre                       ensifentrine     6/26/2024   \n",
       "3      Piasky                    crovalimab-akkz     6/20/2024   \n",
       "4      Sofdra                        sofpironium     6/18/2024   \n",
       "5      Iqirvo                        elafibranor     6/10/2024   \n",
       "6      Rytelo                         imetelstat      6/6/2024   \n",
       "7   Imdelltra                    tarlatamab-dlle     5/16/2024   \n",
       "8    Xolremdi                        mavorixafor     4/26/2024   \n",
       "9      Ojemda                        tovorafenib     4/23/2024   \n",
       "10    Anktiva  nogapendekin alfa inbakicept-pmln     4/22/2024   \n",
       "11  Lumisight                      pegulicianine     4/17/2024   \n",
       "12    Zevtera      ceftobiprole medocaril sodium      4/3/2024   \n",
       "13    Voydeya                          danicopan     3/29/2024   \n",
       "14     Vafseo                         vadadustat     3/27/2024   \n",
       "15  Winrevair                   sotatercept-csrk     3/26/2024   \n",
       "16    Duvyzat                         givinostat     3/21/2024   \n",
       "17     Tryvio                       aprocitentan     3/19/2024   \n",
       "18  Rezdiffra                         resmetirom     3/14/2024   \n",
       "19   Tevimbra                  tislelizumab-jsgr     3/13/2024   \n",
       "20     Letybo           letibotulinumtoxinA-wlbg     2/29/2024   \n",
       "21   Exblifep           cefepime, enmetazobactam     2/22/2024   \n",
       "22   Zelsuvmi                         berdazimer      1/5/2024   \n",
       "\n",
       "                   FDA-approved use on approval date*  \n",
       "0                     To treat severe alopecia areata  \n",
       "1                        To treat Alzheimer’s disease  \n",
       "2      To treat chronic obstructive pulmonary disease  \n",
       "3        To treat paroxysmal nocturnal hemoglobinuria  \n",
       "4             To treat primary axillary hyperhidrosis  \n",
       "5   To treat primary biliary cholangitis in combin...  \n",
       "6   To treat low- to intermediate-1 risk myelodysp...  \n",
       "7     To treat extensive stage small cell lung cancer  \n",
       "8   To treat WHIM syndrome (warts, hypogammaglobul...  \n",
       "9   To treat relapsed or refractory pediatric low-...  \n",
       "10                            To treat bladder cancer  \n",
       "11  To use as an optical imaging agent for the det...  \n",
       "12  To treat certain bloodstream infections, bacte...  \n",
       "13  To treat extravascular hemolysis with paroxysm...  \n",
       "14      To treat anemia due to chronic kidney disease  \n",
       "15  To treat pulmonary arterial hypertension\\nDrug...  \n",
       "16  To treat Duchenne muscular dystrophy in indivi...  \n",
       "17        To treat hypertension\\nDrug Trials Snapshot  \n",
       "18  To treat noncirrhotic non-alcoholic steatohepa...  \n",
       "19  To treat unresectable or metastatic esophageal...  \n",
       "20  To temporarily improve the appearance of moder...  \n",
       "21  To treat complicated urinary tract infections\\...  \n",
       "22  To treat molluscum contagiosum\\nDrug Trials Sn...  "
      ],
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Drug Name</th>\n",
       "      <th>Active Ingredient</th>\n",
       "      <th>Approval Date</th>\n",
       "      <th>FDA-approved use on approval date*</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Leqselvi</td>\n",
       "      <td>deuruxolitinib</td>\n",
       "      <td>7/25/2024</td>\n",
       "      <td>To treat severe alopecia areata</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Kisunla</td>\n",
       "      <td>donanemab-azbt</td>\n",
       "      <td>7/2/2024</td>\n",
       "      <td>To treat Alzheimer’s disease</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Ohtuvayre</td>\n",
       "      <td>ensifentrine</td>\n",
       "      <td>6/26/2024</td>\n",
       "      <td>To treat chronic obstructive pulmonary disease</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Piasky</td>\n",
       "      <td>crovalimab-akkz</td>\n",
       "      <td>6/20/2024</td>\n",
       "      <td>To treat paroxysmal nocturnal hemoglobinuria</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Sofdra</td>\n",
       "      <td>sofpironium</td>\n",
       "      <td>6/18/2024</td>\n",
       "      <td>To treat primary axillary hyperhidrosis</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Iqirvo</td>\n",
       "      <td>elafibranor</td>\n",
       "      <td>6/10/2024</td>\n",
       "      <td>To treat primary biliary cholangitis in combin...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Rytelo</td>\n",
       "      <td>imetelstat</td>\n",
       "      <td>6/6/2024</td>\n",
       "      <td>To treat low- to intermediate-1 risk myelodysp...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Imdelltra</td>\n",
       "      <td>tarlatamab-dlle</td>\n",
       "      <td>5/16/2024</td>\n",
       "      <td>To treat extensive stage small cell lung cancer</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Xolremdi</td>\n",
       "      <td>mavorixafor</td>\n",
       "      <td>4/26/2024</td>\n",
       "      <td>To treat WHIM syndrome (warts, hypogammaglobul...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Ojemda</td>\n",
       "      <td>tovorafenib</td>\n",
       "      <td>4/23/2024</td>\n",
       "      <td>To treat relapsed or refractory pediatric low-...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>Anktiva</td>\n",
       "      <td>nogapendekin alfa inbakicept-pmln</td>\n",
       "      <td>4/22/2024</td>\n",
       "      <td>To treat bladder cancer</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>Lumisight</td>\n",
       "      <td>pegulicianine</td>\n",
       "      <td>4/17/2024</td>\n",
       "      <td>To use as an optical imaging agent for the det...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>Zevtera</td>\n",
       "      <td>ceftobiprole medocaril sodium</td>\n",
       "      <td>4/3/2024</td>\n",
       "      <td>To treat certain bloodstream infections, bacte...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>Voydeya</td>\n",
       "      <td>danicopan</td>\n",
       "      <td>3/29/2024</td>\n",
       "      <td>To treat extravascular hemolysis with paroxysm...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>Vafseo</td>\n",
       "      <td>vadadustat</td>\n",
       "      <td>3/27/2024</td>\n",
       "      <td>To treat anemia due to chronic kidney disease</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>Winrevair</td>\n",
       "      <td>sotatercept-csrk</td>\n",
       "      <td>3/26/2024</td>\n",
       "      <td>To treat pulmonary arterial hypertension\\nDrug...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>Duvyzat</td>\n",
       "      <td>givinostat</td>\n",
       "      <td>3/21/2024</td>\n",
       "      <td>To treat Duchenne muscular dystrophy in indivi...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>Tryvio</td>\n",
       "      <td>aprocitentan</td>\n",
       "      <td>3/19/2024</td>\n",
       "      <td>To treat hypertension\\nDrug Trials Snapshot</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>Rezdiffra</td>\n",
       "      <td>resmetirom</td>\n",
       "      <td>3/14/2024</td>\n",
       "      <td>To treat noncirrhotic non-alcoholic steatohepa...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>Tevimbra</td>\n",
       "      <td>tislelizumab-jsgr</td>\n",
       "      <td>3/13/2024</td>\n",
       "      <td>To treat unresectable or metastatic esophageal...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>Letybo</td>\n",
       "      <td>letibotulinumtoxinA-wlbg</td>\n",
       "      <td>2/29/2024</td>\n",
       "      <td>To temporarily improve the appearance of moder...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>Exblifep</td>\n",
       "      <td>cefepime, enmetazobactam</td>\n",
       "      <td>2/22/2024</td>\n",
       "      <td>To treat complicated urinary tract infections\\...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>Zelsuvmi</td>\n",
       "      <td>berdazimer</td>\n",
       "      <td>1/5/2024</td>\n",
       "      <td>To treat molluscum contagiosum\\nDrug Trials Sn...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "execution_count": 6
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "#todo add current year to compilation table?"
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.16"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}