SlideSeg.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# SlideSeg\n",
    "\n",
    "Author: Brendan Crabb <brendancrabb8388@pointloma.edu>  \n",
    "Created August 1, 2017 <br>\n",
    "<hr>\n",
    "\n",
    "Welcome to SlideSeg, a python module that allows you to segment whole slide images into usable image\n",
    "chips for deep learning. Image masks for each chip are generated from associated markup and annotation files."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Table of Contents\n",
    "[User Guide](#user-guide)\n",
    "\n",
    "1.    [Dependencies](#1.)  \n",
    "2.    [Anaconda Environment](#2.)  \n",
    "      2.1 [Creating Environment from .yml File](#2.1)  \n",
    "      2.2 [Installing C Libraries (Windows)](#2.2)  \n",
    "      2.3 [Installing C Libraries (Mac OS X)](#2.3)  \n",
    "      2.4 [Launching Jupyter Notebook](#2.4)  \n",
    "      2.5 [Change Jupyter Notebook startup folder (Windows)](#2.5)  \n",
    "      2.6 [Change Jupyter Notebook startup folder (OS X)](#2.6)  \n",
    "      2.7 [Jupyter Kernel Selection](#2.7)  \n",
    "3.     [Setup](#3.)  \n",
    "      3.1 [Supported Formats](#3.1)  \n",
    "      3.2 [Parameters](#3.2)  \n",
    "      3.3 [Annotation Key](#3.3)  \n",
    "5.     [Output](#5.)  \n",
    "      5.1 [Image_Chips](#5.1)  \n",
    "      5.2 [Image_Masks](#5.2)  \n",
    "      5.3 [Text Files](#5.3)  \n",
    "6.     [Run](#6.)  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## User Guide <a class =\"anchor\" id=\"user-guide\"></a>\n",
    "\n",
    "### 1. Dependencies <a class =\"anchor\" id=\"1.\"></a>\n",
    "\n",
    "SlideSeg runs on Python 2.7 and depends on the following Python libraries: <br>\n",
    "\n",
    " * openslide 1.1.1 <br>\n",
    " * tqdm 4.15.0 <br>\n",
    " * cv2 3.2.0 <br>\n",
    " * numpy <br>\n",
    " * pexif 0.15 <br>\n",
    "\n",
    "SlideSeg and the necessary Python libraries can be installed using: <br>\n",
    "\n",
    "<code>pip install slideseg</code><br>\n",
    "\n",
    "If pip isn't installed, you may have to enter the following before installing slideseg (OS X):\n",
    "\n",
    "<code>sudo easy_install pip</code><br>\n",
    " \n",
    "If you are using the preconfigured SlideSeg anaconda environment, these dependencies will already be installed. SlideSeg also depends on several C libraries; see section [2.2](#2.2) (windows) and section [2.3](#2.2) (Mac OS X) for installation instructions.  <br>\n",
    "\n",
    "### 2. Anaconda Environment <a class =\"anchor\" id=\"2.\"></a>\n",
    "\n",
    "Make sure anaconda is installed. The SlideSeg environment has an Ipython kernel with all of the necessary packages already installed; however, conda support for jupyter notebooks is needed to switch kernels. This support is available through conda itself and can be enabled by issuing the following command:\n",
    "\n",
    "<code>conda install nb_conda </code>\n",
    "\n",
    "##### 2.1 Creating environment from .yml file <a class =\"anchor\" id=\"2.1\"></a>\n",
    "Copy the environment_slideseg.yml file to the anaconda directory,  .../anaconda/scripts/. In the same directory, issue the following command to create the anaconda environment from the file:\n",
    "\n",
    "<code>conda env create -f environment_slideseg.yml </code>\n",
    "\n",
    "Creating the environment might take a few minutes. Once finished, issue the following command to activate the environment:\n",
    "\n",
    "* Windows: <code>activate SlideSeg</code>\n",
    "* macOS and Linux: <code>source activate SlideSeg</code>\n",
    "\n",
    "If the environment was activated successfully, you should see (SlideSeg) at the beggining of the command prompt. This will set the SlideSeg kernel as your default kernel when running jupyter. \n",
    "\n",
    "##### 2.2 Installing C Libraries  (Windows)  <a class =\"anchor\" id=\"2.2\"></a>\n",
    "\n",
    "OpenSlide and OpenCV are C libraries; as a result, they have to be installed separately from\n",
    "the conda environment, which contains all of the python dependencies.\n",
    "\n",
    "The Windows Binaries for OpenSlide can be found at 'openslide.org/download/'.\n",
    "Download the appropriate binaries for your system (either 32-bit or 64-bit) and\n",
    "unzip the file.  \n",
    "\n",
    "Copy the .dll files in ../bin/ to .../Anaconda/envs/SlideSeg/Library/bin/.\n",
    "\n",
    "Copy the .h files to .../Anaconda/envs/SlideSeg/include/. \n",
    "\n",
    "Finally, copy the .lib file to .../Anaconda/envs/SlideSeg/libs/.  \n",
    "\n",
    "OpenSlide has now been installed.\n",
    "\n",
    "Use the following tutorial to download OpenCV, either from prebuilt binaries or from source:\n",
    "\n",
    "http://docs.opencv.org/3.2.0/d5/de5/tutorial_py_setup_in_windows.html\n",
    "\n",
    "##### 2.3 Installing C Libraries  (Mac OS X)  <a class =\"anchor\" id=\"2.3\"></a>\n",
    "\n",
    "OpenSlide and OpenCV are C libraries; as a result, they have to be installed separately from\n",
    "the conda environment, which contains all of the python dependencies.\n",
    "\n",
    "If you are using Homebrew, enter the following in the terminal:\n",
    "\n",
    "<code>brew install openslide</code> <br>\n",
    "\n",
    "<code>brew install opencv</code><br>\n",
    "\n",
    "OpenSlide and OpenCV should now be installed in your anaconda environment.\n",
    "\n",
    "##### 2.4 Launching Jupyter Notebook  <a class =\"anchor\" id=\"2.4\"></a>\n",
    "\n",
    "The Jupyter Notebook App can be launched by clicking on the Jupyter Notebook icon installed by Anaconda in the start menu (Windows) or by typing in the terminal (cmd on Windows): \n",
    "\n",
    "<code>jupyter notebook</code>\n",
    "\n",
    "This will launch a new browser window showing the Notebook Dashboard.  When started, the Jupyter Notebook app can only access files within its start-up folder.  If you stored the SlideSeg notebook documents in a subfolder of your user folder, no configuration is necessary.  Otherwise, you need to change your Jupyter Notebook App start-up folder.  \n",
    "\n",
    "#####  2.5 Change Jupyter Notebook startup folder (Windows) <a class =\"anchor\" id=\"2.5\"></a>\n",
    "\n",
    "* Copy the *Jupyter Notebook* launcher from the menu to the desktop. <br>\n",
    "* Right click on the new launcher, select properties, and change the *Target field*, change %USERPROFILE% to the full path of the folder which will contain all the notebooks. <br>\n",
    "* Double-click on the *Jupyter Notebook* desktop launcher (icon shows [IPy]) to start the Jupyter Notebook App, which will open in a new browser window (or tab). Note also that a secondary terminal window (used only for error logging and for shut down) will be also opened. If only the terminal starts, try opening this address with your browser: http://localhost:8888/. <br>\n",
    "\n",
    "##### 2.6 Change Jupyter Notebook startup folder (OS X) <a class =\"anchor\" id=\"2.6\"></a>\n",
    "To launch Jupyter Notebook App:\n",
    "\n",
    "* Click on spotlight, type terminal to open a terminal window.\n",
    "* Enter the startup folder by typing cd /some_folder_name.\n",
    "* Type jupyter notebook to launch the Jupyter Notebook App (it will appear in a new browser window or tab). <br>\n",
    "\n",
    "##### 2.7 Jupyter Kernel Selection <a class =\"anchor\" id = \"2.7\"></a>\n",
    "After launching the Jupyter Notebook App, navigate to the SlideSeg notebook and click on its name to open in a new browser tab. In the upper right corner, you should see  Python [conda env:SlideSeg].  If not, click on Kernel> Change Kernel> and change your current kernel to Python [conda env:SlideSeg]. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3. Setup <a class =\"anchor\" id=\"3.\"></a>\n",
    "\n",
    "   Copy all of the slide images into the images folder in the main project directory. Copy the markup and annotation files (in .xml format) into the xml folder in the main project directory. It is important that the annotation files have the same file name as the slide they are associated with. <br>\n",
    "   \n",
    "##### 3.1 Supported Formats <a class =\"anchor\" id=\"3.1\"></a>\n",
    "\n",
    "SlideSeg can read virtual slides in the following formats: <br>\n",
    "   \n",
    "  * Aperio (.svs, .tif) <br>\n",
    "  * Hamamatsu (.ndpi, .vms, .vmu) <br>\n",
    "  * Leica (.scn) <br>\n",
    "  * MIRAX (.mrxs) <br>\n",
    "  * Philips (.tiff) <br>\n",
    "  * Sakura (.svslide) <br>\n",
    "  * Trestle (.tif) <br>\n",
    "  * Ventana (.bif, .tif) <br>\n",
    "  * Generic tile TIFF (.tif) <br>\n",
    "\n",
    "SlideSeg can read annotations in the following formats: <br>\n",
    "  * XML (.xml) <br>\n",
    "   "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### 3.2 Parameters <a class =\"anchor\" id=\"3.2\"></a>\n",
    "  \n",
    " SlideSeg depends on the following parameters:\n",
    " \n",
    "<p style=\"margin-left: 40px\">\n",
    "\n",
    "<b>slide_path:</b> Path to the folder of slide images <br>\n",
    "\n",
    "<b>xml_path:</b> Path to the folder of xml files <br>\n",
    "\n",
    "<b>output_dir:</b> Path to the output folder where image_chips, image_masks, and text_files will be saved <br>\n",
    "\n",
    "<b>format:</b> Output format of the image_chips and image_masks (png or jpg only) <br>\n",
    "\n",
    "<b>quality:</b> Output quality: JPEG compression if output format is 'jpg' (100 recommended,jpg compression artifacts will distort image segmentation) <br>\n",
    "\n",
    "<b>size:</b> Size of image_chips and image_masks in pixels <br>\n",
    "\n",
    "<b>overlap:</b> Pixel overlap between image chips <br>\n",
    "\n",
    "<b>key:</b> The text file containing annotation keys and color codes <br>\n",
    "\n",
    "<b>save_all:</b> True saves every image_chip, False only saves chips containing an annotated pixel <br>\n",
    "\n",
    "<b>save_ratio:</b> Ratio of image_chips containing annotations to image_chips not containing annotations (use 'inf' if only annotated chips are desired; only applicable if save_all == False <br>\n",
    "\n",
    "These parameters can be specified in the cell below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "Parameters = {\n",
    "    'slide_path': 'images/',\n",
    "    'xml_path': 'xml/',\n",
    "    'output_dir': 'output/',\n",
    "    'format': 'jpg',\n",
    "    'quality': 100,\n",
    "    'size': 128,\n",
    "    'overlap': 1,\n",
    "    'key': 'Annotation_Key.txt',\n",
    "    'save_all': False,\n",
    "    'save_ratio': 'inf'\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### 3.3 Annotation Key <a class =\"anchor\" id=\"3.3\"></a>\n",
    "\n",
    "   The main directory should already contain an Annotation_Key.txt file. If no Annotation_Key file is present, one will be generated automatically from the annotation files in the xml folder.<br>\n",
    "\n",
    "   The Annotation_Key file contains every annotation key with its associated color code. In all image masks, annotations with that key will have the specified pixel value.  If an unknown key is encountered, it will be given a pixel value and added to the Annotation_Key automatically. <br>\n",
    "   \n",
    "   The following functions are defined within the slideseg module and used to generate, edit, and read the annotation key:<br>\n",
    "<code>\n",
    "def loadkeys(annotation_key):\n",
    "    \"\"\"\n",
    "    Opens annotation_key file and loads keys and color codes\n",
    "    :param: annotation_key: the filename of the annotation key\n",
    "    :return: color codes\n",
    "    \"\"\"\n",
    "    \n",
    "def addkeys(annotation_key, key):\n",
    "    \"\"\"\n",
    "    Adds new key and color_code to annotation key\n",
    "    :param annotation_key: the filename of the annotation key\n",
    "    :param key: The annotation to be added\n",
    "    :return: updated annotation key file\n",
    "    \"\"\"\n",
    "    \n",
    " def writeannotations(annotation_key, annotations):\n",
    "    \"\"\"\n",
    "    Writes annotation keys and color codes to annotation key text file\n",
    "    :param annotation_key: filename of annotation key\n",
    "    :param annotations: Dictionary of annotation keys and color codes\n",
    "    :return: .txt file with annotation keys\n",
    "    \"\"\"\n",
    "    \n",
    "def generatekey(annotation_key, path):\n",
    "    \"\"\"\n",
    "    Generates annotation_key from folder of xml files\n",
    "    :param annotation_key: the name of the annotation key file\n",
    "    :param path: Directory containing xml files\n",
    "    :return: annotation_key file\n",
    "    \"\"\"\n",
    "</code>\n",
    "\n",
    "Use the cell below to import slideseg, as well as some other useful modules. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import slideseg\n",
    "import sys\n",
    "import os"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Run the cell below to display the Annotation Key. The first function generates a new annotation keys from the folder 'xml/' if no annotation key exists.  The second function displays the key in the notebook. <br>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "if not os.path.isfile('Annotation_Key.txt'):\n",
    "    slideseg.generatekey('Annotation_Key.txt', 'xml/')\n",
    "\n",
    "file = open('Annotation_Key.txt', 'r')\n",
    "for line in file:\n",
    "    sys.stdout.write(line)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5. Output <a class =\"anchor\" id=\"5.\"></a>\n",
    "\n",
    "##### 5.1 Image<span>&#95;</span>chips <a class =\"anchor\" id=\"5.1\"></a>\n",
    "Every generated image chip will be saved in the _output/image<span>&#95;</span>chips_ folder. The chips are saved with the naming convention of _slide filename<span>&#95;</span>level number<span>&#95;</span>row<span>&#95;</span>column.format_. If the chip contains an area that was annotated and the tags are enabled, it will have an associated tag (under the Subject category) with the annotation key. If the image chip does not contain annotations, the 'NONE' tag will be added. To view these tags, switch to details view and click display 'Subject' in the explorer. The files can be sorted according to their tags. Unfortunately, these tags will only be available if the output format is .jpg. <br>\n",
    "\n",
    "The following functions are defined in the slideseg module and are used to save both the image chips and image masks, as well as attaching exif metadata to the images:\n",
    "\n",
    "<code>\n",
    "def ensuredirectory(dest):\n",
    "    \"\"\"\n",
    "    Ensures the existence of a directory\n",
    "    :param dest: Directory to ensure.\n",
    "    :return: new directory if it did not previously exist.\n",
    "    \"\"\"\n",
    "\n",
    "def attachtags(path, keys):\n",
    "    \"\"\"\n",
    "    Attaches image tags to metadata of chips and masks\n",
    "    :param path: file to attach tags to.\n",
    "    :param keys: keys to attach as tags\n",
    "    :return: JPG with metadata tags\n",
    "    \"\"\"\n",
    "\n",
    "def savechip(chip, path, quality, keys):\n",
    "    \"\"\"\n",
    "    Saves the image chip\n",
    "    :param chip: the slide image chip to save\n",
    "    :param path: the full path to the chip\n",
    "    :param quality: the output quality\n",
    "    :param keys: keys associated with the chip\n",
    "    :return:\n",
    "    \"\"\"\n",
    "\n",
    "def savemask(mask, path, keys):\n",
    "    \"\"\"\n",
    "    Saves the image masks\n",
    "    :param mask: the image mask to save\n",
    "    :param path: the complete path for the mask\n",
    "    :param keys: keys associated with the chip\n",
    "    :return:\n",
    "    \"\"\"\n",
    "\n",
    "def checksave(save_all, pix_list, save_ratio, save_count_annotated, save_count_blank):\n",
    "    \"\"\"\n",
    "    Checks whether or not an image chip should be saved\n",
    "    :param save_all: (bool) saves all chips if true\n",
    "    :param pix_list: list of pixel values in image mask\n",
    "    :param save_ratio: ratio of annotated chips to unannotated chips\n",
    "    :param save_count_annotated: total annotated chips saved\n",
    "    :param save_count_blank: total blank chips saved\n",
    "    :return: bool\n",
    "    \"\"\"\n",
    "\n",
    "def formatcheck(format):\n",
    "    \"\"\"\n",
    "    Assures correct format parameter was defined correctly\n",
    "    :param format: the output format parameter\n",
    "    :return: format\n",
    "    :return: suffix\n",
    "    \"\"\"\n",
    "</code>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The main functionality of SlideSeg is performed by the following functions. These functions takes all of the inputs specified in parameters and uses it to generate image chips and image masks.\n",
    "\n",
    "<code>\n",
    "def openwholeslide(path):\n",
    "    \"\"\"\n",
    "    Opens a whole slide image\n",
    "    :param path: Slide image path.\n",
    "    :return: slide image, levels, and dimensions\n",
    "    \"\"\"\n",
    "    \n",
    "def curatemask(mask, scale_width, scale_height, chip_size):\n",
    "    \"\"\"\n",
    "    Resize and pad annotation mask if necessary\n",
    "    :param mask: an image mask\n",
    "    :param scale_width: scaling for higher magnification levels\n",
    "    :param scale_height: scaling for higher magnification levels\n",
    "    :return: curated annotation mask\n",
    "    \"\"\"\n",
    " \n",
    "def getchips(levels, dims, chip_size, overlap, mask, annotations, filename, suffix, save_all, save_ratio):\n",
    "    \"\"\"\n",
    "    Finds chip locations that should be loaded and saved\n",
    "\n",
    "    :param levels: levels in whole slide image\n",
    "    :param dims: dimension of whole slide image\n",
    "    :param chip_size: the size of the image chips\n",
    "    :param overlap: overlap between image chips (stride)\n",
    "    :param mask: annotation mask for slide image\n",
    "    :param annotations: dictionary of annotations in image\n",
    "    :param filename: slide image filename\n",
    "    :param suffix: output format for saving.\n",
    "    :param save_all: whether or not to save every image chip (bool)\n",
    "    :param save_ratio: ratio of annotated to unannotated chips (float)\n",
    "    :return: chip_dict. Dictionary of chip names, level, col, row, and scale\n",
    "    :return: image_dict. Dictionary of annotations and chips with those annotations\n",
    "    \"\"\"\n",
    "\n",
    "def run(parameters, filename):\n",
    "    \"\"\"\n",
    "    Runs SlideSeg: Generates image chips from a whole slide image.\n",
    "    :param parameters: specified in Parameters.txt file\n",
    "    :param filename: filename of whole slide image\n",
    "    :return: image chips and masks.\n",
    "    \"\"\"\n",
    "</code>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### 5.2 Image<span>&#95;</span>masks <a class =\"anchor\" id=\"5.2\"></a>\n",
    "An image mask for each image chip is saved in the _output/image<span>&#95;</span>masks folder_. The mask has the same name as the image chip it is associated with. Furthermore, these masks will have the same tags, allowing you to sort by annotation type. <br>\n",
    "\n",
    "The following function handles the generation of an annotation mask from xml files: <br>\n",
    "\n",
    "<code>\n",
    "def makemask(annotation_key, size, xml_path):\n",
    "    \"\"\"\n",
    "    Reads xml file and makes annotation mask for entire slide image\n",
    "    :param annotation_key: name of the annotation key file\n",
    "    :param size: size of the whole slide image\n",
    "    :param xml_path: path to the xml file\n",
    "    :return: annotation mask\n",
    "    :return: dictionary of annotation keys and color codes\n",
    "    \"\"\"\n",
    "</code>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### 5.3 Text Files <a class =\"anchor\" id=\"5.3\"></a>\n",
    "A text file with details about annotations and image chips will also be saved to _output/textfiles_. For each slide image, this text file will contain a list of all annotation keys present in the image. For each annotation key, a list of every image chip/mask containing that specific key is also recorded in this file. <br>\n",
    "\n",
    "The following functions generates these .txt files: <br>\n",
    "\n",
    "<code>\n",
    "def writekeys(filename, annotations):\n",
    "    \"\"\"\n",
    "    Writes each annotation key to the output text file\n",
    "    :param filename: filename of image chip\n",
    "    :param annotations: dictionary of annotation keys\n",
    "    :return: updated text file\n",
    "    \"\"\"\n",
    "\n",
    "def writeimagelist(filename, image_dictionary):\n",
    "    \"\"\"\n",
    "    Writes list of images containing each annotation key\n",
    "    :param filename: the name of the slide image\n",
    "    :param image_dictionary: dictionary of images with each key\n",
    "    :return text\n",
    "    \"\"\"\n",
    "</code>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 6. Run <a class =\"anchor\" id=\"6.\"></a>\n",
    "\n",
    "To execute SlideSeg, simply run the jupyter notebook cells below. Alternatively, you can run the python script 'main.py'. Make sure that you defined the [Parameters](#3.2) above. If the python script is used, the parameters are specified in the Parameters.txt file. <br>\n",
    "\n",
    "To get started, run the cell below to make sure all of the necessary modules are imported."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import slideseg\n",
    "import tqdm\n",
    "import os"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The following cell defines a function <code>run(parameters, filename)</code> that generates image chips and masks from the slide image and xml file specified by filename. This function uses the slideseg module to open the slide image, generate an annotation mask, find regions of interest, and save chip data. This function is also defined within the module as <code>slideseg.run(parameters, filename)</code>."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def run(parameters, filename):\n",
    "    \"\"\"\n",
    "    Runs SlideSeg: Generates image chips from a whole slide image.\n",
    "    :param parameters: specified in Parameters.txt file\n",
    "    :param filename: filename of whole slide image\n",
    "    :return: image chips and masks.\n",
    "    \"\"\"\n",
    "\n",
    "    # Define variables\n",
    "    _slide_path = parameters[\"slide_path\"]\n",
    "    _xml_path = parameters[\"xml_path\"]\n",
    "    _output_dir = parameters[\"output_dir\"]\n",
    "    _format = parameters[\"format\"]\n",
    "    _quality = int(parameters[\"quality\"])\n",
    "    _chip_size = int(parameters[\"size\"])\n",
    "    _overlap = int(parameters[\"overlap\"])\n",
    "    _key = parameters[\"key\"]\n",
    "    _save_all = parameters[\"save_all\"]\n",
    "    _save_ratio = parameters[\"save_ratio\"]\n",
    "\n",
    "    # Open slide\n",
    "    _osr, _levels, _dims = slideseg.openwholeslide('{0}{1}'.format(_slide_path, filename))\n",
    "    _size = (int(_dims[0][0]), int(_dims[0][1]))\n",
    "\n",
    "    # Annotation Mask\n",
    "    xml_file = filename.rstrip(\".svs\")\n",
    "    xml_file = xml_file + \".xml\"\n",
    "\n",
    "    print('loading annotation data from {0}/{1}'.format(_xml_path, xml_file))\n",
    "    _mask, _annotations = slideseg.makemask(_key, _size, '{0}{1}'.format(_xml_path, xml_file))\n",
    "\n",
    "    # Define output directory\n",
    "    output_directory_chip = '{0}image_chips/'.format(_output_dir)\n",
    "    output_directory_mask = '{0}image_mask/'.format(_output_dir)\n",
    "\n",
    "    # Output formatting check\n",
    "    _format, _suffix = slideseg.formatcheck(_format)\n",
    "\n",
    "    # Find chip data/locations to be saved\n",
    "    chip_dictionary, image_dict = slideseg.getchips(_levels, _dims, _chip_size, _overlap,\n",
    "                                           _mask, _annotations, filename, _suffix, _save_all, _save_ratio)\n",
    "\n",
    "    # Save chips and masks\n",
    "    print('Saving chips... {0} total chips'.format(len(chip_dictionary)))\n",
    "\n",
    "    for filename, value in tqdm.tqdm(chip_dictionary.iteritems()):\n",
    "        keys = value[0]\n",
    "        i = value[1]\n",
    "        col = value[2]\n",
    "        row = value[3]\n",
    "        scale_factor_width = value[4] \n",
    "        scale_factor_height = value[5]\n",
    "\n",
    "        # load chip region from slide image\n",
    "        img = _osr.read_region([int(col * scale_factor_width), int(row * scale_factor_height)], i,\n",
    "                              [_chip_size, _chip_size]).convert('RGB')\n",
    "\n",
    "        # load image mask and curate\n",
    "        img_mask = _mask[int(row * scale_factor_height):int((row + _chip_size) * scale_factor_height),\n",
    "                         int(col * scale_factor_width):int((col + _chip_size) * scale_factor_width)]\n",
    "\n",
    "        img_mask = slideseg.curatemask(img_mask, scale_factor_width, scale_factor_height, _chip_size)\n",
    "\n",
    "        # save the image chip and image mask\n",
    "        _path_chip = output_directory_chip + filename\n",
    "        _path_mask = output_directory_mask + filename\n",
    "\n",
    "        slideseg.savechip(img, _path_chip, _quality, keys)\n",
    "        slideseg.savemask(img_mask, _path_mask, keys)\n",
    "\n",
    "    # Make text output of Annotation Data\n",
    "    print('Updating txt file details...')\n",
    "\n",
    "    slideseg.writekeys(xml_file, _annotations)\n",
    "    slideseg.writeimagelist(xml_file, image_dict)\n",
    "\n",
    "    print('txt file details updated')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that we have imported the necessary modules and defined the function <code>run()</code>, we can execute SlideSeg by running the cell below, which simply passes the parameter and filename information to <code>run()</code>. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "print('running __main__ with parameters: {0}'.format(Parameters))\n",
    "\n",
    "if not os.path.isdir(Parameters[\"slide_path\"]):\n",
    "    path, filename = os.path.split(Parameters[\"slide_path\"])\n",
    "    xpath, xml_filename = os.path.split(Parameters[\"xml_path\"])\n",
    "    Parameters[\"slide_path\"] = path\n",
    "    Parameters[\"xml_path\"] = xpath\n",
    "\n",
    "    print('loading {0}'.format(filename))\n",
    "    run(Parameters, filename)\n",
    "\n",
    "else:\n",
    "    for filename in os.listdir(Parameters[\"slide_path\"]):\n",
    "        run(Parameters, filename)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "celltoolbar": "Raw Cell Format",
  "kernelspec": {
   "display_name": "slidetest",
   "language": "python",
   "name": "slidetest"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}