Part 4c - Siamese Networks with Tag Similarity.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Finding Similar Songs on Spotify - Part 3 - Siamese Networks with Tag Similarity\n",
    "\n",
    "In this tutorial I will demonstrate how to apply machine learning to search for similar songs on Spotify.\n",
    "\n",
    "## Tutorial Overview\n",
    "\n",
    "1. Loading data\n",
    "2. Preprocess data\n",
    "3. Define Model\n",
    "4. Fit Model\n",
    "5. Evaluate Model\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Requiremnts\n",
    "\n",
    "Install the following dependencies to run this tutorial:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/schindler/anaconda/python2/envs/py36/lib/python3.6/site-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.\n",
      "  from ._conv import register_converters as _register_converters\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "os.environ[\"CUDA_VISIBLE_DEVICES\"] = \"1\"\n",
    "\n",
    "import tensorflow as tf"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2017-08-24T10:20:32.488000Z",
     "start_time": "2017-08-24T10:20:32.483000Z"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    }
   ],
   "source": [
    "# visualization\n",
    "%matplotlib inline\n",
    "import matplotlib\n",
    "import matplotlib.pyplot as plt\n",
    "matplotlib.style.use('ggplot')\n",
    "from IPython.display import IFrame\n",
    "\n",
    "# numeric and scientific processing\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "from scipy.interpolate import RectBivariateSpline\n",
    "from sklearn.preprocessing import LabelEncoder, MultiLabelBinarizer\n",
    "from scipy.spatial.distance import dice, pdist, squareform\n",
    "\n",
    "# misc\n",
    "import os\n",
    "import progressbar\n",
    "\n",
    "# spotify API\n",
    "import spotipy\n",
    "import spotipy.util as util\n",
    "\n",
    "# local caching\n",
    "from joblib import Memory\n",
    "\n",
    "# deep learning\n",
    "from keras.models           import Model\n",
    "from keras.layers           import Input, Lambda, Dense, Bidirectional\n",
    "from keras.layers.recurrent import LSTM\n",
    "from keras.layers.merge     import concatenate\n",
    "from keras.optimizers       import Nadam\n",
    "from keras import backend as K\n",
    "\n",
    "# functions from Tutorial Part 1\n",
    "import tutorial_functions as tut_func"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Loading Data\n",
    "\n",
    "We will use the same data that we downloaded from Spotify in Part 1 of the Tutorial. Because we used the joblib library, we will not have to wait that long, because the data is already cached on our harddrive."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Update the following two variables according the credentials you received from Spotify"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2017-08-24T10:20:33.260000Z",
     "start_time": "2017-08-24T10:20:33.256000Z"
    }
   },
   "outputs": [],
   "source": [
    "SPOTIFY_USER = \"\"\n",
    "\n",
    "os.environ[\"SPOTIPY_CLIENT_ID\"]     = \"\"\n",
    "os.environ[\"SPOTIPY_CLIENT_SECRET\"] = \"\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The same playlists as used in Part 1:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "playlists = [\n",
    "    \n",
    "     {\"name\": \"clubbeats\",    \"uri\": \"spotify:user:spotify:playlist:37i9dQZF1DXbX3zSzB4MO0\"},\n",
    "     {\"name\": \"softpop\",      \"uri\": \"spotify:user:spotify:playlist:37i9dQZF1DWTwnEm1IYyoj\"},\n",
    "     {\"name\": \"electropop\",   \"uri\": \"spotify:user:spotify:playlist:37i9dQZF1DX4uPi2roRUwU\"},\n",
    "     {\"name\": \"rockclassics\", \"uri\": \"spotify:user:spotify:playlist:37i9dQZF1DWXRqgorJj26U\"},\n",
    "     {\"name\": \"rockhymns\",    \"uri\": \"spotify:user:spotify:playlist:37i9dQZF1DX4vth7idTQch\"},\n",
    "     {\"name\": \"soft_rock\",    \"uri\": \"spotify:user:spotify:playlist:37i9dQZF1DX6xOPeSOGone\"},\n",
    "     {\"name\": \"metalcore\",    \"uri\": \"spotify:user:spotify:playlist:37i9dQZF1DWXIcbzpLauPS\"}, \n",
    "     {\"name\": \"metal\",        \"uri\": \"spotify:user:spotify:playlist:37i9dQZF1DWWOaP4H0w5b0\"},\n",
    "     {\"name\": \"classic_metal\",\"uri\": \"spotify:user:spotify:playlist:37i9dQZF1DX2LTcinqsO68\"},\n",
    "     {\"name\": \"grunge\",       \"uri\": \"spotify:user:spotify:playlist:37i9dQZF1DX11ghcIxjcjE\"},\n",
    "     {\"name\": \"hiphop\",       \"uri\": \"spotify:user:spotify:playlist:37i9dQZF1DWVdgXTbYm2r0\"},\n",
    "     {\"name\": \"poppunk\",      \"uri\": \"spotify:user:spotify:playlist:37i9dQZF1DXa9wYJr1oMFq\"},\n",
    "     {\"name\": \"classic\",      \"uri\": \"spotify:user:spotify:playlist:37i9dQZF1DXcN1fAVSf7CR\"}\n",
    "    \n",
    "]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Connect to the Spotify API"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2017-08-24T10:20:54.045000Z",
     "start_time": "2017-08-24T10:20:34.076000Z"
    }
   },
   "outputs": [],
   "source": [
    "token = util.prompt_for_user_token(SPOTIFY_USER, \n",
    "                                   \"playlist-modify-public\", \n",
    "                                   redirect_uri=\"http://localhost/\")\n",
    "\n",
    "sp = spotipy.Spotify(auth=token)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Define the local chache directory. This should be the same as in Part 1 of the tutorial."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2017-08-24T10:20:32.875000Z",
     "start_time": "2017-08-24T10:20:32.870000Z"
    }
   },
   "outputs": [],
   "source": [
    "memory = Memory(cachedir='/home/schindler/tmp/spotify/', verbose=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "@memory.cache\n",
    "def get_spotify_data(track_id):\n",
    "    \n",
    "    # meta-data\n",
    "    track_metadata      = sp.track(track_id)\n",
    "    album_metadata      = sp.album(track_metadata[\"album\"][\"id\"])\n",
    "    artist_metadata     = sp.artist(track_metadata[\"artists\"][0][\"id\"])\n",
    "    \n",
    "    # feature-data\n",
    "    sequential_features = sp.audio_analysis(track_id)\n",
    "    trackbased_features = sp.audio_features([track_id])\n",
    "    \n",
    "    return track_metadata, album_metadata, artist_metadata, sequential_features, trackbased_features"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Start loading the Spotify Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 98% (1059 of 1078) |#################### | Elapsed Time: 0:02:11 ETA:  0:00:03"
     ]
    }
   ],
   "source": [
    "# Get Playlist meta-data\n",
    "playlists = tut_func.get_playlist_metadata(sp, playlists)\n",
    "\n",
    "# Get track-ids of all playlist entries\n",
    "playlists = tut_func.get_track_ids(sp, playlists)\n",
    "\n",
    "num_tracks_total = np.sum([playlist[\"num_tracks\"] for playlist in playlists])\n",
    "\n",
    "# Fetch data and features from Spotify\n",
    "pbar = progressbar.ProgressBar(max_value=num_tracks_total)\n",
    "pbar.start()\n",
    "\n",
    "raw_track_data      = []\n",
    "processed_track_ids = []\n",
    "\n",
    "for playlist in playlists:\n",
    "\n",
    "    for track_id in playlist[\"track_ids\"]:\n",
    "\n",
    "        try:\n",
    "            # avoid duplicates in the data-set\n",
    "            if track_id not in processed_track_ids:\n",
    "\n",
    "                # retrieve data from Spotify\n",
    "                spotify_data = get_spotify_data(track_id)\n",
    "\n",
    "                raw_track_data.append([playlist[\"name\"], spotify_data])\n",
    "                processed_track_ids.append(track_id)\n",
    "\n",
    "        except Exception as e:\n",
    "            print(e)\n",
    "\n",
    "        pbar.update(len(raw_track_data))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Aggregate feature-data\n",
    "\n",
    "Currently we only have a list of raw data-objects retrieved from the Spotify API. We need to transform this information to a more structured format."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Aggregate Meta-data\n",
    "metadata = tut_func.aggregate_metadata(raw_track_data)\n",
    "\n",
    "# Aggregate Feature-data\n",
    "feature_data = tut_func.aggregate_featuredata(raw_track_data, metadata)\n",
    "\n",
    "# standardize sequential_features\n",
    "feature_data -= feature_data.mean(axis=0)\n",
    "feature_data /= feature_data.std(axis=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Tag Similarities\n",
    "\n",
    "Tag similarity refers to the semantic relationship between two labels. In the context of this tutorial we are talking about genre labels. These are provided by the Spotify API for each artist. This is not as precise as for each track, but we take what we get. We still need to consider that an artist might change its artistic orientation which means he can cross or change its genre or on the other hand, the genre can change itself (e.g. Rhythm and Blues). Most of all, we should keep in mind that the concept of music genre is highly subjective and its interpretation might vary between the annotators who labelled the provided tracks.\n",
    "\n",
    "So this is how the labels look like:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Track : Guns N' Roses - Sweet Child O' Mine\n",
      "Labels: 'album rock', 'classic rock', 'glam metal', 'hard rock', 'rock'\n"
     ]
    }
   ],
   "source": [
    "print(\"Track : \" + \" - \".join(metadata[[\"artist_name\", \"title\"]].loc[330]) )\n",
    "print(\"Labels: '\" + \"', '\".join(metadata.loc[330].genres) + \"'\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Calculating similarities between Tracks based on labels\n",
    "\n",
    "To estimate the similarities between tracks, we first have to assess, how many labels  we have and how they are distributed. First we convert the string labels into a binary matrix:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "mlb = MultiLabelBinarizer()\n",
    "mlb = mlb.fit(metadata.genres)\n",
    "\n",
    "genres_bin = pd.DataFrame(mlb.transform(metadata.genres), columns=mlb.classes_)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we remove all tracks for which no labels were provided:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "# filter tracks without genre tags\n",
    "genres_bin = genres_bin[genres_bin.sum(axis=1) > 0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we align the metadata and the feature-data to the filtered metadata:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "# copy the metadata\n",
    "metadata_aligned = metadata\n",
    "\n",
    "# create an additional column as an index into the feature-data\n",
    "metadata_aligned[\"featurespace_index\"] = metadata.index.values\n",
    "\n",
    "# reduce the aligned metadata to those tracks with provided genre labels\n",
    "metadata_aligned = metadata_aligned.iloc[genres_bin.index]\n",
    "\n",
    "# reset the index\n",
    "metadata_aligned = metadata_aligned.reset_index()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Term (Label) Frequencies\n",
    "\n",
    "As a next step we will analyze how many labels we have in our dataset, which ones are the most frequently used and so on.\n",
    "\n",
    "So let's see what are the most used genre labels in our dataset:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA6gAAAHgCAYAAACo1UAAAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzs3XmYZGV5uP97YESJCwRbCcMSjKAEN9xx+Rn3iMqm8iBRgoigEUWjXxU0KjExLhEV1KCDyKIoPLgxGFxxQVRUQGSJmCCiIgiOAuICyNC/P95TTHVPz3SdU28Xh5n7c119ddfpU0+9VXW257zbounpaSRJkiRJuq2td1sXQJIkSZIkMEGVJEmSJPWECaokSZIkqRdMUCVJkiRJvWCCKkmSJEnqBRNUSZIkSVIvmKBKkiRJknrBBFWSJEmS1AuLR10xItYHzgZ+mZnPjIh7AScCmwDnAntn5k0RcUfgeOChwG+APTPzsuollyRJkiStVUZOUIFXAD8C7tY8fgfwnsw8MSI+COwHHNn8viYzt4mI5zbr7TlP7Ol2xZYkSZIk3c4smm+FkRLUiNgCeAbwVuBVEbEIeCLwD80qxwGHUhLUXZu/AT4JvD8iFmXmGpPQK664Yo1lmJqaYvny5aMU1zhrQVmMM5k4fSqLcSYTp09lMc5k4vSpLMaZTJw+lcU4k4nTp7IYZzJx+lSWUeMsWbJkpFij9kF9L/Ba4Jbm8d2BazPz5ubx5cDmzd+bA78AaP5/XbO+JEmSJEmrNW8NakQ8E7g6M8+JiMc3i+eqmp0e4X/DcQ8ADgDITKamptZc0MWL511nFMa5fZTFOJOJ06eyGGcycfpUFuNMJk6fymKcycTpU1mMM5k4fSqLcSYTp09lqRkHRmvi+xhgl4h4OnAnSh/U9wIbR8TippZ0C2DQRvdyYEvg8ohYDGwE/HZ20MxcCixtHk7PVyV8e6zGvr3F6VNZjDOZOH0qi3EmE6dPZTHOZOL0qSzGmUycPpXFOJOJ06eyGGcycfpUllHjVGvim5mHZOYWmbk18Fzgq5n5POBrwHOa1fYBTmn+XtY8pvn/V+frfypJkiRJ0jjzoL6OMmDSJZQ+pkc3y48G7t4sfxVw8HhFlCRJkiStC9pMM0Nmfh34evP3pcAj5ljnBmCPCmWTJEmSJK1DxqlBlSRJkiSpGhNUSZIkSVIvmKBKkiRJknrBBFWSJEmS1AsmqJIkSZKkXjBBlSRJkiT1ggmqJEmSJKkXTFAlSZIkSb2w+LYuwOqs2H+XGY+vmmOd9Y9aNpnCSJIkSZIWnDWokiRJkqReMEGVJEmSJPWCCaokSZIkqRdMUCVJkiRJvWCCKkmSJEnqBRNUSZIkSVIvmKBKkiRJknrBBFWSJEmS1AsmqJIkSZKkXjBBlSRJkiT1ggmqJEmSJKkXTFAlSZIkSb1ggipJkiRJ6gUTVEmSJElSL5igSpIkSZJ6wQRVkiRJktQLJqiSJEmSpF4wQZUkSZIk9YIJqiRJkiSpF0xQJUmSJEm9YIIqSZIkSeoFE1RJkiRJUi+YoEqSJEmSemHxfCtExJ2AM4A7Nut/MjPfHBHHAn8HXNes+oLMPC8iFgGHA08H/tgsP3chCi9JkiRJWnvMm6ACNwJPzMzfR8QdgDMj4vPN/16TmZ+ctf5OwLbNzyOBI5vfkiRJkiSt1rxNfDNzOjN/3zy8Q/MzvYan7Aoc3zzvLGDjiNhs/KJKkiRJktZmi6an15RrFhGxPnAOsA3wgcx8XdPE91GUGtbTgYMz88aI+Bzw9sw8s3nu6cDrMvPsWTEPAA4AyMyH3nTTTTNe86rdHz1vuTb9zLfnXWe2xYsXc/PNN7d+3roQp09lMc5k4vSpLMaZTJw+lcU4k4nTp7IYZzJx+lQW40wmTp/KYpzJxOlTWUaNs8EGGwAsmjfWKC+YmSuAHSJiY+AzEXF/4BDgV8AGwFLgdcBbVvOiq2TBmbm0eR7A9PLly0cpygxdnjM1NdXpeetCnD6VxTiTidOnshhnMnH6VBbjTCZOn8pinMnE6VNZjDOZOH0qi3EmE6dPZRk1zpIlS0aK1WoU38y8Fvg68LTMvLJpxnsjcAzwiGa1y4Eth562BXBFm9eRJEmSJK175k1QI+IeTc0pEbEh8GTg4kG/0mbU3t2AC5unLAP+MSIWRcSOwHWZeeWClF6SJEmStNYYpQZ1M+BrEXE+8H3gy5n5OeCEiLgAuACYAv69Wf804FLgEuAo4KXVSy1JkiRJWuvM2wc1M88HHjzH8ieuZv1p4MDxiyZJkiRJWpe06oMqSZIkSdJCMUGVJEmSJPWCCaokSZIkqRdMUCVJkiRJvWCCKkmSJEnqBRNUSZIkSVIvmKBKkiRJknrBBFWSJEmS1AsmqJIkSZKkXjBBlSRJkiT1ggmqJEmSJKkXTFAlSZIkSb1ggipJkiRJ6gUTVEmSJElSL5igSpIkSZJ6wQRVkiRJktQLJqiSJEmSpF4wQZUkSZIk9YIJqiRJkiSpF0xQJUmSJEm9YIIqSZIkSeoFE1RJkiRJUi+YoEqSJEmSesEEVZIkSZLUCyaokiRJkqReMEGVJEmSJPWCCaokSZIkqRdMUCVJkiRJvWCCKkmSJEnqBRNUSZIkSVIvmKBKkiRJknph8XwrRMSdgDOAOzbrfzIz3xwR9wJOBDYBzgX2zsybIuKOwPHAQ4HfAHtm5mULVH5JkiRJ0lpilBrUG4EnZuaDgB2Ap0XEjsA7gPdk5rbANcB+zfr7Addk5jbAe5r1JEmSJElao3kT1MyczszfNw/v0PxMA08EPtksPw7Yrfl71+Yxzf+fFBGLqpVYkiRJkrRWGqkPakSsHxHnAVcDXwZ+AlybmTc3q1wObN78vTnwC4Dm/9cBd69ZaEmSJEnS2mfePqgAmbkC2CEiNgY+A/ztHKtNN7/nqi2dnr0gIg4ADmjiMzU1NeP/V41QrtnPGcXixYs7PW9diNOnshhnMnH6VBbjTCZOn8pinMnE6VNZjDOZOH0qi3EmE6dPZTHOZOL0qSw148CICepAZl4bEV8HdgQ2jojFTS3pFsAVzWqXA1sCl0fEYmAj4LdzxFoKLG0eTi9fvrx14bs8Z2pqqtPz1oU4fSqLcSYTp09lMc5k4vSpLMaZTJw+lcU4k4nTp7IYZzJx+lQW40wmTp/KMmqcJUuWjBRr3ia+EXGPpuaUiNgQeDLwI+BrwHOa1fYBTmn+XtY8pvn/VzNzlRpUSZIkSZKGjdIHdTPgaxFxPvB94MuZ+TngdcCrIuISSh/To5v1jwbu3ix/FXBw/WJLkiRJktY28zbxzczzgQfPsfxS4BFzLL8B2KNK6SRJkiRJ64yRRvGVJEmSJGmhmaBKkiRJknrBBFWSJEmS1AsmqJIkSZKkXjBBlSRJkiT1ggmqJEmSJKkXTFAlSZIkSb1ggipJkiRJ6gUTVEmSJElSL5igSpIkSZJ6wQRVkiRJktQLJqiSJEmSpF4wQZUkSZIk9YIJqiRJkiSpF0xQJUmSJEm9YIIqSZIkSeoFE1RJkiRJUi+YoEqSJEmSesEEVZIkSZLUCyaokiRJkqReMEGVJEmSJPWCCaokSZIkqRcW39YFWGi7nnDxGv9/yvO2m1BJJEmSJElrYg2qJEmSJKkXTFAlSZIkSb2w1jfxreHUk66dY+nMZTvvufFkCiNJkiRJaylrUCVJkiRJvWCCKkmSJEnqBRNUSZIkSVIvmKBKkiRJknrBBFWSJEmS1AsmqJIkSZKkXph3mpmI2BI4Hvgr4BZgaWYeHhGHAvsDv25WfX1mntY85xBgP2AFcFBmfnEByi5JkiRJWouMMg/qzcCrM/PciLgrcE5EfLn533sy813DK0fE9sBzgfsBS4CvRMR9MnNFzYJLkiRJktYu8zbxzcwrM/Pc5u/rgR8Bm6/hKbsCJ2bmjZn5U+AS4BE1CitJkiRJWnstmp6eHnnliNgaOAO4P/Aq4AXA74CzKbWs10TE+4GzMvNjzXOOBj6fmZ+cFesA4ACAzHzoTTfdNOO1rtr90fOWZ9PPfHvedR5z+Jlr/P+3XvHYeWMc84FL5l1n3wO3mXeduSxevJibb76503Nrx+lTWYwzmTh9KotxJhOnT2UxzmTi9KksxplMnD6VxTiTidOnshhnMnH6VJZR42ywwQYAi+aNNeqLRsRdgE8Br8zM30XEkcC/AdPN78OAF67mRVfJgjNzKbB08P/ly5ePWpRbdXnOQsQYJ87U1FSVMtSI06eyGGcycfpUFuNMJk6fymKcycTpU1mMM5k4fSqLcSYTp09lMc5k4vSpLKPGWbJkyUixRkpQI+IOlOT0hMz8NEBmXjX0/6OAzzUPLwe2HHr6FsAVI5VGkiRJkrTOmrcPakQsAo4GfpSZ7x5avtnQarsDFzZ/LwOeGxF3jIh7AdsC36tXZEmSJEnS2miUGtTHAHsDF0TEec2y1wN7RcQOlOa7lwEvBsjMiyIigf+hjAB8oCP4SpIkSZLmM2+CmplnMne/0tPW8Jy3Am8do1ySJEmSpHXMvE18JUmSJEmaBBNUSZIkSVIvmKBKkiRJknrBBFWSJEmS1AsmqJIkSZKkXjBBlSRJkiT1ggmqJEmSJKkXTFAlSZIkSb1ggipJkiRJ6gUTVEmSJElSL5igSpIkSZJ6wQRVkiRJktQLJqiSJEmSpF4wQZUkSZIk9YIJqiRJkiSpF0xQJUmSJEm9YIIqSZIkSeoFE1RJkiRJUi+YoEqSJEmSesEEVZIkSZLUCyaokiRJkqReMEGVJEmSJPWCCaokSZIkqRdMUCVJkiRJvWCCKkmSJEnqBRNUSZIkSVIvmKBKkiRJknph8W1dgHXJEUccMe86Bx100ARKIkmSJEn9Yw2qJEmSJKkXTFAlSZIkSb1ggipJkiRJ6oV5+6BGxJbA8cBfAbcASzPz8IjYBDgJ2Bq4DIjMvCYiFgGHA08H/gi8IDPPXZjiS5IkSZLWFqPUoN4MvDoz/xbYETgwIrYHDgZOz8xtgdObxwA7Ads2PwcAR1YvtSRJkiRprTNvgpqZVw5qQDPzeuBHwObArsBxzWrHAbs1f+8KHJ+Z05l5FrBxRGxWveSSJEmSpLVKqz6oEbE18GDgu8CmmXkllCQWuGez2ubAL4aednmzTJIkSZKk1Rp5HtSIuAvwKeCVmfm7iFjdqovmWDY9R7wDKE2AyUympqZm/P+qEco0+zldjBbj2omUZZw4ixcvHrsMNWIY5/YVp09lMc5k4vSpLMaZTJw+lcU4k4nTp7IYZzJx+lQW40wmTp/KUjMOjJigRsQdKMnpCZn56WbxVRGxWWZe2TThvbpZfjmw5dDTtwCumB0zM5cCS5uH08uXL29d+C7PWYgYfYgzNTU1dhlqxDDO7StOn8pinMnE6VNZjDOZOH0qi3EmE6dPZTHOZOL0qSzGmUycPpVl1DhLliwZKdYoo/guAo4GfpSZ7x761zJgH+Dtze9Thpa/LCJOBB4JXDdoCixJkiRJ0uqMUoP6GGBv4IKIOK9Z9npKYpoRsR/wc2CP5n+nUaaYuYQyzcy+VUssSZIkSVorzZugZuaZzN2vFOBJc6w/DRw4ZrkkSZIkSeuYVqP4SpIkSZK0UExQJUmSJEm9YIIqSZIkSeoFE1RJkiRJUi+YoEqSJEmSemGUaWbUM/e85JBVF14C9xx6ePU2b5tYeSRJkiSpBmtQJUmSJEm9YIIqSZIkSeoFE1RJkiRJUi+YoEqSJEmSesEEVZIkSZLUCyaokiRJkqReMEGVJEmSJPWCCaokSZIkqRdMUCVJkiRJvWCCKkmSJEnqBRNUSZIkSVIvmKBKkiRJknrBBFWSJEmS1AsmqJIkSZKkXjBBlSRJkiT1wuLbugC67Zx00d7zrrPn/T46gZJIkiRJkjWokiRJkqSeMEGVJEmSJPWCCaokSZIkqRdMUCVJkiRJvWCCKkmSJEnqBRNUSZIkSVIvmKBKkiRJknrBBFWSJEmS1AuLb+sC6PZvyXkXrLps1uMrdnjAZAojSZIk6XbLGlRJkiRJUi/MW4MaER8BnglcnZn3b5YdCuwP/LpZ7fWZeVrzv0OA/YAVwEGZ+cUFKLckSZIkaS0zShPfY4H3A8fPWv6ezHzX8IKI2B54LnA/SivPr0TEfTJzRYWySpIkSZLWYvM28c3MM4DfjhhvV+DEzLwxM38KXAI8YozySZIkSZLWEeMMkvSyiPhH4Gzg1Zl5DbA5cNbQOpc3y1YREQcABwBkJlNTUzP+f9UIBZj9nC5Gi3HtRMoycpxLKsUZwW0ZZ/HixVVe3zi3j7IYZzJx+lQW40wmTp/KYpzJxOlTWYwzmTh9KotxJhOnT2WpGQe6J6hHAv8GTDe/DwNeCCyaY93puQJk5lJg6WCd5cuXty5El+csRIxJx7lnpTijGCXO7BF7u8aZbWpqqsr7MM7toyzGmUycPpXFOJOJ06eyGGcycfpUFuNMJk6fymKcycTpU1lGjbNkyShZQ8cENTNvreCMiKOAzzUPLwe2HFp1C+CKLq8hSZIkSVq3dJpmJiI2G3q4O3Bh8/cy4LkRcceIuBewLfC98YooSZIkSVoXjDLNzCeAxwNTEXE58Gbg8RGxA6X57mXAiwEy86KISOB/gJuBAx3BV5IkSZI0inkT1Mzca47FR69h/bcCbx2nUJIkSZKkdU+nJr6SJEmSJNVmgipJkiRJ6gUTVEmSJElSL5igSpIkSZJ6wQRVkiRJktQLJqiSJEmSpF4wQZUkSZIk9YIJqiRJkiSpF0xQJUmSJEm9sPi2LoA0sGL/XWY8vmqOddY/atlkCiNJkiRp4qxBlSRJkiT1ggmqJEmSJKkXTFAlSZIkSb1ggipJkiRJ6gUTVEmSJElSL5igSpIkSZJ6wQRVkiRJktQLzoOqtc5886k6l6okSZLUT9agSpIkSZJ6wRpUaQ67nnDxvOuc8rztJlASSZIkad1hDaokSZIkqRdMUCVJkiRJvWCCKkmSJEnqBRNUSZIkSVIvmKBKkiRJknrBBFWSJEmS1AtOMyMtoFNPunaOpTOX7bznxpMpjCRJktRz1qBKkiRJknrBBFWSJEmS1As28ZVuB4444oh51znooIMmUBJJkiRp4ViDKkmSJEnqhXlrUCPiI8Azgasz8/7Nsk2Ak4CtgcuAyMxrImIRcDjwdOCPwAsy89yFKbokSZIkaW0ySg3qscDTZi07GDg9M7cFTm8eA+wEbNv8HAAcWaeYkiRJkqS13bwJamaeAfx21uJdgeOav48DdhtafnxmTmfmWcDGEbFZrcJKkiRJktZeXfugbpqZVwI0v+/ZLN8c+MXQepc3yyRJkiRJWqPao/gummPZ9FwrRsQBlGbAZCZTU1Mz/n/VCC82+zldjBbj2omUZeQ4l1SKM4JJxqn1nc8XZ7KfTc+2nVkWL15c5fWNc/uJ06eyGGcycfpUFuNMJk6fymKcycTpU1mMM5k4fSpLzTjQPUG9KiI2y8wrmya8VzfLLwe2HFpvC+CKuQJk5lJgafNwevny5a0L0eU5CxFj0nHuOe8aky3PkkpxRrGufucLFWdqaqrK6xvn9hOnT2UxzmTi9KksxplMnD6VxTiTidOnshhnMnH6VJZR4yxZMkrW0D1BXQbsA7y9+X3K0PKXRcSJwCOB6wZNgSVJkiRJWpNRppn5BPB4YCoiLgfeTElMMyL2A34O7NGsfhpliplLKNPM7LsAZZYkSZIkrYXmTVAzc6/V/OtJc6w7DRw4bqEkLYx7XnLIzAWXrNpk/Opt3jax8kiSJEnDuo7iK0mSJElSVSaokiRJkqReqD3NjKR1wEkX7T3vOnve76MTKIkkSZLWJiaokm4zS867YNVlsx5fscMD5o2zYv9dZjyePRfu+kcta1kySZIk3RZs4itJkiRJ6gVrUCWJVWthwZpYSZKkSbMGVZIkSZLUCyaokiRJkqReMEGVJEmSJPWCfVAlqaJdT7h43nVOed52EyiJJEnS7Y81qJIkSZKkXjBBlSRJkiT1ggmqJEmSJKkXTFAlSZIkSb1ggipJkiRJ6gUTVEmSJElSLzjNjCT10KknXTvH0pnLdt5z48kURpIkaUKsQZUkSZIk9YIJqiRJkiSpF2ziK0lrsSOOOGLedQ466KAJlESSJGl+JqiSpHnd85JDZi64BO45a52rt3nbvHFOumjvedfZ834fXeP/l5x3wdzLh/6+YocHzPs6kiSpf2ziK0mSJEnqBWtQJUnrpBX777LKsqtmPV7/qGWTKYwkSQKsQZUkSZIk9YQ1qJIkjaFWTeyuJ1w87zqnPG+7UYslSdLtkgmqJElrkVNPunbWktmPYec9N543jiNAS5JuCzbxlSRJkiT1ggmqJEmSJKkXbOIrSZIWTK05dCVJ6wZrUCVJkiRJvWCCKkmSJEnqhbGa+EbEZcD1wArg5sx8WERsApwEbA1cBkRmXjNeMSVJkiRJa7saNahPyMwdMvNhzeODgdMzc1vg9OaxJEmSJElrtBCDJO0KPL75+zjg68DrFuB1JEnSOuCki/aed5097/fRCZREkrTQxk1Qp4EvRcQ08KHMXApsmplXAmTmlRExe7A+SZKkiVty3gVzLx/6+4odHjCZwkiS5jRugvqYzLyiSUK/HBEXj/rEiDgAOAAgM5mamprx/6tGiDH7OV2MFuPaiZRl5DiXVIozgknGqfWdzxdnsp+N284k4tT4zid1zBk9jtvOQseY1DFn1DijcNtZ2Bijxrlq90evumzW400/8+2JxZlt8eLFVT4P49x+4vSpLMaZTJw+laVmHBgzQc3MK5rfV0fEZ4BHAFdFxGZN7elmwNWree5SYGnzcHr58uWtX7/LcxYixqTjjFIlPcnyLJl3jX59zn0qy6TjrKvbzu3xu+pbnD5tO7e3Y866HmdS206fjjmTjrPrCfPXD5zyvO1av/bU1FSV92GchY/Tp7IYZzJx+lSWUeMsWTLKUXiMQZIi4s4RcdfB38BTgQuBZcA+zWr7AKd0fQ1JkiRJ0rpjnBrUTYHPRMQgzscz8wsR8X0gI2I/4OfAHuMXU5IkSQvp1JNmNw9ftbn4zntuPG+cI444Yt51DjrooFGLJWkd0zlBzcxLgQfNsfw3wJPGKZQkSZLWXfe85JBVF14ys8n41du8bWLlkTQ5CzHNjCRJknSbqzVFkSNAS5NjgipJkiRNwIr9d1ll2eyRm9c/alnrOHONJj5KnIUaYEsahwmqJEmSpM5q9V+WwARVkiRJ0m3MwbU0YIIqSZIkaa3gAFu3f53nQZUkSZIkqSZrUCVJkiRpSK0RoNWeCaokSZIkLQCnKGrPJr6SJEmSpF4wQZUkSZIk9YJNfCVJkiSpx1bsv8uMx1fNsc76Ry2bWJxdT7h43nVOed52864zF2tQJUmSJEm9YIIqSZIkSeoFE1RJkiRJUi+YoEqSJEmSesEEVZIkSZLUCyaokiRJkqReMEGVJEmSJPWCCaokSZIkqRdMUCVJkiRJvWCCKkmSJEnqBRNUSZIkSVIvmKBKkiRJknrBBFWSJEmS1AsmqJIkSZKkXjBBlSRJkiT1ggmqJEmSJKkXTFAlSZIkSb1ggipJkiRJ6gUTVEmSJElSL5igSpIkSZJ6wQRVkiRJktQLixcqcEQ8DTgcWB/4cGa+faFeS5IkSZJ0+7cgNagRsT7wAWAnYHtgr4jYfiFeS5IkSZK0dlioJr6PAC7JzEsz8ybgRGDXBXotSZIkSdJaYNH09HT1oBHxHOBpmfmi5vHewCMz82VD6xwAHACQmQ+tXghJkiRJUp8smm+FhapBneuFZ2TCmbk0Mx+WmQ9r1l/jT0ScM8p6xlk7ymIcv3Pj+J0bx+/cOLf/shjH79w4fuezfua1UAnq5cCWQ4+3AK5YoNeSJEmSJK0FFmoU3+8D20bEvYBfAs8F/mGBXkuSJEmStBZYkBrUzLwZeBnwReBHZVFeNGbYpWMXzDiTiGGc21ecPpXFOJOJ06eyGGcycfpUFuNMJk6fymKcycTpU1mMM5k4fSpLzTgLM0iSJEmSJEltLVQfVEmSJEmSWjFBlSRJkiT1ggnqhEXEJnMsu9dtURbd/kXEHW/rMqxNImLrOZY9/DYoyioiYr2IuNttXY5xzbXNznVc1Nqjz/uVJK3L+nptsVCj+FYREXfMzBtnLdskM3/bIsZDM/OcWct2zsxTW5Zlp8z8/KxlL8nMD7aJA5zaxPpdE2N7IIH7t4zTWUQ8ZE3/z8xzJ1UWgIh41Zr+n5nvHjHO+5g13+6sOAe1LNd+mXn0rGVvz8yD28SpJSI+kpkvHHp8F+AU4EktYtTaH96SmW8aerw+cHxmPq9NnHFFxLPW9P/M/HTLkJ9uPo9fNvH/Dng/8ICW5apyvIiIjwMvAVYA5wAbRcS7M/M/28SpISK+CZwBfBP4VmZe3zHUpyNit8z8cxN3M+BzwEM7lOlZwGMp+/2ZmfmZjmUaW8V9a1PgP4AlmblTc4541Oxj0Qhx/hrYNjO/EhEbAovH+M7GVWW/ap7bp/dVzbjbcq3zVUS8E/h34E/AF4AHAa/MzI+N+Pyqx+S+nYeb1+/FNhgRGwP/CGzN0PV822udvhl3G1yA8tQ6Jlf7viLinsCdhmL8vOXzq1xbRMQrgGOA64EPAw8GDs7ML7WJM5deJ6jUuZA5KiL2ycwLmhh7Aa8EWl00AG+MiBsz86tNnNcBjwfaJqj/QUlSnwHcFzgeaH1RHxF/AxwOPAq4BfgO8M+ZeekITz9sDf+bBp7YtjxNmTYEtsrMH7d86l2b3/cFHg4sax7vTLkgHtXZLV93Ps+JiBsy8wSAiPgvoFWNZUScypqT5l1ahPtlRByZmf8UEX8J/DdwVJvyUG9/2CoiDsnMtzU1YicDrW9sRMR6wPmZ2fUGzc5r+N800DZBfTHw2YjYGXgIZX99eody1TpebJ+Zv4uI5wGnAa+jnExaJ6gR8UBWPTG2+Xz2oVxAPxv4z4i4EfhmZv5zy6J8Fjg5Ip5NmS97GfD/WsYY7I/bAJ9oFr04Ip6cmQeO+PwqN8aG1Nq3jqWc8N/QPP5f4CRg5IuhiNgfOADYBLg3ZS7yD9LiZtZQrBo3W6rsV+O+r4i4gLmPx4uA6cx8YIcydT2IycH9AAAgAElEQVTvDccYa1tujH2+ajw1M18bEbtT5rTfA/gaMGpyUPuYXOt9VUk0Ku9bGwD3aR7+eHCt28JpwFnABZTrwLavfz1r3h9a16qNeW06MO42OFyecT9jqHBMboz1fQFExC6Ua/glwNXAX1NmS7lfy1C1ri1emJmHR8TfA/cA9qV8Vmt9glrjQuY5wCebL+GxlLsXT+1Qll2Az0XEa4CnAds1y1rJzP+OiDtQvry7Artl5v91KM/HgQ8AuzePn0s5uT1yhDI8ocPrrVFz4fEuYAPgXhGxA/CWURKwzPzXJsaXgIcM7kRGxKGUxGckmXlch6KvybOAZRFxC7AT8NvMfGnLGO+qVZjMfGNEvCMiPki5SfP2zPxUyzC19od9gRMi4hDgCcDnM/M9bYNk5i0R8cOI2KrtHcDm+fu2fc488b4fEQdR9s8bgKdk5q87hKpyvADu0BwvdgPen5l/jojWQ69HxEeABwIXsfLE2OpiMTMvjYg/ATc1P08A/rZtWTLzqOai4bOUhPnFmfnttnGAvwPun5nTABFxHOXEP6q7zr9KK7X2ranMzGbfIjNvjogVLWMcCDwC+G4T4/+aO+5djH2zpeJ+Ne77emaH11ytcc57s4y7LUOd8xXAHZrfTwc+kZm/jYiRn1z7mEy99wV1Eo0q+1ZEPB44DriMkhBu2dzganNT/k6ZucYbbWuSmbWPgTDGtemQsbbBgUqfMdQ5JsOY31fj34Adga9k5oMj4gnAXh3iVLm2oHyuUL6rYzLzhxGxaE1PGFWvE9QaFzLNRdVzmxi/oNyZ+VOHsixv7lx8hXKX4TmDk8koYtXmp3cDLgVeHhFdqvgXZeZHhx5/LCJe1jIGEXF/YHtmNhU4vm0c4FDKQfvrTYzzYo5+R/PYinLhO3AT5XtvJSLuQbkbNPt9jVQzHDP7w72Isu18C3hLtGxinpnfGHXdNZRnuMnU94A3Nr+nI+JZbWrBxt0fYmbz8MOBD1E+m29ExEOyW/PwzYCLIuJ7wB+GytrqIq9plXA/Zn7nbxnxubNruv8CuA44utk/W5Vl3OPFkA9RTq4/BM6I0rTsdx3i7JiZ23d43q0i4ifAcsoFyNHAyzNz5LvAs2osF1FuOp4H7BgRO3aosfwx5Zjxs+bxlsD5oz55cGOsllrnGuAPEXF3mu0xInakbItt3JiZNw0u6iJiMWtoyTGPzjdbau9XjPm+MvNn86/VyqGMf96DMbblmuerxqkRcTGleeVLm/PpDS1jDMo2zjG59vuCOolGrX3rMMox4sdNnPtQErk2LQQ/2tTofg64tTtcx89m7GajjRrXprW2wRqfMdQ5JkOd7+vPmfmbKP1G18vMr0XEOzqUpda1xTlN5dK9gEMi4q50rB2erZcJao0LmVi1Kc8mwPrAd5sT40hNeYaaQCxqfm8A/A2l2UmbJhCzm5+eM+dao/taRBwMnNiUa0/gvwcH9VE2+Ih4M+VO+PaUKv6dgDMpzY7bujkzr+tyl2vIR4HvRcRnKO9p945lOYFyV/QZlDb2+wBt7tafw8zvfFET6xnN479pW6CI2BZ4G6smzaPEmt1k6geUO4w7M2ItWK39gVWbh19DeU+H0b15+NiJQlOr/BeUWr0PU2qzvtciRJWa7orHCwAy8wjgiKFFP2vumLb1nYjYPjP/p8NzB46g1AzuReln8o2IOCMzfzLi82ffrf/MapaP6u7Aj5obG1C6B3wnIpbB6Dc4IuJOwH6seiH9wtU+aebza+1bA6+mtBa6d0R8i9Js6jktY3wjIl4PbBgRTwFeSvumxsDYN1uqtSBpVHlfzQXm+ygtADagfF9/6NCkscZ5D8bblquerzLz4OaC93eZuSIi/gDs2iYGVDkmVz8PUyfRqLVv3SGHmoVn5v82NVpt3ERpkvkGVh6DWn82Ua/ZKFS4Nq21DVLnMwZ4FeMfk6HO93VtlPFHzqC0YrsauLltQSpeW+wH7ABcmpl/bPavKq0oepmgUudCpkpTnlpNIAbNTyPizsANmbmiebw+3fpT7Nn8fvGs5S9k9A3+OZTO5z/IzH2j9M/4cIeyAFwYEf8ArN8kYwcBbWu73xoRnwf+v2bRvpn5gw5luXtmHh0Rr2hqML8RESPXZGbmQoyqfAzwZuA9lBP2vqxsGjFfeWrs7LX2h+rNwzPzGzFz0Im/oFwwtvHozHxgRJyfmf8aEYfRrvnqNwCijKh9ZWbe0DzeENi0RZzqTabmqoUARqqFGHIc5YL3V5Q7t6373GXm4cDhzclxX0rt0RaM+F3VrrEE3jT/KiP5KHAx8PeUz/V5lIuzUVVtNpqZ50QZROi+lO+pS7+pgykXDhdQzhGn0fLYXuNmS639asjY76vxfkrTw5OBh1GaY2/TIc7Y571G5215gc5XmwNPaW7eDLS9WTzuMXkh3leNRKPWNnh2RBxNOf5AOe60rbh4FbBNZi7v8PrDajUbhTrXplBnG6zxGZOZ51Y4JkOd72tXSs3yP1Pez0a0vx5YbX9sWvarzdJN66fAfWZ9V2PrZYJa40Jm0JSnuUN2Ua7s13hXSo1Pq6Y+UTprfzUzr2sebww8PjM/27JopwNPBn7fPN6Q0i/n0W2CVDp4/6nZuG6OMsT01XS7KwnwcspdoRspTSi+SDnozWtWU57Lmp9b/9ehucrgwHFlc3F/BeVCupWIOBA4ITOvbR7/JbBXZv5X21jAhpl5ekQsarbNQ6OMivrmFuU5DnjFrPIcNkpNzwLsD/8BvHNWWV6dmf/SJk7z3NmDTmxO+0EnBk0p/xgRS4DfUJqctHUyM/fFFc2yVlNi1DpeVKiFGPgIsDfjDc5wGKUG9S6UgR7eRBnRt22c+1DGEtiamQM2tap9b25sbMrK7+Z7mXl12/JQLhj2iIhdM/O4KKMbfrFFOWrvWz+ktAA5qUXt9Owy3UIZQO2o5vi6Rdsm5pVvtlTZr2q8r6FYl0TE+s3N4mMiokti2fm8N6ssY2/Ltc5XFVtWVTkmV3xf61Fu8nVONJoKheMy8/m0H6Bwtn+i9Gc9qCnLGUDba4uLgD+OWQ6o12y0yrVpxW2wxmdMROwBfCEzL4qIfwEeEhH/nu27NI31fTXb3ymZ+WTKeXyccVeOpcLATxHxIuAVlGvs8yg3Or5Dx8FWh/UyQR2IiC8De8w6MJ2YmX/fIsyRlFEDB/4wx7JRvDmHhn3PzGubnahtgnqnzBwkp2Tm75sao1aaZgr/BDyuWfR14EMt7+qc3Vw4H0W5q/R7ul38kpl/pGzob5hv3TkMN+WZrUtTnn+PiI0oTeXeR+nv+8oO5do/Mz8weJCZ1zTJVJcE9YbmBPl/Ufpj/BJoO7DCAwf7wlB5HtwyRq39YafMfP2ssjwdaJ2gUmfQic812/J/UkYTnqbbXe3FmXlrP+gsfY026BCn1vFirFqIIT/PzGXzr7ZGZ1FuSlw1ZpyTKTcgPkxJVDqJiKB831+nHDveFxGvycxPtgw1OGZeG6VP/q/o0PedevvWLpRaiIwyMMxJQGaL/mAR8fUmzmLKRcOvI+Ib2WGAjko3W6rsVxXf1x+b1z8vypQWVwJ3blueMc97t6q0Ldc6X9VqWVXrmFzlfTU34w/LzEdREoXWsjQ3vUdEbDC8PXeMdSPw7uanqxWUbfhrzOzT2HZMkyrNRgdi/LFNqmyDlT5jgDdm5skR8VhKS5t3UY7tbQZ+gjG/r2b7+2NEbDQ4Ho+h1sBPr6DcWDsrM58QEdtRodsW9DxBBe4xxwV52wvXRcN3WJuDVJf3vd4cy7rE+UMMDSYTEQ9l5Z3GNo6k9EMcHKT3bpa9aNQAuXIkvA9GxBeAu2XmyIOMDIu5p1O5jtL39kODpl2rKUftpjzXNDvvdZSaJyLiMR3irNfUeA76q6xPaebWxSspNWEHUe6wP4HSrKxtef4yM69pyrMJ7bfBWvvD+jE0T3GUJnudhv6nzqAT72zK8qmI+BzlxNhlUIVfR8Qug2QuInalDAzUVq3jRa2a4YubmsFTmXlibNPk7uSI2CUiBjfFvpEt5/hs3JyZR3Z43mxvAB4+qGmKMpDGV4C2CerS5ubnv1Ca/92Fbk0uq+xbTY3sO4F3Rmk2+kbgHbRr9r5RlikEXkQZWfHNEdHp2E6dmy219qta72tvyj76MkpTuS0po8W2Ms55b5Ya23Kt81WtllW1jsk1z8NfijIrxKe71rxTWnh9K0r/4OFB/VolQRHxTMq1wF9Tzg1dpnb5LO1ves5lrmajnZKMSrWfVbbBSp8xrLyR+gzgyMw8JcosE23V+L5uAC5oKvCGt7+2NyVqDfx0Q2beEBE014QXR8R9O8RZRd8T1BUxNP1ElH5qbQ8ql0YZ3n5wQfRSyui5bZ0dEe+mDJ89TWna02Wgo1dSps65onm8GSvb7Lfx8Mx80NDjr0ZpGjayiDg9M58EkJmXzV7W0qWU/hyDedz2BK6izD91FOWCYJQy7cJQrXBmfq5DWd7HqrUWcy2bzxcptRgfpHznL6FMGt3F1pn5fUot9b5wa7OR77aIcRjw7YgYXLTsAby1ZTlq7Q8fA06PiGMon80L6d7cpMagE9+h+X6bi6IbI+Jc2n/nL6HcQR7csf8FI267s9Q6XtSqhdiQkpgOT3vSapqZiHgbpab7hGbRQRHx6Mw8pGVZTo2Il1LGFhhn5Mn1cmYzyN8w942BNcrMwed5Bt27OEC9fYsoI8EG5Ti6AnhtyxCLo8wbHoxZu0edmy219qta72u3LH2qb6C5EI8y4fzhLeNUOe9RZ1uudb6q1bKq1jG55nn4VZSa8hVRpszqkrBc0fysx3hTVb2XclPkgq7JcpYuCTXm+XxTZr6OoWajUZr4vq5DrBq1n7W2wbE/48YvI+JDlO5574gy93uXc02N7+u/m59x1Rr46fLmu/os8OWIuIayf4yt7wnqG4AzY+UAN4+j9FVr4yWUkaoGTQ+/0iEGlAvMN1KaWy2i9BttM4k2cOt8cNuxsg/ExR0PKCsi4t7Z9FGKMjnySNXzUToy/wUw1dQcDJrW3o0yilsXD87Mxw09PjXKCJ+Pi4iRmtNExNspTQUGF8CviIjHjHoBHBGPovRzukfMHAn6brQfdAfKwfnFlKbUg++86yBSh7DqnK5zLVutzDw+Is5mZdv+Z2X7UVmr7A+Z+c6m5uLJzaJ/y8yR++3NssqgE5k5Uv+eiPgrSp/VDaM0dx7ells3nW/2px2jNHdalE1/wg5qHS8G/dlurYXo0rQn6wy09Qxgh2ymlonSJ/oHlO24jX2a368ZWtalKf8XIuKLzEwOTmsZo2Z/6ir7VkR8l9I65mRKF5cuSe5bKBf2ZzbnnL8Busy3DRVutlTcr2q9r31YNRl9wRzL5jP2ea9RY1uucr4at2VV7WMyFc/DWaFfda6ct/2ulOT29/M8ZXV+AVw4TuIU9eb5fAqrJqM7zbFsFDeMW/tZsXXf2J9xIyhTbL2raUGyGTPPX6MFqfB9NUnuhsBWOTRCcVtZaeCnzBzMd3tolKbLG9H9BtIMvU5QM/MLUeZd3LFZ9M/ZcvSr5q7kcyuU5Q/Awc0Od0vXg1LM0Xc0Itr2HYWyc3wtIgYXL1sz+tDOL6bU5C6h1MoM/I5yEdLFPWbVdm8FTDX/G7WvxtMZ7wJ4A0rzvMXMvLP5OzrcGWoOskdTmqdMU3bgVm30I2InyvvaPCKGh/S+G936eNyBlaNqth4uvdb+0BhMdzPd/N3Vy5vajFuT0igjMI9ysfj3lAvLLZjZz+R3wOvnesKaROm7/Gaa/bO5OfaWtklhxePF+pTEcGua43WUqUtGak4Wq86/PLucbZsFbQwMajo3avncwWtWadKfma+JMkfwYyn7xNLhpqgtVOlPXXHf2iczLx4nQGaezNDNrybJfXbHcGPfbKm4X431viJiL+AfgHs1TTQH7kaptWyrxnmvyrZc43w1EBEPZOYxZ5scvTtA1WNyzfcF47fSitK/8qOUQf2IiOXAP2Zm236trwVOa/aF4ZYkbZoKjzXPZ0T8E6Wlx9/EzKbyd6XMOdtKRCwCzq9R+znmNjhQ4zOGsk+f3ZRjq2ZZl2P02POyRsTOlD6wG1COYztQjqVt542f3cXsIc21ReupHZubulsC1zc/92dmbtFJrxPUxqNZeTCBMsHtyCJiC0rzzsdQDm5nUkZCvbxlnAdQ2tAPH5T2ycwL28ShQt/RxrcoE+0OmuN+iNKkZl65crqIl2fm+1q+7uq8mlLb/RPKCfZelAmW70y7pp+dL4Bz5ZQyx2bmzyLizk2i0Emlu5NXUA5suzCz1uF6Sn+PNuV5BbA/8KmmPB+LiKVtvsOK+0OtAWpgjNqMLNM3HRcRz87MT3V47dk+AlxIuWMKZf88hpb90yoeL06l6XNCt9F3B/MvP4bSJ+ik5vEetG9y/DbgB81d0kWU43Lb2tO5ToxA60E0Br5NaTlyC/D9Ds+HSv2pa+1bWfrwrDK1UGaOPJVArGx6Pzv2SHO7znpOjZsttfarcd/XtykDIk0xc07n64EuNTS1znuDsnXelmvVpkXER4AHUgYSGhxzRu4OUPuYXLGWcHWttB6bmQe3CLMUeFVmfm2ofEfRciYGSvec31P28a59ased5/PjwOcpx/bhz+D6bN/lgsycjogdmtYonWs/x90Gh9T4jKE0qR0M5Hknyn7+Y9rPE1tjXtZDKV1tvt7EOC/KNF5tDY+gfidKLnEuLUdKjoh/o1yvXcrM72qtH8V3rCafjWMoO+EezePnN8ue0rI4H2LVg9JS2h+Uxu472jieckdy0ARwL8pdvT1W+4xVfSTKkNlbZeYBUQbkuG/bO4oAmXla8/ztWNl0eTAgwntHDFPlAhhYEmU+1bsAW0XEg4AXDzUbGdXYd7sy84fAD6MMULOY8Zpl7Ac8cpB0R+kj8h3KRfGoau0PYw/qsYbajLvSvjbjW81d9hlzemVmqyHTgXtn5nCNzL9GxHktY0C948UW2WKu0tly5fzLLwCeMGipEaU/15daxvpElFFUH07ZP1+Xmb/qUKxaJ8YXUQYz+iorb5K8JTM/0rI8tfpTV9m3os7UQsPH8DsBu9OxX1Clmy219qux3leWAah+BjwqZs69vCGln3arpseVznu1tuWxz1eNHTNz+5bPmUutY3Kt9wWrb6XVJkG98+C4DpCZX29uSLS1SWY+df7V1miseT5z5WCSe0UZoXbbzDwmIqYi4l6Z+dMOZTorIh6emd/PZmyTDmptgzU+YzLzAcOPo7TsnD3P6yhqzMt6c2ZeFxHDy1o3Yc7Mlw8/jtLK5aOrWX1NgnJ8H2tU67n0OkFl/CafUEYCPmbo8bER0WXKkVoHpc59R2e576xE92sdEt2PUHaOwUXz5ZTmU10GJoJywtiasl09sG1zgYoXwO+lNDNa1sT9YawcebSNGne7Bp7G+M0yFjFzW1nB3FPzrEmt/aHGoB41azOOocKcXsCfmjvqZ8Ktoz93GWW71vHi8xHx1MxslUzOYQkl8R/cFb8LI/Y3b07GwwY1gksiYkm2nAuu4onxNZQ+gL9p4tydsk21SlCz9Ke+gJIoL6J7f+pa+9bYUwvNrrmKiE9QbiB1UeNmS5X9qtb7ilXnXt6C9nMvD4x13mvU2JZrna++ExHbZ/vxDWardUyueR6G8bspXBoRb2TlMev5QJdE7isVju215vl8M/AwSn/EYyjXKR+jtAZp6wnAiyPiZ5RRZgcDUbW50VprG6zxGa8iS//NVnM4N2p8XxdGxD9QWv5s28TqMofzbH8Etu3wvAsp+1SXOcjXqO8JKox/MFkeEc9n5eADe9Gtr0mtg9Jw39FFlOGvuwxg8oOI2DEzzwKIiEfSvs/AvTNzz6Ymi8z8U5Q+BK1FxEcpJ/rzWJlETdOiVmToQnj4AvjOwM8ys1V/zcz8xaw7TF1uAtS42zVwKKs2y9i6ZYxjgO9GxKBv0m60P9nX2h/GHtRjuDajw+vPVmtOr5cAxzdJE8A1rBzUp41ax4uzgM9EmUP3z3QfJv/twLnNDSAok9UfOuJzD1vD/2o05el6YrycmTVe11MGxWgtMz9Paeo2jlr7Vq2phYZtC2w171pzq3GzpdZ+NVvX91Vj7uUq571GjW251vnqOEqC8CtKv70uCQbUOybXPA/XaKX1QsrIz4ObRmfQ7RruQOA1EXETHY/tmXljRLwf+DIr++d2GXRzd+DBNP0GM/OKKINAdbFTx+cNq7UNHgi8NiJuZIzzZ8wcdHM9ykjUv25Zllrf18spN31upLTY+SLw723LEjOnyFqP0gUo28Zh5T51ITP7+bbqEzuX3iaoTaL0LuocTN4PvIfyZXy7WdbW2Ael5iLzT5ST6vAovjeu8YlzeyTwjxExmLx9K+BHTU3AqDvyTU3TpsE8SPdmaANr6WHA9jneaGn/Rdnxz6d8Nvdv/r57RLykxV2wX0TEo4HpKEN6HwT8qEN5qtydbMzVLKOVzHx3k2AMBtLYNzPbDk5UZX/ICoN6RMSZmfnYiLiemU1UupxExp7Tq9k/75uZD4rS347M/F2bGENqXcQcRkngxx0m/1jKBfQrKYnpm4C/GuWJmfmEMV53FbNOjOsDf0u3E+MvKTdsTmni7Qp8b3AxkfMMhFF5+4N655q5phYaaVTrgaH3NBhQ7Vd0G5ETxrzZUnO/qvi+asy9DHXOezDmttyodb76CKWPcNd+7wO15lmsdh6u0UoryzzkbQeXm8tGlGT7Xpn5liiD72zWJkDU6597U5b+o4PvqktrH+DWG8/jqrINZoVRmxvDcW6m9Elt3b+6xveVmX+kJKjjTh/2rqG/b6ZUBLUaL6FxHGWe7nGPF6vobYLa7CyvoIzg2+lgEmUEzGePm8k3cV6f7Ue8nCHLaHSHZeaj6DYgw7Cnjfl8KKMqfoGyk5xAac7xgo6xLqRc8F45RnkuA/bLZjS8KH1WXkPpZ/tpRu8z9xLKADubU+5Mdxl1cn3g6Mx8PjNHIuxqrGYZzUXe+ZnZeXS0yvvDFzPzybQftOBWmfnY5neNk8jYc3o1++fLyp+dE9Nqx4vG/1FnmPz/opw8NszMZVFG3fsUM/uDzqm5EbFa2X5kxVonxp80PwOnNL9H2p5qbn+19q2mPGNPLVTxwgzGvNlSa79qYtV6XzXmXoY65z0Yc1uufL76eWYum3+1eY19TK59Ho6I3YGvDt5fRGwcEbtl5mdbxPgyZfqn4WmpTszMv29ZnA9QjslPpEyfdD0jHpOH1Oqfm1Hm+dw4SvP3F9LyplhltbbBwfezLTMHnGuVwGe9qYVq9qceS5ZBRWtYnplHzL9ae4ump8e97lk4USb1PjYzu47OSER8PTMfX6EsX83MsUelioh/pSSnn65w0Tm25g7njpQbAGdly2l8huJ8DdiBMphHp2r+iDgvM3eYa9lc/1toTRPWnbNC5++I+AvKHa9Bh/0vUvq6jVxj3dxEOCSbKQ06lqPW/rAM2LvthfMccYYT73HLtJgx5/Rqaor+ROkrdesI0NlyRMOKx4tjKXPIfZ4xhsmPiHMz8yER8YPMfHCz7Ic5sx/76p57zBr+PZ0dRoaNMlfiIyi1K99vW4tRU9Ny5PKm+dXjKaNHHj+4AG0Rp9a+9U1KEvhN4FvZcc7QGHM6jSbG+sDbM7P1nH+z4lTZr5pYNd7XepRB555KOV58Efhw23NyjfNeLbXOVxHxX5SuVacy8z21vhlZ6Zhc8zw81zXGrcfEEWOssn7bGM1zOh+Th2KcP7u13FzLRoz1FIb2h8z8ctsYtdTaBqMMPvYKSh/z8yjXut9pe26OWVMLAZ1G5a/5fXU1R4uhga7Nn99N+Y6WMfO7WuunmanR2fpbUdp8zz4xtv3wftBclJ88K07bg/argDsDN0fEDXRvTlbL5pRmdouBx0UZ4KFLrdihFcry44g4Ejixebwn8L8RcUdK/4GRRBly++UMzaEFnS4aLqNsP8uY+Z13uZO7ffOzuPnZlTL1TJtteTPgooj43qzytHlftfaHG4ALmrvJw3Fa1Ro2tSs/jKG5BLtoLqSfzsrv/KnRYr7QIYNka7jGfZqWE41T73jx0+ZnA1YOk9/lxtafm89o0ITrHozYHCczuzRNXq2oN/puLZ8CHhYR21D6dC+j9O15ess4tfatfShN558N/GeU/lPfzMyRp6WKOiPgk5krIqLG3f0q+1XF93ULpYZo3FqiQ8d8fk2XUed8tSHlQnN49NPWU3xUPCZfRr3z8FwD+bW9Dr4lZs59+9dM+Jg8pFr/3Mz8ckR8l5Xzjm7S5QZSJVW2QUpy+nBK5csTImI7SmuQtuaaWqjLqPw1+1N3Url1DZS+y1CS/4G1f5oZ6nS2HmxAw3PIdfnwNqEMVjH8vNY7zAJsHJ1FvbmmajUXeAGlqdUrKReuZwL/j5KctukH91nKheapjNcm/ormZz1GbGq1BidQ3suFY5Spy4F1tlr7w383PzXUSLzHnS908JrjDkYzUOV4AfxPZp48vCAi2kwlNXAE8BngnhHxVkpTu39pEyAiNgX+g/Gnjagy+m5Ft2QZwGV34L2Z+b6IaNu3GyrtW5l5aUT8Cbip+XkCpZ9uGzVGwB8Y+2ZLxf2qyvuKiGdSuo78NeU6qNON4orN5Gqodb76cGbOGHAxyqjLbVU5JlP3PHx2U+PzAcq++XLaJwhvoMx9O/juH0cZEbqtsY/J1BvF98WU49afKN/VoI932xuztdTaBm/IzBsigihzXV8cEfftEKfWqPxjf1/NMe8Vs5qYH9axJdNDKDdDp4Ezs/2YJlC65V06K26V7abXCWpW6GydlQb4qF2L0BO15pqa3WxgA+AOwB/anPAz80+UNvpzjRraps3/DTXaxA/6HVTy68zs0sfpVjUuhiruD13miVydGp/zWPOF1lbxeHEIJTGYb9l85TkhIs5h5VQqu2Vm24HDjqXOtBHVRpgsyuAAACAASURBVN+t5M9RRjLfB9i5WdZ6Gota+1ZE/ITShOzjlM/25YOErKVxR8AfqHWzpZYa7+u9wLMYc/CxGue9Wiqer95HGaxwvmXzqXJMrnwefjnwRspxaxEdxqfIzC80F/aDrlH/nB26Ro17TI66/XP/H3C/Lu9jgdTaBi+PMuDcZ4EvR8Q1dJsPeuxR+St+Xw8c7n6SmddERKvm5U153kSZs3twHD82Ik7OzLYjAn+SVb+Xk6nQr7bXCaoWXK25plapGY6I3Sh9zG4Lh0eZ1+tLVG4TP4Y3R8SHgdMZs1/PWujelCaM/zdGjFrzhfZCROxEqS3aPCKGb7bcjTKwUGuZeTFw8RjFqjVtRI0RSwdN4vZn1ab8be8k70sZWO2tmfnTpovAx1rGqOkIyl3tvSjNp74REWdkM3f2iGpMpwH07uZsrff1CyoMPlbrvBdlsJQjgU0z8/4R8UBglw4Xi51FxKMorQDuETOn1bgbpRtQW707JmfmH4CDuzw3Vp0PepDobNU0+W19fTHOMblpfn+PiNggx++f+xPKdF+3qdrbYGbu3vx5aHPM2IgyMGhbwwPFDWo+Wx0XK35f60XEX2YZTZqI2IRuudxelJZMNzRx3k4ZgHOkY07TXPp+wEYxcyDFuzE0INU4TFDXbbXmmlpFZn42IjqdCCp4AGWI8icys+ny2G3ix7AvsB3lDvtYzanXQlsDz2/68pxDGRzmm5l5XosYteYL7YsrgLMp/ZSHm6BdD4zcF7GyWtNGjDVi6aznfRP4Ct3mOQaguUF3ENzaXOqumfn2rvHGlZmHU26y3YVy3DiUMsjHyBdoWWE6jblEM7DLuHG6qvi+Xguc1jTT7Dz42Bzl63reO4rS9P1DTZzzI+LjdJjfcAwbAHehXBcO74u/o+Xou43eHJNj5tRWqxixO8lCzwfdxWXU6Z97CPDtpg/q8P5QYyT6Nmpvg0TEY4FtM/OY/7+9Nw+zraqu9t8LKAgIiBp7bBDhhwoRUVARmw9sQDCoDENECdghRjDYGxGCIIJihz9pBaWNA4wiCBhEQFGRRkBQ8TNCVFCjJApII939/phrU6eae+/Ze6+qc27VfJ/nPlX71D2rVp2zz95rzjXnGCWp+Rha7n66nrXQf9H//TqUeK9OK8c7Agd2nMsqRBk+wMpMvi8vi/WBVxAVLdsNPH4rkTjuTQaoc0TJciyRETWj1/I7m2pFsQLhD9dWEfFJU2vZO7ID8KQKWcWabGz76X0GkLRXWbwu9bEO467sbl68VbD94TKPBxEXtvcQJXhtMqZV/EIlnWf7/yzrsdnG9lXAVZJOdlG+LMHT45rM6QjobRsBVUv2VrXd1dvzfkrAsz1xP7wS+KOkC23vvdQnzhKSDiV2UFcHfkAISn23w1ArEKXCKwFPkfQUt/dHnMqiLk/ShL3HzeV4LeCFbmHvMUCNv+tAom1kFSbEx1pT475XWNX2JZrsk92pUqIrpYXkQklfbNqrSnC5urvZA9XycK7BJ5b9X5ZOrRL+ytTqzz2SEK2r7mXZhtrnYKmk25QIpo4jNghOJCwV24zzFKIM+glMrtZpm5To/X7ZPl7SZURCZBHwqjZVkJIOI65RfyW0P84tx1sTui/DzuN04HRJz7H9gzZ/w7DM+wC11H1vy/QTq61Nw0eBQzy5Mfldtodtar+cCYPxdYA/le/XAn4N1BKRaEM1rykmZ1DuIbIzr2w5xhclPQa4lGKzYPvqDnO5inhd/9DhufdTGr0/Q9xk7yMWi//cMYi+uEI59S5lPoP84wyPLRFJxw6WQJZdmtOJPpihKRfs9zAhMgJ0umAj6UPEDWN1QvDk3bRfkPfyC5W0CrAq8LDy2W4W4msAj+4wXi1BoXMVthojD55s/0jSC+hpG1GRMyVtY/usnuOsafsWhbrwcbb3ldTap1ohXnWO7VvLOb0JcECH0r+LiXvNf7edw8BcDiZU0KcK4PUNULsKo+1r+6vNge0/l8VjqwC14t+1tu2XLPu/LZMa9z2AmxR2R011wmto6a1a8X51kKTdiaqEy4kSvk/a/njLcap4ONco5XdlMSuF7ciGTPbWPL7m7xiGism+e0aVkFsCtc7BHYg2iR8B2P6twsu0LacCRwDH0K9ap/P7JWmNcp9aG/g9oVHQ/KyN4vJl5evlhEhXwwUdp7aDpJ8QAlvnABsD77Tdu01m3geo1FOSe7ntDzYHjsbkbRhSdc1FxVDSEcDXm0VV6TXbqse8+nBtKSPq6zW1IuFl+ak+k7G9paQHEuVbLwS+IWl120vdfZ6BRxB/26X086Y7mVD7a/oY/p4wVd6s5TgQOyK7SLqeluXUCgGXfwCeWEpDGh5MiJe04UZJh9t+WwnEvkE3q4Xmgn00PS7YhVcRi7tvABcSkvB3Lv0p0/gdcIGkrn6hbyXUox9NXLibAPUW4hxoyxepIyhUJXiqgerZRtRiL+CDkhq1264lhCtJehQgJt6vLuxj+9RSUvZSYtfmcFpeLzxFtbkjfwesX6MyQqFWeYdDqOn4kjA5u2Vyooa9B9T7u76lnv2Rte57hbcTthUbSLqRKEHcueUYte5XG5ZrzuuAs4D3EdfEtsFB32tyQ5VSfgBJ6xF9zFODy6FVR0ti5YVljLMIt4mLgDkPUCtyvqS3MH0tOCqbmVrn4F22F0tqEj9dlHchAvjDOz63FicTJbXNZldDK8Vl1xW4BHiJ7feWKpkbiJLj86mg47AQAtRa6p4rDpZClnLElTuM8yzbuzcHts+W9JEK8+tCFa8pR/P39kCvG3VZ2D2//FsLOJNupW379pnHAItsnzBwfKKkf+o41st6zONHxM3+YUzug7kVaBWs2N5H0sElUfJM4GO2v9JhTtUu2A6z8gcTQfzWwNGS/tv2Fi2GmckvtM0cPqPwsPyg7Rqfx1qCQrWCpxrUSvZVwfUsu/YHvknI7F9adqK6CHY17++2wOG2T5e0X6U5tuU6opytRun+d4Dnl4TWeUQG/rWEh9+w1LD3gHp/19uB9yo8Zjv1R9a675WxrgO2KgvoFWzfuqznzECt+9UDJD2ASAZ8zvbdzQK/Jb2uyQNUKeUvHEesDz5F2DftSvuy9dcQu0RX2N61VMscU2l+o+IfytdBwbFR2szUOgct6UhgLUlvJsSOhk7Ia6I17wxJexA7jiMJ4G2/onwdRbXl0mgU77cBTrH9v1NaFTqzEALUWkpyJwLnSTqO+ODuRogMteWmUv51YhlnZ9rvglXBddUZv6/+JvUXEoufg4Cz3LGHtGI5z/kKwYt/I96r1xK7umuX3zP0xcn9LJNOKUHcL7v+bVN6pS4hpPYvARZLelXbXXMqXrBLudTzgRcQ/SK/oWViokapU1lwbkP4I/allqBQreCpBlWSfbVKESUtIoKkJ9r+iKTHAY+yfUmbccqO5akDx9cBr24zRuHGshjaCjhY0srMvHM4a2iiv+h24EpJU1XDuwh9LLJ9u6Q3AofZPkTtfWJ72XvU/rsqJjdq3PdqtBBBvfvVkUSp8lXAdxTida37/yqWn9Yq5Qd4kO3zJC0q9+T9JH2XdkntO2zfJ+keSWsQrUSjCuSqMIZBT61z8BOSti7PXR/4sO1zWwwx2JoH0dbUMJIAXmOikzHAGZKuJUp89ygl+W0r4GZkIQSoVZTkyk35aiY8qz5i+5sd5rMTcTH8KhO9Mzt1GGfcqGFS/1CiF3FLYE9J9wE/sL1PnSm25rXl61unPL4bc3txeqCkXYDnTAk0gaFLsrebcnwFkfnajm5qwruUrzUu2AcTn4PPApe2LB2szX9IejXw7z17p95FHUGhWsFTDWol+2qVIn6eCHBfTCQV/lLGfVabQRT9x28kJPMHy/7a2tWIqJT4hKPH8lFM/nwMO58TbL9+WY8tgcH+olr6AosU9g+vI14naLl2cA97j8Js/F01aO57TTDWlNu17cXv1UJUqHK/cniID1pb/UrSKAWCmlL+zrvdA9xZ1oK/KLvLNwJ/03KMyxQiX0cT5+NfiGTvnKPJFmQNNwOXOURslktqnIOlBP+btrcC2gSlg/OoGrj3eb9UWSejFrbfr9AGuKUk+W+jWx/+NBZCgFpNSc722cDZXZ9fPjAfsL1Xn3mMI66gcFcWddcBjyNsFZ7LRPnAnDNGWcXdicXhVDlvGDK4rLlbXm7wO9v+Xo3xbG9bY5xK7A2sBtwj6U66J7QuVw9BIUnvLUmxZudo6vhzLf8P9WwjapUiblYqC66A+xf1XUoJTyC8CF9KJNheB/yswzhHDgaRtn8n6RBit7ANTx08KPeNoUzPZ6G/CKI3+wPAV23/pOyAnz/MEyV92vY7tQSbDw+pCzBLf1cNzmTyDsti4BZJf+t2Nlm9W4j63q8k7Wz7RE32nxxkznvNS5XEU23/utKQ7yQW+XsSSa0XAW9oM4DtPcq3R0g6B1jD9kh0AYiE2gZMJDFfTQiIvVHSi2y/c0Tz6kTNc7AES7dLWtNFPXwM6PN+VdXJUE+hS0kvtv3twU2TKaW9vS0UF0KA2lfd8yLbW0i6lRkakzv0rAy10FjeWMIF5Wbg8mFv1JJ+CfycEBw4Ati1a5lvDUoPxNuIHV0IlbMj53qHz/ZFwEWSLnN7FVhgiZm7wd8xdMBTyps+QSR+elMqE6Z+Pm8mdk0OsD1nJfC1Sv8kXUWU/X3ZdhtvsYYmQLpsqf9rbqmV7KtVinh3Cd6aMuqH06039sm2d5T0SttfUgjHdamO6RxYlv//AeCDwIMkNeVsiwgBqKM6zKcKnrB+WK0cX8fwnoBNIqK3zceY8kyiLeHrxHu1LaFC/1ZJp9o+ZMhxqrQQqZ+6bCMgU6v8uTcOgZuv0uJztAyeYPtSYtdzV7hfffuHww4wWE5p+7+mPjbHPBl4se17yjwOJxJiWxNaAcsbtc/BO4GrFVYqgyX4o0jwQo/3yxP+2O+wfViFufQVunwBYU00ddMEulXlTWMhBKi9lORchFoq9qxcoVBiPZXJH5jeb2ZbVM8KA+ImvSkhpAITN+rdW9yo13MoRfZCoZQ7U7a+bfnp4cQO7ufL8evLY2/qNcHunCBpTyYC5guBI4YMmLsIkiyNWqWwEFUJ9zIhm/735esthBruTBfASaiiJVApn1mPyYu8tjYW2xOBl0up+pdjmOF2AmyfUQKcp9luXSY6S1SxjaBe6fxniVaJR0g6kCihblMS2dB8fv5cFvi/J5SKh2KGwLLJbLcKLG0fRNgrHGT7A8t8whxRynu/QNhArSNpY+CtAztJS8T25eVrVZuPGpTP+eOYvHvQ1hLoocAmtv9SxtwXOI24Rl8ODBWg1mghUk91WdtHlmvOLa6gTFx2aA4HHmH7aZI2Ara3fUDLoS6W9KwSWPblAwy0TCzlsWmMaYnlY4igrtkhXI1Yz91bSqKXiqRNlvbzDp+HXtQ+BwlngK7WWLNBr/cLwPZhkp7LdNultirSvYQuHY4CKxCK7u46ztJYCAFqFSU5hUfZDbb/KumFwEbA8S6iBi1YmxBFGtxGr5Jt6MAXqWOFAXVu1I8uJY3PI16Ti4C9bN/Qci6bDny/CiF73daqBkJxeeOB42+XnbFR8Xk6BsxTS+QUqrmLm/erA00p7L2S7qBfX9DzbA8aZ18t6Xu2nydpWKuFKn2NCjuXvYgS8yuBzYlgt1VPmUOA4xDgEIW1wT5Er+2KLcYYt4qLKrYRtUrnbZ8k6XImFvV/Z7tLae5RZcG5D7ETtjrw4RbzqBpY2v6Awg96aulVXw/TrnyaKH/+epnHVZK2XPpTgiVUR0ALm60ljNvr+qVQzv9H4JcD8+vSO7oOkYhouBt4vO07hl1wNrhnCxEV1GVdUZmY2Jl5DyF4g+0fl+qEtgHqi4hE938RSf3W547Czm8b4DFTqonWICzOhmGwxHIwcOtqRVaDQwjRsAuI12VL4KOl2uFbQzy/cQVYhVg3XVXG2YjYVW6jpF+FyufgacCdtu+F+6taurhvUOZ0/+aA7TOW9v+XQN/3C0knAOsS65Nm53MxQyaiVFGZuFTT/ROQAWoXXE9J7ivAppKeTARwXycWxdu0nE9N5dy+1LLCgDo36uOI13THcrxzeWzrNhOZoST005IuosWis3CvpHWbEs2yS9fX87MPvQPmskN0AhGwL5L0R+ANtn/SZpyKFQUAq0vazPYPyxyfTQQJMPzioVZf416EyM7Ftl8kaQMmBFBaIekJhHDOa4nz5r0dhhmbigvqJftqls4/DLjd9nGSHi7pibavbzOA7WYRfyE9hM9KYDm4iLnA9pltx5H0MSLB8lMmL0CWGaBqCb2eA3Ns6wXdPO83mtxfNOx18BVdft+SkPR0YiE2eP3axfY1bYcC1q3QQnIyscPXCJxsB5xSFpw/HXoy0cd1MCHYs4huCb9a6rJVlIkJe5hLppw3w17PB3l5h+dM5bdEu8T2TK4muhX452EGmIUSy97Y/oKks4BnE+fMB23/tvx4mZU3Ltohkv4NeIvtq8vx04B3z86sh6LWOXgeoareJLIeRJTUPneJz5gBSQcRr/FJ5aE9JT23bUKy7/tV2JTwie1ayVRbmfhcSe9m+nvV24Jn3geoks5n5nLPtpnS+0oAtwPw6bLN3lZqv6ZiZA1qWWFAnRv1w20fN3D8RUmtm/ynlK2sQHyguwRU7yH65a4jPsyPp/StjIgaAfNRwN62zy9jvJDIdLe9YFex+Ci8CThW0urE63wL8KZy7hw05Bi1+hrvtH2nJBSiJddKWr/VXwNI+iGx230qsGOXUuPC2FRcVEz2VSmdL1UamxJCVMeVMU8kKjCGef6ShDiA9jvDMyxi9pL0vA67qjsA67sI5rRkNno9f1NKyhYrRKj2ZEgRKQ/YaymsItaz/S2FCFCX9ceRTL9+HUXL6xdwDSE694cOc7ifcu07i9hpWgTsbrvpG2/jE3sIsF3HCoCGWuqyNRT5ISz11mViffEaogqjFbZ/pfBIX69JRDGRwBx2jKuAq8oO7krAOrZ/3nYuhWMVVoHr2H5LqZBZv0syqiuSNij3pmat85vy9ZGSHtkhkNugCU4BbF8j6W+rTLYbtc7BVQarLGz/RdKqHeazLfC3Li1okr5EOCEMdW2v/H5dAzySDp8lmBXxzyZ2GbQNq+JyMe8DVCZngVYhVLO6ZPHulrQTYa/R9MR1UZitpRhZgypWGFDtRn1TKek8pRzvRDeP2EMHvr+H8NNq7Rzs8Etbjwkl1ms7LhprUSNgXq1Z3AHYvqAEgm2pYvFR5nAp8HRJaxI7oYNl88OWjtTqa7yhLPK+RmQG/0Rk39uyi+1rOzxvEmNWcVGLWqXzOwDPoJTb2f5tKf0club/DmaTGXisLb0WMQNcR9xbWl9rPDu9nrsT/d2PAW6gpYcpgKQ3A28hEi7rEiX0RxDl2W2odf06iKhOuIbJ5W2td5gdfbZ9e/z/u09wWhKGB5VrZy91WVdQ5C+8nUgebCDpRqL6ok3QDvRPRE3hZUQS54HAE0sQtn/L9/1Y4v1ugqgbiETknAWoRIvNW5i81mnoEsj9TNIxxOu6mKheG9W6tOY5eJukTZoAUNKmhF9nF9YCmkT3mi2fW/P9ehjwU0mX0OPapRAHO8f2rSXhsgnR+95q420WAt77mfcBarl5DPI9SV1u4rsSN+oDbV8v6YnEh7kttRQje+OeVhgzjQdcLuktA8FpG3YDPkf0HiwGvs9EdqbNPKpc3Mpu9x5E0L0Y+K6kI2xXMSFuS6WA+TpJ+zChrrkzsXBoSy2LDyStTCSOngCs1JSE2d5/KU+bRK2LpO2mh3W/Un2xJnDOsM9XkckHtlH4GE4dv+2u3MOBNzNdEGEUFRe1qFU6f5dD5bPZoWkVqDQ7wiWQ3KtJjCj6UWdaSAxDn0VMw+1En9J5TF6ADK08Wa4TBzFd0bV1Vtv2TXQILKbwdmJ3+YdlzF9Iaus/CfWuX18iSmqvppvyc20uk/RlIjE2+J4PVSlRPgdfo6jduqjLdmHq9Xjgdwx9PS78yvZW5XO5gu1bO06pbyJqkP2I8/CCMtaVilaMNqxr+7VlwwJHG9PUBNesYvst5WutQG5Xou2isUD8DlHVMhIqnoN7AadK+i2xhns0E8nsNjQJrfOZ6B0dOvFY+f3ar8IYAPvYPrVUJ7yUSNwcQXs/clRHtGka8z5A1URDMES55zOJ7fFW2P4pRVq/LGAebPtjHabUSzGyJupvhbEkdqeDLYJD4bRTj9QgFS9uxxM9Kk2/yU7EwmjHJT5jFhnIeP24ZLz2lXRAy/KQ3Yieymbh8x26lS3XsvgAOJ1iSUSHXaMG9bNYmEbH3ailyeR32ZU7HfguIaAwyv7nmtQqnbekI4G1yg7dbrQUhSlsNLhrX5Itz+gwTq9FzABfL//6cBywL5HsexHx+nZaRGvC+mQSLZMkf7V9V5N8krTSTGMOQa3r1022l2q9NcesQSQmXjLwWNtS/lpqt1Wux8D1ZSf3y4QdRVd6JaKmcI/tm6XWBVWT5lNK1Jv5rEu/16kzMyXRCWX/Vkn00tZyBHBWj9LnmtQ6B59IJDfWIRIdm9PyulOSDxeV5z6LuI6+z/bv206mxvtVsUqmWU9sCxxu+3RJ+7UdRD1Fm5bGvA9QmdwQfA+RbX1j20EUqlvbE6/ZlcAfJV1oe6l9TDPQSzGyMr2sMJZCq4WQQrl3acIebT2ral3c1p9Sinh+x1LEWsyU8TqcITJekk6w/XpCEKmGB1hj8fE36mfxAfBY2y/rMxn1tFiohe0jy7ffsv29KXPsUpK2qu339Z9ZfyQ9lkjWbEEkIzqpbNcqnbf9CUlbEz3L6wMftn1u23GAFSQ9xPaf4P6kZut7o+1Tyn2i1yKmVNY8iH59cg8qr/MiRx/ofpK+SwStbRksXVyFWOi1LXu/UFJjxbM1sUhrpYJZEmIfrHT9ulzRM/x1Ju9YzqmtxsDvrVHK/yLCf/VXdFS7LfS+HhfWJ9qh3g58QdKZwL85fL3bMFMi6uiOc7pG0j8AK5Zr0J5EpVYb9iWqah4n6SSi1PgfO86nL1WS6Apxt4/Tr/S5JrXOwWbNtBYhtnkoQ66ZGprqBNvPpH/isPf7pdCKOQz4/4j3a0XgNrd3ULixfK62Ag4uGzsrtBwD+os2LZF5H6BWrI9e0/YtChuK4xweQF36O6ooRtbAFawwGjRZPXO7GR5bGl3KgZdGrYvbFZI2t30xgKTNgO8t4zmzSZ+M1zMVIiW7STqeKUkEt1Rccz2LDwjFvqd7QKShA70tFipzGNHTsazHlsWZkraxfVadafWiisp2rdJ5SQeX4P3cGR5rw6HEOXhamY+AA1uO0fAsJlR876NlEAYgaTv698ndqfCo+4VCzfpGQiG2Nba/MmV+pzCkJcIA7ycSw1cTfeJn0c0CpZbtUrNDvvnAY11696qgOuKJNdRuoc71GNt3EBoCLkn5zxDrnlbri4qJKIB3ELZ6fyWuZd8kNBTazOdcST8izp1FRJLupo7z6UutJPq+9C99rkmVc5DJa6Yjuu4SUq86ocb79TlC5f1UIjh8A+Hb3hZRerJt/1nSoxheSXiQXqJNS2PeB6gzLIYuIhb3bfsIVypvoJjwDe0yn1rlp1VQHSsMCBueTQAGdlROo/TELA1P8eisQK2L22bAGyQ1O8rrEGICV9PDw68HfTJeRxBZ3ycRO8uDAerQIkJTSub/wISgFZLWbhvoFrYA/lHS9cTCoUvmv5bFQi8kPYcQz3i4JqvErkGLhZmkW5mo/Pigwqrpbvr5zfaliso29UrntwamBqMvn+GxpWL7eEmXEcHJIuBVpaWjFQp7mGfR04qAmfvk2iZa3wmsSuwQfYTYXdul5RhLYj3iWjg0DuGoo+m+89VQxXapYu9eLXqLJ3pAMbkLmvCsXQnYtZTgd70eN2O+gFhbvBy4lA5ihRBBIQOJqB5sWP6tVP69kqgka/u3PYa4nq8EbClpVNZftZLoNUqfezML52CtXcJa1QlV3i/b/ylpRYe/63GShq4CkLSG7VuIRNgF5bG1idd56M0iTViaPZgKok0zMe8DVOothvYnsm0X2b5UIezxiw7zqVV+2htVsMJQ+EQ+FVhT4eXWsAYDmeBljFHbu69GwAORXRonOme8Sr/VZyUdbvttPeYwWDK/DvCn8v1awK+Jno+21Mj817JY6MsDibL9lZjch3oLLRSyXddntha1VLZ7ZZElvY1IOj5poIplEfG6d6pwKAFp66B0CttQR8V3psViq/KpgUz/X+hpjTUlWbKY0E0YKgkwsOBc0jzbXpOr2S5J2pbpO5YjSRQzHuKJtT1rryfaoQy8x/Zty3jKksap4RHbcBLh7HANHTUTJB1LBLQ/GRhjTq2/Bj5XD2Aiib6Y6Ofvch2rUfpcg6rnIPV2CXutUSq/X7crBCmvlHQIsXPZpi/7ZOJ1nuqHCu0cD2bD0mwSCyFArVICYftUIpBrjq8jdkLbUqv8tAY1rDDWJ072tZiw34FICrx5yDFqn+hVSp36ZqRrY/t2Bm6Ctn9Hy7KKnsHp/SXzCkGFrzelp5JeTmQpu4z5K0kbA88vD33X4Vs3FKposdAXh4DBhZK+WOP8Ufguf9v2zeV4LeCFtr/Wd+wOVFHZpn8W+WTgbEKU6P0Dj9/acQe/JjVUfHsvFiWdSyQdB5WJ/832S9tOpmeypFlwNrY0jfru6whRoLYc4wq93eX6tSqxM3IMkTwaRUKrYeTiic31StHj9hMX1V2FYu6GQNvr2cZlp6YvNTxiG/5ou3XZ/RQ2t71hhbn0oXYgN1Pp8wGVf8cyqX0O1lgzFQ5waHjcj0Ic6PVL+P9Tqfl+vZ7Yvf8n4J+Bx9EiFrH9ivK1V/tjWetQqnt+11SlKvQTHtFn7IaFEKBW2VKv1CMC9cpPO6OKVhi2TwdOl/Qc2z/oMh8PqJKpvzjI2AWW85Rn2d69ObB93GWdOAAAIABJREFUtqRWvTwNkvYikhnNjeRESUfZPmwpT7sfV7RYqMjtkj7O9OtF2x63fW1/deD5f1YIQs15gOpKKtv0LJ0vwfrNxA4uAAprq9bK4ZWppeI7uFg8hQ59csDDPF2ZuFUPqiZM5WfEQwgKDSw4n2d7MJB8v8J7u+2OZa3e7ufa3kjSj23/q6RDmcMdsBkYJ/HEw5n8et42w2PDsEapIngeE+1VrUXV6OkRO4V9FX6fUy2c2rz3P5C0YZc2gFrUXuOUQO5fJH206053ZWqdg7V46uCBQrBt6H74mu/XwFh3EIrmnZB0nu3/s6zHhuBUJjyBIVoFTyXaXXoxbwPUWSiB6NUjMhv9HT2obYUBsIOknxAfmnMIwZp3lkB4KFRHHCSZG25SWN0MGnt3KfeESPxs1twYJR0M/ICJsvxhqCViUIuTCFXsVxC2S7sAf+wwzkz9MiO5bkt6CrFIeITtp0naCNjedttM+2xUkHSytqqFKloRNItFemgdAPdJWqckFVAIpLW9tg96wQ4+tyn1bZNsWU3SFi4KrgrfvKHL0lSpt3uAO8rX2yU9mrh2zZrh/LLwGIknAos8oMjp6O3vcs3pJao20DLUyyN2CrsCGxDrwq7luV8igtTfM7o1XFXK5/EYIjGyTqloeqvtPUY0pVrnYC8kfQBo1MebaoBFwF2M6H4j6RVEwvLxxFqgVcl72WxbFXhYSYo1Jb5rED6xbVnJ9l3NgcNO7IEdxpk+cI1BxpTaJRB9e0Rqz6czrm+FAfAS2+8tJYk3EDel84kAZlj2Y7yU5JIlsxOh/Nfs7n2HgR2tlixissfnvbT3bKwlYlCLh9r+gqS9Bsp+u/iXXSbpk8D/Tyyk3kH0joyCo4n+nSMBHH68J9OyFGyWKhw6eXzWwhWtCEoi4N1MF9JrExD+C3DRwDm3JfCWNvNwERIqVS1TvfsObzMWkYQ6VtKaZYybaVceXqW3e4AzS7n8x4EflTnNuer3lGB7Gm2qmSpynaQ9mXiP9wBa61PQX1RtsGWor0dsw8a2n97heYMcS5RZXk137+9x41PE5svXAWxfJWnLpT9lVql1DvbC9kHAQZIOcnuxu9ni08CrgKvdzdrlrYSI3qOJa1/DLcQ6oy1/lLS97a8DSHolUEXVej4HqLdWHq9Xj8iYlp3WKpeCyEhCCIWcYvt/1V4RbiyU5JJlU3r99lIo5t5n+y89hjsO+KGkrxKBxiuBL7Qco5bFQi2a68XvFGIsvwUe22GcdxBlf18ux/9Bd7/Zvqxq+5Ipn897RjSXvtZWs0GtXfxTCdXtY5icuBka2+eUEt3GCuOf3d0K40vE4uWz5XgnQnxw6Au17cuBjcv1YlHTU93i+VV7u203JdNfUfhzrtJ2TpVogu31iZ33JrmxHZH0GwW7E+/1h4hA8DxaJjcKvUTVXLxhS3l4rUT6xRXKc3/dLMbnE7Z/M+Xa3unaU4la52AtLpG0pitoQUh6JLERsxi4tEOVzW+AazoGp9j+DPAZSe8Yto1qGewOnCTpc+X4BobvzV0q8zlAnUmhqqGNUlXDOPWI9GIWyqUAzpB0LVE6tYekhwNtrXzGRUkuWQaSnk4sUtcuxzcRolvXtB3L9iclXUDs0CwCdrV9Rcsxxi0BdEDZLXoXkfRZgxA0aEUpe37/Mv/j3HCTpHUp5Z6SXsMseJ+1oLO11SxRaxf/HtttdyiBUFW3fe1A/+hvy9d1SsnvMvtGZ6C30KDCl/ijwKNtv1zShsBzbLdNRB0jqbP4kyYrzU/92Zxbhdj+1/K7/wPYxBOiMPsxIMo4x3P6A+Gz2JdBUTUI7Y8uomo1E+lbALuon8L/taVy5Az6lxyPC78pZb6LS3nmnrS0OapJxXOwFlW0ICS9iYgbvk2ce4dJ2t/2sS2GeS9wVqmOGTz/hqq2kPRi298mLHimXQ/bnse2fwlsLml1IvlYbXNw3gao7qlQNcN449Qj0pfa5VLYfn/pHbzFYah+G7ET1oYa4iDJ3HAksLft8wEkvZDoyXju0p40EyXo+YntH5Vxni/peg+IvCxv2D6zfHszEbjMB95OvMcbSLoRuJ7oK5tTVMHaapaotYt/hqQ9iPL5wQXIMArFexM7DYfO8LO2faMNNYQGv0hUSjR9tf+XqApoG6D2FX/abik/m1OrkCmsQ/S1NdzFHKv41sY9RdVmKZFeo//9QcTnskbJ8biwO/AZwt/1RmLt9falPmNhUUsL4j3AM2z/D4CkhxKbMG0C1AMJ+7BViLV8W15ABMgzXQs7n8c9q+hmZN4GqIOULOt6TFbTHKp8Zkx7RHpRu1wKQNIDiG39LUuZyIVEmVqbedUQB0nmhtWa4BTA9gWS2nhxDfIVYFNJTybKGs8gxDWmKUwvL5QKgjczvY+wyw7CWOCw1tqqvM8r1MyUtqSGtVV1Ku7i71K+Dvr1DVX1Y/st5WvvpIjqCg0+zLaL6Ai275HUpYSwl/hTUzY6hpxAlBF+lfh7diBKq5dbJD2W2OnsquI7G4n0GuXh43oOdaaU/79u1PMYY2ppQdzA5PbDW4mS3Tasbfsly/5vM2N7X0krAGfbdtdx5oJ5H6CWLfW9iP6vK4menB8wfCa5jwfcuHO76lhhQDSzPwD4fDl+fXnsTcMOIGlTQjHtCUxe1C+36njzmOsk7cOEr+HOxI5aF+4rC9ZXAZ+xfZikViW+Y8jphJjMtxhtL081JK1M+K09AVip6Vey3dYqpBeuYG01ztSo/lEoNU4VNjrCxatuSGoK+91Wdgua8vDNieqCtvQWfyq/f6bE883A5bav7DCvXtg+UNLZTHhBt25zGEN6qfjORiI9mRlJTyJ2UDcnPqM/IPrW51yYqMxnRdvjdN+spQVxI6G3cTrxOr+SSEztDUNveH1L0kts/0eH30/5PfdJ+idgVgJUSSvb/uuy/+fSmfcBKhGcPgu42PaLSnnY0N5BTY/IPKWWFQaEL+Zgn9K32/Yplfm8h/mljjdf2Y34HP070UvxHULCvwt3S9oJeAMTO2IPWMr/Xx5Y1fb7+g4ysHB4DvGZGOXC4XTKIp6B0tMR0tvaalwpQnwbMjlxeHyLIY4nsvONCMZORDJpxyU+YwqVg4K9Ce2GdRX+pw+nwy5YRfGnTcu/M8rxtsClwO6STrV9SIcxe1H6g7v0CFdhFqrF+qr4Dj5v2i55x0R6MjMnE7uDO5TjvyfarDYb0Xz+U9JpwHE9Ba2q0GhBSFq9ZynrL8u/htPL1zYbYW8H3ivpLibEGIe2mRngXEnvJmKA+71vh2wluR9Jxw5WhpVe1NOBtn6q01gIAeqdtu+U1ET110pav+0gCsPpvaaIMxy6PJfsUc8KA+BeSeuWhulmYd02A/bH+aiONx+x/SdCSKEGuxIJkgNtXy/pibSzJxpHzpS0je2zeo4zTguHx9qeDQ/TrtSwtho7ivjGC4kA9Syit/UiIugclt7CRjUp/eUvIMqzFwE/t333Mp42DYXf7MuAJ9neX9I6kp5t+5KWQz2UECX6Sxl3X0Jga0siATPnAeoYULtarJeK7wDvHvh+FaKKY2Tq4fOURbZPGDg+seywjYqNiHvdMaUc9VhCDO2WpT9tdlAln9gBUbTVStDbGtu1PqdN7DLYa9xFQPZGSYfbfluJi75BWNL1ZiEEqDcoJKG/RmQM/sSEsmEbNppBnOEZtSY5ImpZYUDsfJ4vqdnZeQLtd9T2lXQMISk+X9Tx5iU1y7FLhnTPMu4mZSfhY3VmOjL2Aj4o6a/E56yVmfYA47Rw+L6kp9u+ekS/fyo1rK3GkdcQu8FX2N5VoYDb1qOzhrBRbZ7NxPViE4VqbpugG6KF5D6iRWd/Ypf4K0SVVBumihLdDTze9h3lM7vgmIVqsUEV38WEGEzrhL7DomiQ7/VIpPdG0keBQ6ZsVrzL9qjsv2pwvqT3A/9GvFevBb4haW1ov6vWl6JvcDRwtMKP9RTgU2VX9SO2/3Mu50Mln9gi/PUFega6krYnkmkAFwyIMg5NjVaSMs4+kg6WdAShoP8x21+pMfa8D1BtNzsP+0k6H1iTKAdrywqSHlJ2jigf3OX99atihVH4HqHs2mzrH0mUI7ZhV2ADYuHZlPgu7+p485XZKsc+hm72AWND2eV5aiPk0pNxWjhsAfyj+lk01KSGtdU4ckfpEbpH4Rv6B9pntTdjQtgIIiD7WSN8NNfvmaQTgHUJHYimsmYx7XaFATazvUnTo14SxV2ULE8mPDGbErvtgFMUAmAjLykcJaV/+Y1M16YYOriUtCLwatudVXwHxlp74HAFYhH8yL7j9uDltj/YHJRzcBtG509dg9eWr2+d8vhudNtV60U5f7Yl1oRPIFTJTyJ6tM8CnjKX84FqPrGfpmegK+ljRELupPLQXpK2sN3Kjq6vToEmK+hfQvToXkJYFb2qxsbS8h5gtaKUsXblUGIH4TTizRQh97zc4rpWGMcT6nqNLUzrnidgY9tP7zmPZG6YrXLsmXyLlytsL1aocdbw4xynhUMtG5UquI611ThyWan6OZooN/0LceNvwziVYkP0e27ojubyA9xdFq+N2NLD6ZAgs/0RSWcx4b28u+3Lyo8XuprpCcC1xEJ6f+L1aOWJWT6Pr2TCA7UPg5729xBifG+sMG5XVhwUgZH0IGDlEc6nN7V20yryC6Jd4+O2vz/w+Glddi4rUM0ntkKguw3wt7bvg/vbD6+gvV96X52CqTY1VxCbS9tRaWNpQQWofbB9vKTLiNKiRcCrxqF5uw+qa4VRo+fpYkkbLu+v6wKhSjl22W18rO1Gan2+iJJdLOlZti/tM8g4LRwqi+b0RhWsrcaRgXKvIySdA6xh+8ctx/hVKR9rVGG/a3tkPajANcSu1+96jvNZwh/2byQdSJRDd9q5KqWjXawi5jtPtr2jpFfa/pKkkwlfzLZ8T9LnmC7C0koIapyugYUTgfMkHUcsxHdj+bcE2hE4x/atkj5EVDF9xKNTkn7hkiqQbNfSvmjDoE/sDYSKbxef2FqB7lpAUz21ZofnQ881u+fAbikD1BaUwGk+BU81rTBq9DxtAewyRiWEyZKpUo5ddhu/RtlttP21mpMcIS8iFEH/i1icdTqXSxD2Ngb6TYAjuwjMzEN6W1uNK5I2YiBxKOnJbZI/kvYiko/Nc06UdJTtw5bytOpIOoO4LjwY+KmkS5ic0GpVAmr7JEmXE60ki4C/s91pJyNZIs215c8KNenfE+diW55bvg7aUC1meIs/YPyugbYPKaXyzTn4EdtdAvhxYh/bp0ragtg5/wSR7BuViu95ZR34ZeDfm9a6UeF6PrE1At2DiPX2+cT5tyXwgQ5z6bVml/TZpf28RiIhA9SFzaquYIVRqNHzNG5lacmSqVmOXWW3ccyoVQ47b4OwCtSwtho7JB1LqFj+hO7JnzcS/Zq3lTEPJjQB5jRAJRa6vZnSh/gHJpRhkbT2XIu4zHOOKsI/HyJ65VYHPtx2ENt924Yaxu4aaPts4OxR/f5ZoNmg2BY43PbpkvYb1WRsryfp2YSS779I+imh4junCu2SDqO0E8xE2yCsb6BbKs4uImy2nkUEqO+z/fsWY1xN/E0PYGLNvhh4PO024Ga9+iQD1IVNLSsMqBBcjlsJYbJUapZjV9ltHCdKieUWwHq2jyvl9Kt3GGpeBmGVqGFtNY5sbnvDnmMsYvJrcS8j6O9udB8U1lG/awQ4St/eI1oMNdiH2NAcz7mIy3zGdqMY/R16vK6a2Vf1ZuBy21e2GGosroGSLrK9haRbmRy0dFVoHydulHQksBVwsKSVCUGqkeGwjrpEoZr8SaKMeq4txC5b9n8ZHklPIZIrj7D9tFIps73tA4Z5flNxZvuZFKGlDryi4/OmzmVSWbukBxOfgz4+sZPIAHVhU8sKI4PLhUfNcuyxEt+pgcJXcVPC9/E4Ilt5IvC8lkPN1yCsBoPWVouIDPCs98XMAT+okPw5DvhhEesC+DvC3mBUnMpEySfEOXwqQ9rDjGEf4rxF9WxUNi3/zijH2wKXEsnIU20P6zU7FtdA21uUr7X9YscBEZsMn7D9Z0mPIq6vo5lMqJfvQOygrkv0nT97rucxQxDW2b+0cDTxuh5Zxv9x6fEeKkAt9Ko4q71WL20AJwBrA4sk/RF4g+2f9B07A9QFiupaYSQLj2rl2BV3G8eJHYBnAD8CsP3bkmFsy3wNwnpj+zxJ6xFJgEXAtY2y5nLOl4gg9fd0TP7Y/qSkC5hQqd11hIInACvZvt931PZd6mAPU+5brwOeWJR41wEeWXZbkjrUslF5KLBJs6NSknanET1zlwPDBqhjcQ2cUmY+jeW5zNz27Qy0ENj+Hf0FzfpwFfA1YH/bbe0Kq6NK/qVEW90lU1R872k5xouAt0r6FeNRcXYUsLft8wEkvZAIxJ+7tCcNQwaoCxTXtcJIFhg1s3AVdxvHibvKZ6yxw1ityyDzOAjrjWbwcZM0tI/bGHMs0WfXyWNY0grAj20/jZIgGQP+KGl7F2sqhQXJTR3G+TzxmryYsDS7FfgKQ+7EJkNRy0ZlHeCugeO7gcfbvqNUbQ3FGF0DB8vM1wH+VL5fC/g1kLv89XiS+1tS1aS3f2nhJknrMmGT9RraJwLGreJstSY4BbB9Qdf1zlQyQF3YzEdxmmT5o9Zu4zjh0tOzlqQ3E1YExyzjOdOYx0FYDfr6uI0rv3YPj2Hb90m6StI6Y1Qh8zZCSfhz5fgGIghvy2a2N5F0Bdy/u9d6JzZZKrVsVE4m1hinl+PtgFPK4nXo8nVNWKD8WGGBsq+kA9zSrqYvTZm5pCOArzfaHZJeTvRuJvV4mKT3Ak8FVmketN1KAbom7u9fCqHYexSwgaQbCU/fnVvOY9za6a6TtA9x74X4e66vMXAGqAubeSdOkyyXVNltHCdsf0LS1sAtROb/w7bP7TDUfA3CalDDe3kcubb0JZ1Bd4/hRwE/Udi6DHpQtrJ1qcj1tjeXtDqwyPatHce5W9KKTOxAPJwOu8zJknElG5VSgn0WE2Xmu9tuRGfaKJnOZIFyOKOzQHmW7d2bA9tnS/rIiOYyXzmJsJh5BWHNsgvwxxHOp4p/qe3rgK3KGmeFHtfBcWI3wr++uT99h0ol+BmgLmzGrVQgWZjMtNt49Ijn1AtJBzssnM6d4bE2zNcgrAY1vJfHkQcRgelLBh5razPzr1Vn1J//lHQacKz7+ZZ+lhBM+RtJBwKvoX1vZLIMXMlGxfbl9LejGCsLFKJM80PETvNiYsfof0Y4n/nIQ21/QdJeRQn8QkkXjnA+vfxLl6BoTbMja/uT/ac4t0g6wfbrCUGk3p6nM5EB6gJmnorTJMsZFXcbx4mtganB6MtneGxZzNcgrAY1vJfHirI7+GPbn+o51DZTkyEKL9RRLfI2IhQ5v1B6ZI8lfA1vaTOI7ZMkXc7E7t7f9Qx4k4LG10Zl3CxQdgL2JRIlEDtGO41uOvOSu8vX30naFvgt8NhRTcY9/UuBpmVpfaJfvmnh2I44f5ZHninp8cBuko5nio1ZDdGwDFAXMPNUnCZZDikB6fIelCLpbUTP6JMk/XjgRw+mW2A574KwilRTkh4XbN8raXugb4BaK0FShVLKdjRwdBEXOQX4VNlV/Yjt/2wx1rXAtbMz04XLGNuojJUFSll471WsUO5zRd/H5H4OkLQm8C6ivWUN4J9HO6Xu2P5XAEn/QShb31qO9yPstpZHjgDOIbySL2e6R3Vvb+oMUBc281GcJlnOmCFjD2Hofhnhv3fd3M+qMycTpXEHAe8fePzWjhnFeReE1WIMxSJq8f0iJvRlJvePLlMUZhYSJFUoO8PbEr1JTwAOJfrMng+cBTxlVHNLJlNURm+w/ddiGbERcLyLL+pcM24WKJKeTmgDrF2ObwJ2sX3NqOY037B9Zvn2ZkIrZb4wVdn6LuJ6uNxh+7PAZyUdbvtts/E7MkBd2Mw7cZpkueSTRAnPyUQW7u+BRwI/J0oBXziymbVnRaJUeVp/iqS12wap8zgIS5ZM4x+3/8BjiwlrlWVRO0FSi18A5wMft/39gcdP62jXkMweXwE2lfRkwvvx68R5tc1IZzU+HMl038ejqOD7mASSnkIIYT3C9tMkbQRsb/uAEU+tLycAlxSLx8XEJlEXheyxYbaCU8gAdaFTxQojSXryMtuDioxHSbrY9v6SPrjEZ40njVceTOnJoFLZSzK/sd15x8D2zcSuw7j1xG20pFLI2RLYSDpzn+17JO0AfNr2YY2tTwLMou9jcj9HE2XcRwIUi6GTgZEEqJIeAXwUeLTtl0vaEHiO7S+0Gcf2gZLOJipHAHa1nZ+tJZAB6gJmnorTJMsf9ynk7E4rx68Z+Nk4mXUvk8YrL0m6UmsxNGZ8WNIBwB1E39LGwDttnzjaaSUzcLeknQhrj+3KYw8Y4XzGjVnzfUzuZ1Xbl0zxHb1nVJMBvkjotPxLOf6/RAtG62tyadWYUw/f5ZUMUBcwFa0wkqQPryMk3D9PBKQXAztLehDwT6OcWB8kPQRYj8lG48urYl8yd3yRSouhMeIltt9bduVuIHx8zydE+ZLxYlfCVuNA29dLeiL5Pg0y6Pu4iIq+j8n93FR6oZv2s9cwwr5j4GG2LekDAKXC4N5lPSnpRwaoC5uxUnpMFiZFBGm7Jfz4ormcSy0kvQnYi5DGvxLYHPgBw/URJgub+bgYanbgtgFOsf2/U3ZHkjHB9k+BPeH+JNuDbX9stLMaH2z/ifL6JLPG24m+3g0k3UjsUO88wvncJumhTATMmxOtFMkskgHqAmRclR6ThUnx330zoWZ3/zXJ9m6jmlMF9iL8zi62/SJJGxBZ9yRZFvNxMXSGpGuJEt89ymf+zhHPKZkBSRcA2xPX4iuBP0q60PbeI53YiJF0BktpObG9/RxOZ15TktZbld7eFRpblhHyLkIsbF1J3wMezuRWpGQWyAB1YTKuSo/JwuR04LvAt4Dlfaeo4U7bd0pC0sq2r5W0/qgnlSwX7M08WwzZfr+kg4FbitfrbcArRz2vZEbWtH1LqQI5zva+UxLZC5VPjHoCCwVJKwOvpiStm2oL2/sv5Wmzhu3LJb2A0GpZBPzc9t2jmMtCIgPUhUlVK4wk6cmq87Dv+QZJawFfA86V9CfCSidJlortH823xZCkVYg+vS2KrdlFhI1EMn6sJOlRgJjog17w2L6w+b7oI6xj++cjnNJ85nSiauRy4K8jnguSriJ0AL5s+5ejns9CIQPUhUlaYSTjxJmStrF91qgnUgvbO5Rv95N0PrAmoV6aJEulBHN7AFsQ1+PvSjrC9vJcEns8cCtwWDneiVBB3XFkM0qWxP7AN4GLbF8q6UmEj20CSNqO2E19IPBESX8L7J8lvlV5rO2XjXoSA2wPvJawZryPCFZt+9ejndb8ZtHixcuVi0OSJPMMSbcCqxGZ0ruJpMli22uMdGJJMgIkmQjmGuXUnYCH2F5ugzlJV9neeFmPJcm4I+lyQuzuAtvPKI/92PZGo53Z/EHSUcBhtq8e9VymImk9YB/gdbZXHPV85jO5g7rASSuMZNTYfvCo55AkY8T6UwK380uJ2fLMFZI2t30xgKTNSEG+saTs4L8ReCqT1wXLs2hdTe6xfXOqUNdH0tVE1chKwK6SriMS103SemRJAElPIMreX0toZbx3VHNZKGSAuoBJK4xklEjaoIgHbTLTz4uhdZIsNOZjMLcZ8AZJTUncOsDPmgVp7j6NFScA1wIvJcp9Xwf8bKQzGi+ukfQPwIplN21P4PsjntN84RWjnsBMSPohYZV1KrBjURlOZpkMUBc2aYWRjJK9gbcAhw48NthzkImSZCEyH4O5ceonS5bOk23vKOmVtr8k6WSiJzUJ3kGIR/0VOIV4bT4y0hnNE2z/atRzWAK72L521JNYaGSAurBJK4xkZNh+S/n2cOCcYm2wD7AJecNPFi7zLpgb44VnMp1GMfrPkp4G/J6w+0gA27cTAWoqHM9zJO1s+0RgG0nbTP257U+OYFoLhgxQFzZphZGMAx+ybUlbAFsTO6qHEztJSbKgyGAuGTFHFW2KfQg/3tWBD492SuODpE2BD1I8OpvHl9PKhmTprFa+pk7GCMgAdQGTVhjJmHBv+botcITt0yXtN8L5JEmSLEhsH1O+vZC0nJuJk4D3AFcD9414LsksYvvI8jVb30ZA2swkSTJSJJ0J3AhsBTwTuAO4JC0okiRJ5gZJey/t51nOGEi6yPYWo55HMndIOgQ4gFibnANsDLyzlP8ms0TuoCZJMmpE9N19wvafJT2KyFAnSZIkc0OWMQ7HvpKOAc4jhJIAsP3vo5tSMsu8xPZ7Je0A3ADsCJzPhFd1MgtkgJokyUgpohP/PnD8O+B3o5tRkiTJwiLLGIdmV2ADwnakKfFdzMA9LJl3PKB83QY4xfb/pg/u7JMBapIkSZIkSYKkLwF72f5zOX4IcKjt3UY7s7FhY9tPH/UkkjnlDEnXEiW+e0h6OHDniOc071lh1BNIkiRJkiRJxoKNmuAUwPafgGeMcD7jxsWSNhz1JJK5w/b7gecAm9q+G7gNeOVoZzX/yR3UJEmSJEmSBGAFSQ8pgSmS1ibXioNsAewi6XqiB3URsDhtZuYvknYkvNrvlfQhwqv9AMIjOJkl8qKTJEmSJEmSQPhQf1/SaURvpYADRzulseJlo55AMufsY/vU4tX+UuATpFf7rJMBapIkSZIkSYLt4yVdBryY2B18le2fjnhaY4PtX416DsmcM+jVfnh6tc8N6YOaJEmSJEmSJEkyhfRqHw0pkpQkSZIkSZIkSTIdAd8EXlYExNYmvdpnnQxQkyRJkiRJkiRJplC82v9ACGQB3AP8YnQzWhhkgJokSZIkSZIkSTIFSfsC7wM+UB56AHDi6Ga0MMgANUmSJEmSJEmSZDo7ANsT/qfY/i3w4JHOaAGQAWqSJEm4wKWaAAADhElEQVSSJEmSJMl07rK9mLBdQtJqI57PgiAD1CRJkiRJkiRJkulY0pHAWpLeDHwLOHrEc5r3pM1MkiRJkiRJkiTJDEjaGngJ4Q38TdvnjnhK854MUJMkSZIkSZIkSQaQtCIRkG416rksNLLEN0mSJEmSJEmSZADb9wK3S1pz1HNZaKw06gkkSZIkSZIkSZKMIXcCV0s6l6LkC2B7z9FNaf6TAWqSJEmSJEmSJMl0vlH+JXNI9qAmSZIkSZIkSZIkY0HuoCZJkiRJkiRJkkxB0tUUD9QBbgYuAw6w/T9zP6v5TwaoSZIkSZIkSZIk0zkbuBc4uRz/PWE3czPwRWC70UxrfpMBapIkSZIkSZIkyXSeZ/t5A8dXS/qe7edJ2nlks5rnpM1MkiRJkiRJkiTJdFaXtFlzIOnZwOrl8J7RTGn+kzuoSZIkSZIkSZIk03kTcKyk1YnS3luAN0paDThopDObx6SKb5IkSZIkSZIkyRKQtCawyPafRz2XhUAGqEmSJEmSJEmSJFMogem+wJbloQuB/W3fPLpZzX+yBzVJkiRJkiRJkmQ6xwK3Air/bgGOG+mMFgDZg5okSZIkSZIkSTKddW2/euD4XyVdObLZLBByBzVJkiRJkiRJkmQ6d0jaojmQ9DzgjhHOZ0GQO6hJkiRJkiRJkiTT2R04vvSiAvwJ2GWE81kQZICaJEmSJEmSJEkygKQVgPVtbyxpDQDbt4x4WguCVPFNkiRJkiRJkiSZgqTv2N5y2f8zqUkGqEmSJEmSJEmSJFOQtA/Rc/pl4Lbmcdv/O7JJLQCyxDdJkiRJkiRJkmQ6u5Wvbx94bDHwpBHMZcGQO6hJkiRJkiRJkiTJWJA7qEmSJEmSJEmSJFOQtAqwB7AFsXP6XeAI23eOdGLznAxQkyRJkiRJkiRJpnM8cCtwWDneCTgB2HFkM1oAZICaJEmSJEmSJEkynfVtbzxwfL6kq0Y2mwXCCqOeQJIkSZIkSZIkyRhyhaTNmwNJmwHfG+F8FgQpkpQkSZIkSZIkSTIFST8D1gd+XR5aB/gZcB+w2PZGo5rbfCZLfJMkSZIkSZIkSabzslFPYCGSO6hJkiRJkiRJkiTJWJA9qEmSJEmSJEmSJMlYkAFqkiRJkiRJkiRJMhZkgJokSZIkSZIkSZKMBRmgJkmSJEmSJEmSJGNBBqhJkiRJkiRJkiTJWPD/AEDKLXraU/KsAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f9938668da0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "genres_bin.sum(axis=0).sort_values(ascending=False)[:50].plot(kind=\"bar\", figsize=(16,6));"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we can see, the \"rock\" genre is the most dominating one. Thus, providing this label to a track does not add more relevant information to it to distinguish it from other tracks. The term \"romantic\" on the other hand is less frequent which makes it more discrimitative."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "tf            = genres_bin.sum(axis=0)\n",
    "idf           = np.log(genres_bin.shape[0] / tf)\n",
    "genres_tf_idf = genres_bin / idf"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "Based on this rescaled label weights, we can calculate the tag-based similarties between all pairs of tracks. We will use the Dice metric"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "# the pdist function calculates all pairwise distances\n",
    "dists = pdist(genres_tf_idf, 'dice')\n",
    "\n",
    "# pdist returns a list of results. We use the squareform function to convert this into a symmetric matrix\n",
    "tag_based_track_similarities = pd.DataFrame((1 - squareform(dists)), \n",
    "                                            index   = genres_tf_idf.index,\n",
    "                                            columns = genres_tf_idf.index)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can use this to search for similar tracks using the tag-similarities:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "index                                     44\n",
      "track_id              41Y5sPsWLArrksKhXheQ8f\n",
      "artist_name                    Damien N-Drix\n",
      "title                                 Inkali\n",
      "album_name                            Inkali\n",
      "label                       Spinnin' Records\n",
      "duration                              218890\n",
      "popularity                                51\n",
      "year                                    2017\n",
      "genres                       [electro house]\n",
      "playlist                           clubbeats\n",
      "featurespace_index                        44\n",
      "Name: 33, dtype: object\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/schindler/anaconda/python2/envs/py36/lib/python3.6/site-packages/ipykernel_launcher.py:9: FutureWarning: \n",
      "Passing list-likes to .loc or [] with any missing label will raise\n",
      "KeyError in the future, you can use .reindex() as an alternative.\n",
      "\n",
      "See the documentation here:\n",
      "http://pandas.pydata.org/pandas-docs/stable/indexing.html#deprecate-loc-reindex-listlike\n",
      "  if __name__ == '__main__':\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>artist_name</th>\n",
       "      <th>title</th>\n",
       "      <th>album_name</th>\n",
       "      <th>year</th>\n",
       "      <th>genres</th>\n",
       "      <th>playlist</th>\n",
       "      <th>sim</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>33</th>\n",
       "      <td>Damien N-Drix</td>\n",
       "      <td>Inkali</td>\n",
       "      <td>Inkali</td>\n",
       "      <td>2017</td>\n",
       "      <td>[electro house]</td>\n",
       "      <td>clubbeats</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Don Diablo</td>\n",
       "      <td>Momentum</td>\n",
       "      <td>Momentum</td>\n",
       "      <td>2017</td>\n",
       "      <td>[big room, deep big room, edm, electro house, ...</td>\n",
       "      <td>clubbeats</td>\n",
       "      <td>0.933333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>52</th>\n",
       "      <td>SNBRN</td>\n",
       "      <td>Lovin' Me</td>\n",
       "      <td>Lovin' Me</td>\n",
       "      <td>2018</td>\n",
       "      <td>[deep tropical house, edm, electro house, hous...</td>\n",
       "      <td>clubbeats</td>\n",
       "      <td>0.933333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>KSHMR</td>\n",
       "      <td>Kolkata</td>\n",
       "      <td>Kolkata</td>\n",
       "      <td>2017</td>\n",
       "      <td>[big room, deep big room, edm, electro house, ...</td>\n",
       "      <td>clubbeats</td>\n",
       "      <td>0.933333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>38</th>\n",
       "      <td>Henry Fong</td>\n",
       "      <td>Bubblin Anthem</td>\n",
       "      <td>Bubblin Anthem</td>\n",
       "      <td>2017</td>\n",
       "      <td>[big room, deep big room, edm, electro house, ...</td>\n",
       "      <td>clubbeats</td>\n",
       "      <td>0.923077</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>295</th>\n",
       "      <td>AC/DC</td>\n",
       "      <td>Dirty Deeds Done Dirt Cheap</td>\n",
       "      <td>Dirty Deeds Done Dirt Cheap</td>\n",
       "      <td>1976</td>\n",
       "      <td>[album rock, classic rock, hard rock, heavy ch...</td>\n",
       "      <td>rockclassics</td>\n",
       "      <td>0.875000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>36</th>\n",
       "      <td>Marco V</td>\n",
       "      <td>Chills</td>\n",
       "      <td>Chills</td>\n",
       "      <td>2018</td>\n",
       "      <td>[bubble trance, progressive house, progressive...</td>\n",
       "      <td>clubbeats</td>\n",
       "      <td>0.875000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Timmy Trumpet</td>\n",
       "      <td>Punjabi</td>\n",
       "      <td>Punjabi</td>\n",
       "      <td>2017</td>\n",
       "      <td>[australian dance, catstep, edm, melbourne bou...</td>\n",
       "      <td>clubbeats</td>\n",
       "      <td>0.875000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>TS7</td>\n",
       "      <td>Real Raver</td>\n",
       "      <td>Real Raver</td>\n",
       "      <td>2018</td>\n",
       "      <td>[bassline, deep groove house, house, speed gar...</td>\n",
       "      <td>clubbeats</td>\n",
       "      <td>0.875000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>40</th>\n",
       "      <td>TJR</td>\n",
       "      <td>Time To Jack</td>\n",
       "      <td>Time To Jack</td>\n",
       "      <td>2017</td>\n",
       "      <td>[big room, deep big room, edm, electro house, ...</td>\n",
       "      <td>clubbeats</td>\n",
       "      <td>0.875000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       artist_name                        title                   album_name  \\\n",
       "33   Damien N-Drix                       Inkali                       Inkali   \n",
       "4       Don Diablo                     Momentum                     Momentum   \n",
       "52           SNBRN                    Lovin' Me                    Lovin' Me   \n",
       "23           KSHMR                      Kolkata                      Kolkata   \n",
       "38      Henry Fong               Bubblin Anthem               Bubblin Anthem   \n",
       "295          AC/DC  Dirty Deeds Done Dirt Cheap  Dirty Deeds Done Dirt Cheap   \n",
       "36         Marco V                       Chills                       Chills   \n",
       "6    Timmy Trumpet                      Punjabi                      Punjabi   \n",
       "16             TS7                   Real Raver                   Real Raver   \n",
       "40             TJR                 Time To Jack                 Time To Jack   \n",
       "\n",
       "     year                                             genres      playlist  \\\n",
       "33   2017                                    [electro house]     clubbeats   \n",
       "4    2017  [big room, deep big room, edm, electro house, ...     clubbeats   \n",
       "52   2018  [deep tropical house, edm, electro house, hous...     clubbeats   \n",
       "23   2017  [big room, deep big room, edm, electro house, ...     clubbeats   \n",
       "38   2017  [big room, deep big room, edm, electro house, ...     clubbeats   \n",
       "295  1976  [album rock, classic rock, hard rock, heavy ch...  rockclassics   \n",
       "36   2018  [bubble trance, progressive house, progressive...     clubbeats   \n",
       "6    2017  [australian dance, catstep, edm, melbourne bou...     clubbeats   \n",
       "16   2018  [bassline, deep groove house, house, speed gar...     clubbeats   \n",
       "40   2017  [big room, deep big room, edm, electro house, ...     clubbeats   \n",
       "\n",
       "          sim  \n",
       "33   1.000000  \n",
       "4    0.933333  \n",
       "52   0.933333  \n",
       "23   0.933333  \n",
       "38   0.923077  \n",
       "295  0.875000  \n",
       "36   0.875000  \n",
       "6    0.875000  \n",
       "16   0.875000  \n",
       "40   0.875000  "
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "query_track_idx = 33\n",
    "\n",
    "print(metadata_aligned.iloc[query_track_idx])\n",
    "\n",
    "# get the similar tracks from the distance-matrix\n",
    "similar_tracks_idx = tag_based_track_similarities[query_track_idx].sort_values(ascending=False).index\n",
    "\n",
    "# filter the aligned metadata\n",
    "result        = metadata_aligned.loc[similar_tracks_idx]\n",
    "result[\"sim\"] = tag_based_track_similarities.loc[query_track_idx,\n",
    "                                                 similar_tracks_idx].values\n",
    "\n",
    "# show results\n",
    "display_cols = [\"artist_name\", \"title\", \"album_name\", \"year\", \"genres\", \"playlist\", \"sim\"]\n",
    "result[display_cols][:10]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Interposed Question\n",
    "\n",
    "*The tag-similarities seem to be quite accurate. Why don't we use them instead of this complicated and computing intensive Neural Networks which train on audio data?*\n",
    "\n",
    "Well, this seems obvious on the first hand, but remember that we don't have labels for all artists. Thus, for those without labels, we cannot calculate similarities. But, we have audio features for all tracks. Thus, we use the subset of provided labels to train a general model based on audio content."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "# Train the Siamese Networks on the Tag-Similarity Ground-Truth\n",
    "\n",
    "We use the same Siamese Network architecture presented in part 2 of this tutorial. Also the method to create the pairwise training-data is the same. Please refer to part 2 for further details."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "def create_siamese_network(input_dim):\n",
    "\n",
    "    # input layers\n",
    "    input_left  = Input(shape=input_dim)\n",
    "    input_right = Input(shape=input_dim)\n",
    "\n",
    "    # shared fully connected layers\n",
    "    shared_fc_1 = Dense(100, activation=\"selu\")\n",
    "    shared_fc_2 = Dense(100, activation=\"selu\")    \n",
    "    \n",
    "    # siamese layers\n",
    "    left_twin  = shared_fc_1(shared_fc_2(input_left))\n",
    "    right_twin = shared_fc_1(shared_fc_2(input_right))\n",
    "\n",
    "    # calc difference\n",
    "    distance = Lambda(tut_func.euclidean_distance,\n",
    "                      output_shape=lambda x: x[0])([left_twin, right_twin])\n",
    "\n",
    "    return Model([input_left, input_right], distance)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create Data-Pairs\n",
    "\n",
    "Now we have to prepare and partition the input data. We use the approach which considers identity. Thus, we will reduce the similarity values which have been looked-up from the distance-matrix by 0.1:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "def create_pairs_with_sims_and_identity(feature_data, metadata, num_pairs_per_track, sims):\n",
    "    \n",
    "    data_pairs = []\n",
    "    labels     = []\n",
    "    \n",
    "    for row_id, q_track in metadata.sample(frac=1).iterrows():\n",
    "        \n",
    "        # identical pair\n",
    "        data_pairs.append([feature_data[[row_id]][0], feature_data[[row_id]][0]])\n",
    "        labels.append(1)\n",
    "                \n",
    "        for _ in range(num_pairs_per_track):\n",
    "            \n",
    "            # search similar and dissimilar examples\n",
    "            pos_example = metadata[metadata.playlist == q_track.playlist].sample(1)\n",
    "            neg_example = metadata[metadata.playlist != q_track.playlist].sample(1)\n",
    "\n",
    "            # create feature pairs\n",
    "            \n",
    "            # similar pair\n",
    "            data_pairs.append([feature_data[[metadata.loc[row_id].featurespace_index]][0], feature_data[[pos_example.featurespace_index]][0]])\n",
    "            sim_val = sims.iloc[row_id, pos_example.index].values[0] - 0.1\n",
    "            labels.append(np.max([0, sim_val]))\n",
    "            \n",
    "            # dissimilar pair\n",
    "            data_pairs.append([feature_data[[metadata.loc[row_id].featurespace_index]][0], feature_data[[neg_example.featurespace_index]][0]])\n",
    "            sim_val = sims.iloc[row_id, neg_example.index].values[0] - 0.1\n",
    "            labels.append(np.max([0, sim_val]))\n",
    "            \n",
    "\n",
    "    return np.array(data_pairs), np.array(labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Execute the function to prepare the data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(20307, 2, 69)"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_pairs, labels = create_pairs_with_sims_and_identity(feature_data, \n",
    "                                                         metadata_aligned, \n",
    "                                                         10, \n",
    "                                                         tag_based_track_similarities)\n",
    "\n",
    "data_pairs.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Prepare the Siamese Neural Network"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "# define the model\n",
    "model = create_siamese_network(data_pairs[:,0].shape[1:])\n",
    "\n",
    "# define the optimizer\n",
    "opt = Nadam(lr=0.001)\n",
    "\n",
    "# compile the model\n",
    "model.compile(loss      = tut_func.contrastive_loss, \n",
    "              optimizer = opt)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Train the network on the tag-similarity ground-truth"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAD8CAYAAABw1c+bAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3Xu4JHV95/F3Vfe5dffMiDbDZGa4CURJWMXILQ95EC8gCDqoyy+MJAsGHckjkR2SzUo0Oy6KsvogS1Y3ZMKSUWIg311BBxFxHpCgZsERdZzoxCyOCIczF84MczlzbtPdtX9U9Tk959qnz6V76vd5PU8/3fWrqq7ft/ucb/36V1W/CqIoQkRE/BE2uwIiIrKwlPhFRDyjxC8i4hklfhERzyjxi4h4RolfRMQzSvwiIp5R4hcR8YwSv4iIZ7LNrsAkdDmxiMjMBfUs1KqJn56enobWKxaL9Pb2znFtjg4+xw5+x6/Y/YwdRuNfvnx53euoq0dExDNK/CIinlHiFxHxTMv28YuIX6IoYnBwkEqlQhDUdYwSgF27djE0NDSPNWsdURQRhiGdnZ0z+ozGUuIXkZYwODhIW1sb2ezM0lI2myWTycxTrVpPqVRicHCQrq6uht9DXT0i0hIqlcqMk76PstkslUpldu8x3QLOuXuAy4HdZnbGBPP/E3B1zfudDhxrZnudc88BB4EyUDKzs2ZVWxFJrdl0Xfhmtp9VPbvXDcAXgC9PNNPMPgd8DsA5905grZntrVnkzWY27yfZRlFE9PA/MvT6s+D4U+d7cyIiR61pu3rM7Elg73TLJVYD982qRg0KgoDo0QcZ/vHTzdi8iKTAaaed1uwqLIg56+N3zuWAS4Cv1hRHwLedc88459bM1bYmlStQ6Ts475sRETmazeWRlHcC3x/TzXO+mfU455YCm5xz/5r8ghgn2TGsATAzisXijCuwZ/ES6O9raN00yGaz3sYOfsefhth37drV8MHduTwonM1miaKIW265hccee4wgCFi7di1XXHEFu3btYs2aNRw8eJBSqcRnP/tZzj77bNauXctPfvITgiBg9erVXH/99XNWn4l0dHSMfN+NfPdzmfivYkw3j5n1JM+7nXMPAucAEyZ+M1sPrE8mo0bG3ii3dxIcPODtuB0as8Tf+NMQ+9DQ0MhpmZX7/5bohV/VtV4QBETR9OM6BsefTHjVB6ddrlQq8fDDD7N161Y2bdrE3r17ecc73sHZZ5/Ngw8+yAUXXMCNN95IuVxmYGCALVu20NPTw+OPPw7A/v37KZVKddW9UUNDQyPfd9PG6nHOLQHeBHy9pizvnFtUfQ1cDPzLXGxvUrkC0SF19YjI7PzgBz/giiuuIJPJcOyxx3LeeeexZcsWzjzzTMyM22+/nW3btlEoFDjhhBN4/vnn+fjHP853vvMdFi1a1OzqT6ue0znvAy4Eis65bmAd0AZgZncli70b+LaZHapZ9TjgQedcdTv/YGbfmruqjxfk8lRe2F7fuKQi0rLqaZlXZbPZOW9hT/YL4rzzzuOrX/0qjz32GDfeeCPXX389V155JZs2beKJJ55gw4YNPPTQQ3z+85+f0/rMtWkTv5mtrmOZDcSnfdaWbQde32jFGpIrEB3qU+IXkVk577zz+Pu//3uuvPJK9u3bx9NPP81f/uVf0t3dzbJly7j66qvp7+9n69atvPWtb6WtrY3LLruME088kbVr1za7+tNK12Vy+TzRYD9RqUSgKwBFpEGXXnopzzzzDBdddBFBEPCxj32MpUuXYmbcddddZLNZ8vk8d955Jzt27OCmm24auZr25ptvbnLtpxfUc1CkCaJGbsRSeewbRPevJ/z8vQSLlsxDtVpbGg7wzYbP8ach9v7+fnK53IzXm4+unlZX+1mNObhbV4dHusbqyefj5/5DUy8nIuKxVCX+IFeIX/T3NbciIiItLFWJn1zS4j+kxC8iMpmUJf64xR8NqKtHRGQyqUz8avGLiEwuZYm/enBXiV9EZDKpSvxBewe0teusHhGRKaQq8QOEhUVq8YvIvJtq7P4XXniBt7zlLQtYm5lJXeIPcgUiJX4RkUmlblyDsLCIsrp6RI5qd/9wF796ebCuZesdlvnkYzr5wFnHTTr/1ltvZcWKFVx77bUA3H777QRBwFNPPTUy1PKf//mf8/a3v72uelUNDg5y880389Of/pRMJsO6des4//zz+cUvfsFNN93E8PAwURSxfv16li1bxoc+9CF27NhBpVLhxhtvZNWqVTPaXj1Sl/iDwiLofanZ1RCRo8yqVatYt27dSOJ/6KGH+MpXvsIHP/hBFi1axN69e3nnO9/JxRdfPKObnW/YsAGAxx57jGeffZbVq1fz3e9+l3vvvZfrrruO97znPQwPD1Mul3n88cdZtmwZ9957LwAHDhyY6zCBFCb+ML8Inq/vBg4i0pqmapmPNVdj9Zxxxhn09vayc+dO9uzZw5IlS1i6dCmf+MQnePrppwmCgJ07d/LSSy+xdOnSut938+bNvP/97wfg1FNPZeXKlWzfvp03vvGN/NVf/RU7duzg0ksv5dWvfjWvfe1r+eQnP8mtt97K2972Ns4999xZxzWR9PXx5xfprB4Rachll13Gww8/zMaNG1m1ahUPPPAAe/bs4ZFHHmHTpk0Ui0WGhoZm9J6TdUO9+93v5u/+7u/o7Ozk6quv5nvf+x6nnHIKjzzyCK997Wv5zGc+wx133DEXYY2TusQfn9VziCgZIlVEpF6rVq3i61//Og8//DCXXXYZBw8epFgs0tbWxve//326u7tn/J7nnnsuDz74IAC//OUvefHFFznllFP49a9/zYknnsh1113HRRddxLZt29i5cyddXV28973v5frrr2fr1q1zHSKQwq6eIF+AqAKDA6MXdImI1OE1r3kNhw4dYtmyZRx33HG85z3v4ZprruHSSy/lt3/7tzn11FNn/J7XXHMNH/3oR3nrW99KJpPhjjvuoKOjg40bN/LAAw+QzWZZunQpa9euZcuWLXzqU58iCALa2tr4zGc+Mw9Rpmw8foD8T57iwBc/TXjb3QSvqr8fLg3SMCb7bPgcfxpi13j89dN4/GMEheRGxxqvR0RkQqnr6gnzGpNfRBbGtm3b+MhHPnJEWUdHB9/4xjeaVKP6pC7xj7T4dWaPyFGlRbudp3T66aezadOmBd/ubD+raRO/c+4e4HJgt5mdMcH8C4GvA9WT5x8ws1uSeZcAdwIZ4G4zu21Wta1DmI8Tf9TfV19nl4i0hDAMKZVKZLOpa4/OqVKpRBjOrpe+nk94A/AF4MtTLPNdM7u8tsA5lwG+CFwEdAObnXMbzeznDda1LkG+2uJXV4/I0aSzs5PBwUGGhoZmdGVsR0fHjM+tP1pFUUQYhnR2ds7qfaZN/Gb2pHPupAbe+xzgWTPbDuCcux9YBcxv4u/KQRCqq0fkKBMEAV1dXTNeLw1nNC20ufpN9bvOuS1AD/BnZvYzYAXwQs0y3cCk1x8759YAawDMjGKx2FBFstksQT5PZ6XM4gbf42iVzWYb/tzSwOf4FbufsUNj8c9F4v8RcKKZ9Tnn3gF8DTiNic8nnfSIhJmtB9ZXl2t0D14sFom68gzu6WXYs1aA7y0fn+NX7H7GDuPO46/LrM/jN7MDZtaXvP4m0OacKxK38I+vWXQl8S+C+Zcr6IbrIiKTmHWL3zm3DNhlZpFz7hzinckeYB9wmnPuZOBF4CrgfbPdXl1yeR3cFRGZRD2nc94HXAgUnXPdwDqgDcDM7gL+PfDHzrkSMABcZWYRUHLO3QA8Snw65z1J3//8y+Vh394F2ZSIyNGmnrN6Vk8z/wvEp3tONO+bwDcbq1rjdPtFEZHJpW6sHiBu8WusHhGRCaU08RegdJjo8HCzayIi0nLSm/hBrX4RkQmkNPEnN2BRP7+IyDipTPxBtcWvYRtERMZJZeJHY/KLiEwqnYm/K+7q0SmdIiLjpTPx59XVIyIymXQm/i4d3BURmUwqE3+QzUJHJxxSi19EZKxUJn4gPpd/QC1+EZGxUpz480Rq8YuIjJPqxK8+fhGR8VKc+As6q0dEZAKpTfxBrqAWv4jIBFKb+NXVIyIysRQn/gIMDhCVy82uiYhIS0lv4q9evaubrouIHCG9iV9X74qITCi1iT/QeD0iIhNKbeIfuQuXWvwiIkfITreAc+4e4HJgt5mdMcH8q4H/nEz2AX9sZluSec8BB4EyUDKzs+ao3tNL7sIVHTpEsGAbFRFpffW0+DcAl0wx/1fAm8zsdcAngfVj5r/ZzM5c0KQPoy1+jdcjInKEaVv8Zvakc+6kKeb/c83kU8DKOajX7I3ccF19/CIitaZN/DN0HfBIzXQEfNs5FwF/Y2Zjfw2McM6tAdYAmBnFYrGhCmSzWYrFIlEUsTubpSsqs6jB9zraVGP3lc/xK3Y/Y4fG4p+zxO+cezNx4v+9muLzzazHObcU2OSc+1cze3Ki9ZOdQnXHEPX29jZUj2KxyMi6XXkG9vQy1OB7HW2OiN1DPsev2P2MHUbjX758ed3rzMlZPc651wF3A6vMbE+13Mx6kufdwIPAOXOxvbppvB4RkXFmnfidcycADwB/aGb/VlOed84tqr4GLgb+Zbbbm5FcXjdcFxEZo57TOe8DLgSKzrluYB3QBmBmdwH/BXgV8D+dczB62uZxwINJWRb4BzP71jzEMLl8AfoOLugmRURaXT1n9ayeZv4HgA9MUL4deH3jVZu9oCtPtHtHM6sgItJy0nvlLsQtfnX1iIgcId2JP7kLVxRFza6JiEjLSHniz0OlAkMDza6JiEjLSHni19W7IiJjpTrxBxqvR0RknFQn/uoInWrxi4iMSnni15j8IiJjpTzxJ2Py6y5cIiIjUp741eIXERkr3Ym/KwdBoMQvIlIj1Yk/CMM4+aurR0RkRKoTPwBdebX4RURqpD/x5wtEh5T4RUSq0p/4cwUYUFePiEiVB4k/D2rxi4iMSH3iD5IROkVEJJb6xB939ajFLyJS5UHiz8PwMNHhw82uiYhIS/Ag8WuEThGRWh4kfo3QKSJSa9qbrQM45+4BLgd2m9kZE8wPgDuBdwD9wLVm9qNk3jXAx5NFP2VmX5qLitcryBWIQBdxiYgk6m3xbwAumWL+pcBpyWMN8NcAzrlXAuuAc4FzgHXOuWMarWxDqi1+ndkjIgLUmfjN7Elg7xSLrAK+bGaRmT0FvMI59xvA24FNZrbXzF4GNjH1DmTu5eM+/kgtfhERYO76+FcAL9RMdydlk5UvnJEWvxK/iAjU2cdfh2CCsmiK8nGcc2uIu4kwM4rFYkMVyWazR6wbLVnCbiBHRKHB9zxajI3dNz7Hr9j9jB0ai3+uEn83cHzN9EqgJym/cEz5ExO9gZmtB9Ynk1Fvb29DFSkWi4xbt72D/t7dDDb4nkeLCWP3iM/xK3Y/Y4fR+JcvX173OnOV+DcCNzjn7ic+kLvfzHY45x4FPl1zQPdi4OY52mb9NF6PiMiIek/nvI+45V50znUTn6nTBmBmdwHfJD6V81ni0znfn8zb65z7JLA5eatbzGyqg8TzI1cg0gidIiJAnYnfzFZPMz8CPjzJvHuAe2ZetTmUK6jFLyKSSP+VuxB39eisHhERwJPEr6GZRURGeZH44xa/Er+ICHiT+OPbL0aVcrNrIiLSdH4k/nxy9e7AQHPrISLSAvxI/F3JmPw6wCsi4kfiD/Iar0dEpMqLxD9yFy4d4BUR8SXxq8UvIlLlSeJPxuTX1bsiIn4lfjRej4iIJ4m/oxPCUOP1iIjgSeIPgiBu9auPX0TEj8QPJIlfXT0iIh4l/rxuuC4igleJXy1+ERHwKPEHeSV+ERHwKPHTpZuxiIiAT4k/Hyf+KIqaXRMRkabyJ/HnClAuw/BQs2siItJUHiX+ZLweXcQlIp7L1rOQc+4S4E4gA9xtZreNmX8H8OZkMgcsNbNXJPPKwNZk3vNm9q65qPhMBbkCEcT9/K8sNqMKIiItYdrE75zLAF8ELgK6gc3OuY1m9vPqMma2tmb5PwHeUPMWA2Z25txVuUEamllEBKivq+cc4Fkz225mw8D9wKopll8N3DcXlZtTGppZRASor6tnBfBCzXQ3cO5ECzrnTgROBh6vKe50zv0QKAG3mdnXGqzr7FSHZu7vI2hKBUREWkM9iX+iPDnZOZFXAf/HzMo1ZSeYWY9z7tXA4865rWb2y7ErOufWAGsAzIxisbF++Gw2O+G6lY52XgIKAeQafO9WN1nsvvA5fsXuZ+zQWPz1JP5u4Pia6ZVAzyTLXgV8uLbAzHqS5+3OuSeI+//HJX4zWw+sTyaj3t7eOqo2XrFYZKJ1o0q8L+p7aRf9Db53q5ssdl/4HL9i9zN2GI1/+fLlda9TT+LfDJzmnDsZeJE4ub9v7ELOudcAxwD/t6bsGKDfzIacc0XgfOCzddduDgVhBrpyOrgrIt6b9uCumZWAG4BHgW1xkf3MOXeLc6721MzVwP1mVtsNdDrwQ+fcFuA7xH38P6dZNCa/iAhBiw5hEPX0TNabNLWpfvaV/+uN8Kpjydzw8dnUrWXpJ6+/8St2P2OHcV09dZ274s+VuwD5gq7cFRHv+ZX4c3ndcF1EvOdV4g9yebX4RcR7XiV+HdwVEfEu8edheIiodLjZNRERaRrPEr8GahMR8TTxq7tHRPzlVeIPRkboVItfRPzlVeJXi19ExLfEn68OzawWv4j4y6/E36WbsYiI+JX4kxa/LuISEZ95lfiDtnZoa9ewDSLiNa8SPxBfxKU+fhHxmIeJv0Ckrh4R8ZiHiV8jdIqI3zxM/BqTX0T85l3iD3J5nc4pIl7zLvHHQzOrq0dE/OVh4o/7+KNKpdk1ERFpCg8TfwGiCAb7m10TEZGmyNazkHPuEuBOIAPcbWa3jZl/LfA54MWk6Atmdncy7xrg40n5p8zsS3NQ78bla8bkrw7aJiLikWkTv3MuA3wRuAjoBjY75zaa2c/HLPqPZnbDmHVfCawDzgIi4Jlk3ZfnpPYNCLryRJAc4D2uWdUQEWmaerp6zgGeNbPtZjYM3A+sqvP93w5sMrO9SbLfBFzSWFXniMbrERHP1dPVswJ4oWa6Gzh3guXe65y7APg3YK2ZvTDJuisarOvcqHbv6CIuEfFUPYk/mKAsGjP9EHCfmQ05564HvgS8pc51AXDOrQHWAJgZxWKxjqqNl81mp1y3XDlML1AIA7oa3Earmi72tPM5fsXuZ+zQWPz1JP5u4Pia6ZVAT+0CZranZvJvgf9Ws+6FY9Z9YqKNmNl6YH0yGfX29tZRtfGKxSJTrRsNHQbg4K6dHGpwG61qutjTzuf4FbufscNo/MuXL697nXoS/2bgNOfcycRn7VwFvK92Aefcb5jZjmTyXcC25PWjwKedc8ck0xcDN9ddu/nQ2QVBqIu4RMRb0x7cNbMScANxEt8WF9nPnHO3OOfelSz2Eefcz5xzW4CPANcm6+4FPkm889gM3JKUNU0QBMnQzDq4KyJ+CqJowi73Zot6enqmX2oC9fzsK//FGoKTf5Pwg3/W0DZalX7y+hu/YvczdhjX1TPRcdVx/LtyF+Ix+dXVIyKe8jTxq6tHRPzlZeIPcgUlfhHxlpeJn7yGZhYRf/mZ+Lvirp4WPbAtIjKv/Ez8+QKUSjA83OyaiIgsOD8T/8h4PernFxH/eJr48/HzIfXzi4h/vEz8QbXFrzN7RMRDXib+kRa/zuwREQ95mvjjFn+kFr+IeMjrxK+uHhHxkZ+JvysXP6urR0Q85GXiD7JZ6OhSi19EvORl4gcgn9cN10XES/4m/q48kW64LiIe8jfx5zVCp4j4yd/Enyuoq0dEvORt4g+68qCuHhHxkLeJn3xBY/WIiJf8Tfy5AgwNEJVKza6JiMiCytazkHPuEuBOIAPcbWa3jZl/E/ABoAS8BPyRmf06mVcGtiaLPm9m75qjus9OdbyegX5YtLi5dRERWUDTJn7nXAb4InAR0A1sds5tNLOf1yz2Y+AsM+t3zv0x8Fng95N5A2Z25hzXe/Zqh21Q4hcRj9TT4j8HeNbMtgM45+4HVgEjid/MvlOz/FPAH8xlJedDkCsQgU7pFBHv1JP4VwAv1Ex3A+dOsfx1wCM1053OuR8SdwPdZmZfm3Et54OGZhYRT9WT+IMJyia8S7lz7g+As4A31RSfYGY9zrlXA48757aa2S8nWHcNsAbAzCgWi3VUbbxsNlvXuqX+lewBFmUCOhvcVqupN/a08jl+xe5n7NBY/PUk/m7g+JrplUDP2IWcc28DPga8ycyGquVm1pM8b3fOPQG8ARiX+M1sPbA+mYx6e3vrDOFIxWKRetaNhg8DcGDXTvoa3FarqTf2tPI5fsXuZ+wwGv/y5cvrXqeexL8ZOM05dzLwInAV8L7aBZxzbwD+BrjEzHbXlB8D9JvZkHOuCJxPfOC3+bo0Jr+I+Gna8/jNrATcADwKbIuL7GfOuVucc9VTMz8HFID/7Zz7iXNuY1J+OvBD59wW4DvEffw/pxW0t0M2q2EbRMQ7QRRN2F3fbFFPz7jepLrM5Gdf+U//A8GZ5xL+4Ycb2lar0U9ef+NX7H7GDuO6eiY6JjuOv1fuQnxmj1r8IuIZzxN/QTdcFxHveJ/4dR6/iPgmVYl//+DMBlwLcnmd1SMi3qlrkLajQbkS8ScP/4pjCzs4//gcF5y0mGKubeqV1OIXEQ+lJ/FHEe6MV/H97gG+9OOX+PKPX+KM43JcePJifvf4ReTbM+NXShJ/FEUEQV0Hw0VEjnqpSfztmZDLX/NKrj2/yE+39/Dkcwd44rn9/I+ndnLXD3Zx9soCbzppMW9cnqctk/Rw5fIQVWBwALpyzQ1ARGSBpCbx11q+uJ2rXlfk9//dq/h/ewZ54rkDfO+5A/zz8wcptIecf8Ji3nTSYl7TlY9Peu3vU+IXEW+kMvFXBUHAbxa7+M1iF3/0O0vZsuMQ//TcAZ741X4efXYfx2ZXcN4pl7P4Wz8kXLyETGeOsKuLsCtHmMsRZjJkwoAwgDAYfYb4mEI5iihV4ke5QvIcUYoiSuWIcjRaVqm5Ti4ionrdXO3lc1FUOx0vUy2LIqhUy0aWrZ2O6OjYTVA+TEc2pCMTJM8h7dmAjkxIR+1zNqQ9E9AWhoQBZMKATBJfJqw+xzFnasuC+HMNgCAgeW7tbrLBUoX9gyX2D5bjx1CJA4NlBkoV8u0hhfYM+fYMheR1PB3SlQ1bPjaRRqQ68dfKhgFvXFHgjSsKDByu8HT3Qf7pFy/x8Mrfo0IIB4gfAPQnj1lsj4hMEJEBskE0cvpUbR4JRp6jI8qCmmWr00e8TtYJgvi0rOp8wgwDpQrDUcBQJWA4ih8LYcJ6Jq/DALIBtIUB2TB+bgsDsplqWUBbJiQbBmSTnVEmDAhHdjYBYRjEO9+QI6YzSVkAlDMH2fnyQQ4MJQl+sMT+oTLD5cauTs8EjOwQ8skOIRsGVKJ4p16JIiqV0dflSnysqRIxsrOvRBEdmZDOtoDObEhnNt6hdLbVvj5yXns2/mupvar+yEZBzY4/mV58IKDvQF/8N1Gzc45fQ0gQfx/Jzr36/VQbNPGywci6tWVj33Mmaj+XcvV10miqjGkYVRtNE4/9O7nF+wP6+/poy8R/V22ZsOZ19W9tdDrUztyfxF+rqy3kwpOXcOHJS+I/wlKJysH9VPa9THn/Psr791E5sJ/KwX2UDxygcvAAlb4DVPoOUkma7pmoTKZSJhtVyEYlMpUK2agcl0eVGf+DzJcyAYfDLEOZdobCNoaT56FMG4OZdspBhnIQUkke5eRRIZywvBzEu5pKENQ8Q4U4q1Qjr51fIaQUZjgcZikFo8+lMBuXB1kGk/mHk/LabVcIjpwOqtNHHrBvLw+z+PAhlpQOseRwPytL/Swp9bN45HmAJZUBlpQHWFwepCMqMZDpoC/byaFsF31hJ4eynfRlOuOysJO+TAeHMvFzX6aTUhASRhEhERni5yxHTocQvw4gJGI4yDKYPPYlzwPEz8PBBCcdNOTFOXqfo9HMYs8kv2Jrd5zAuF/hY6dHf/3GDZhMGJAdOx0GRywzLg9MsNOpLSm0Z/joBStmFE8jvEz8tTJhQKa9DV5VjB9TiCqVeIiHgT5Gvq6xX2QwtnyyXcCYZs3YMZNqp0deR+P/KmvKjnnFK3h5374jNhsS0AbkRuo1UtHR96mUoVKBcnn0daUM5cqY6WT+TEVAVIk/v+ojGvt6GCqDNdNREltU8x9YGe3nSuKOW91x63dxrpOBvr5kncroupU2iBZDVEjePxrZRp6IfG2fGhFEg0Bcl6j6+ZYjOFxTp9p6VmM5orxmOq7okesm8ZQrMBQmOwayDIRt8c4ggiCKCKLRZYMoir+1KP4O4vnxewbEn0VERCUKRn4RVJLfk1GyE45qdsbVHfVoeUiUlEXJMmPnzfy7j5IGUSV5lAmjCtmoQpiUZyujr0MiwmrMdf95BSMNi8NBNm5YjJvOcjhpZBwOs1SCMFkz/hyr4k8rqPk3Dkb+l+MGUIZyGFJKGkzlIJO8zlAKw5GGVLWs9gd3RO3/YO1P/9HXpaAMSvytJQjD+P68LXqP3myxSNDVuoNVzcevoNorEBcViwwdZYN1ZYB2YNEs32eqgcqimh3d6E5vgh1wuTw6PdnrkZ1wnQ2T2p1wdXtRsiOMyiN1imrrUt1RHrEDn6D+yet8V45D/ZN0zU7V7jqiXtGRjYKx2xmJvZLseA8f0QAZbaTUvB6Jv7YBE9Vsu3a95JHLAxdP/kXPESV+kZSLW62Zlr5OfzaNgnyxyMBRtsNvthb+UxARkfmgxC8i4hklfhERzyjxi4h4RolfRMQzSvwiIp5R4hcR8YwSv4iIZ4LagaBaSEtWSkSkxdV1LVyrtviDRh/OuWdms/7R/PA5dt/jV+zNr0eLxF+XVk38IiIyT5T4RUQ8k8bEv77ZFWgin2MHv+NX7P6acfytenBXRETmSRpb/CIiMoXUjMfvnLsEuJP43hZ3m9ltTa7SgnLOPQccBMpAyczOam6N5o9z7h7gcmC3mZ2RlL0S+EfgJOA5wJnZy82q43yaJP5PAB8EXkoW+wsz+2Zzajh/nHPqjemtAAACY0lEQVTHA18GlgEVYL2Z3enD9z9F7J9ght99Klr8zrkM8EXgUuC3gNXOud9qbq2a4s1mdmaak35iA3DJmLKPAo+Z2WnAY8l0Wm1gfPwAdyTf/5lpTPqJEvCnZnY6cB7w4eR/3Yfvf7LYYYbffSoSP3AO8KyZbTezYeB+YFWT6yTzxMyeBPaOKV4FfCl5/SXgigWt1AKaJH4vmNkOM/tR8vogsA1YgQff/xSxz1haEv8K4IWa6W4a/ECOYhHwbefcM865Nc2uTBMcZ2Y7IP4HAZY2uT7NcINz7qfOuXucc8c0uzLzzTl3EvAG4Gk8+/7HxA4z/O7TkvgnumLNt9OVzjez3yHu7vqwc+6CZldIFtRfA6cAZwI7gNubW5355ZwrAF8F/qOZHWh2fRbSBLHP+LtPS+LvBo6vmV4J9DSpLk1hZj3J827gQeLuL5/scs79BkDyvLvJ9VlQZrbLzMpmVgH+lhR//865NuLE9xUzeyAp9uL7nyj2Rr77tCT+zcBpzrmTnXPtwFXAxibXacE45/LOuUXV18DFwL80t1YLbiNwTfL6GuDrTazLgqsmvcS7Sen375wLgP8FbDOzz9fMSv33P1nsjXz3qbmAyzn3DuC/E5/OeY+Z3drkKi0Y59yriVv5EJ+i+w9pjt85dx9wIVAEdgHrgK8BBpwAPA9caWapPAA6SfwXEv/Uj4hPZ/xQtc87TZxzvwd8F9hKfEojwF8Q93Wn+vufIvbVzPC7T03iFxGR+qSlq0dEROqkxC8i4hklfhERzyjxi4h4RolfRMQzSvwiIp5R4hcR8YwSv4iIZ/4/1PRDk6UyEtIAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f98d46be358>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "model.fit([data_pairs[:, 0], data_pairs[:, 1]], \n",
    "                labels, \n",
    "                batch_size       = 24, \n",
    "                verbose          = 0, \n",
    "                epochs           = 25, \n",
    "                callbacks        = [tut_func.PlotLosses()], \n",
    "                validation_split = 0.1);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Evaluate by Example\n",
    "\n",
    "The following function calculated the distances between a given query track and all other tracks of the collection. The result is a list of distances where the smallest distance coresponds with the most similar track. The list is sorted descendingly and the top-ten similar tracks are presented below the information of the query track. The Spotify playlist we created at the beginning will also be updated with the query results. Thus, you can listen to it in your Spotify client."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "def similar(model, query_idx):\n",
    "    \n",
    "    print(metadata.iloc[query_idx])\n",
    "    \n",
    "    # calclulate predicted distances between query track and all others\n",
    "    res = [model.predict([feature_data[[query_idx]], feature_data[[i]]]) for i in range(feature_data.shape[0])]\n",
    "\n",
    "    # reshape\n",
    "    res = np.array(res)\n",
    "    res = res.reshape(res.shape[0])\n",
    "\n",
    "    # get sorted indexes in ascending order (smallest distance to query track first)\n",
    "    si = np.argsort(res)\n",
    "    \n",
    "    # output filter\n",
    "    display_cols = [\"artist_name\", \"title\", \"album_name\", \"year\",\"playlist\"]\n",
    "    \n",
    "    return metadata.loc[si, display_cols][:10]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's check the results for some individual tracks by supplying the index to our dataset:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "track_id                                         637McrDyDQ9CffkFuZGRpq\n",
      "artist_name                                                       KSHMR\n",
      "title                                                           Kolkata\n",
      "album_name                                                      Kolkata\n",
      "label                                                            Dharma\n",
      "duration                                                         206250\n",
      "popularity                                                           54\n",
      "year                                                               2017\n",
      "genres                [big room, deep big room, edm, electro house, ...\n",
      "playlist                                                      clubbeats\n",
      "featurespace_index                                                   33\n",
      "Name: 33, dtype: object\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>artist_name</th>\n",
       "      <th>title</th>\n",
       "      <th>album_name</th>\n",
       "      <th>year</th>\n",
       "      <th>playlist</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>33</th>\n",
       "      <td>KSHMR</td>\n",
       "      <td>Kolkata</td>\n",
       "      <td>Kolkata</td>\n",
       "      <td>2017</td>\n",
       "      <td>clubbeats</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>Hardwell</td>\n",
       "      <td>Get Low</td>\n",
       "      <td>Get Low</td>\n",
       "      <td>2018</td>\n",
       "      <td>clubbeats</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>34</th>\n",
       "      <td>Garmiani</td>\n",
       "      <td>Fogo</td>\n",
       "      <td>Fogo</td>\n",
       "      <td>2017</td>\n",
       "      <td>clubbeats</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>79</th>\n",
       "      <td>Sander van Doorn</td>\n",
       "      <td>Riff - SvD x David Tort Remix</td>\n",
       "      <td>Riff (SvD x David Tort Remix)</td>\n",
       "      <td>2017</td>\n",
       "      <td>clubbeats</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Don Diablo</td>\n",
       "      <td>Momentum</td>\n",
       "      <td>Momentum</td>\n",
       "      <td>2017</td>\n",
       "      <td>clubbeats</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>70</th>\n",
       "      <td>Stadiumx</td>\n",
       "      <td>Spacebird</td>\n",
       "      <td>Spacebird</td>\n",
       "      <td>2017</td>\n",
       "      <td>clubbeats</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49</th>\n",
       "      <td>Calvin Logue</td>\n",
       "      <td>Warrior</td>\n",
       "      <td>Warrior EP</td>\n",
       "      <td>2018</td>\n",
       "      <td>clubbeats</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>41</th>\n",
       "      <td>Tom &amp; Jame</td>\n",
       "      <td>In The Air</td>\n",
       "      <td>In The Air</td>\n",
       "      <td>2017</td>\n",
       "      <td>clubbeats</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>38</th>\n",
       "      <td>Ummet Ozcan</td>\n",
       "      <td>Krypton</td>\n",
       "      <td>Krypton</td>\n",
       "      <td>2018</td>\n",
       "      <td>clubbeats</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>36</th>\n",
       "      <td>Hardwell</td>\n",
       "      <td>Safari</td>\n",
       "      <td>Safari</td>\n",
       "      <td>2018</td>\n",
       "      <td>clubbeats</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         artist_name                          title  \\\n",
       "33             KSHMR                        Kolkata   \n",
       "16          Hardwell                        Get Low   \n",
       "34          Garmiani                           Fogo   \n",
       "79  Sander van Doorn  Riff - SvD x David Tort Remix   \n",
       "6         Don Diablo                       Momentum   \n",
       "70          Stadiumx                      Spacebird   \n",
       "49      Calvin Logue                        Warrior   \n",
       "41        Tom & Jame                     In The Air   \n",
       "38       Ummet Ozcan                        Krypton   \n",
       "36          Hardwell                         Safari   \n",
       "\n",
       "                       album_name  year   playlist  \n",
       "33                        Kolkata  2017  clubbeats  \n",
       "16                        Get Low  2018  clubbeats  \n",
       "34                           Fogo  2017  clubbeats  \n",
       "79  Riff (SvD x David Tort Remix)  2017  clubbeats  \n",
       "6                        Momentum  2017  clubbeats  \n",
       "70                      Spacebird  2017  clubbeats  \n",
       "49                     Warrior EP  2018  clubbeats  \n",
       "41                     In The Air  2017  clubbeats  \n",
       "38                        Krypton  2018  clubbeats  \n",
       "36                         Safari  2018  clubbeats  "
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "similar(model, 33)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "# Using LSTMs to train on Sequential Data\n",
    "\n",
    "This is again identical to part 2. We just use the improved ground-truth data which we have created from the genre tags using tag-similarity metrics."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "def aggregate_features_sequential(seq_data, track_data, len_segment_frames, len_segment_sec, m_data, with_year=False, with_popularity=False):\n",
    "    \n",
    "    # sequential data\n",
    "    segments = seq_data[\"segments\"]\n",
    "    sl       = len(segments)\n",
    "    \n",
    "    mfcc              = np.array([s[\"timbre\"]            for s in segments])\n",
    "    chroma            = np.array([s[\"pitches\"]           for s in segments])\n",
    "    loudness_max      = np.array([s[\"loudness_max\"]      for s in segments]).reshape((sl,1))\n",
    "    loudness_start    = np.array([s[\"loudness_start\"]    for s in segments]).reshape((sl,1))\n",
    "    loudness_max_time = np.array([s[\"loudness_max_time\"] for s in segments]).reshape((sl,1))\n",
    "    duration          = np.array([s[\"duration\"]          for s in segments]).reshape((sl,1))\n",
    "    confidence        = np.array([s[\"confidence\"]        for s in segments]).reshape((sl,1))\n",
    "        \n",
    "    # concatenate sequential features\n",
    "    sequential_features = np.concatenate([mfcc, chroma, loudness_max, loudness_start, \n",
    "                                          loudness_max_time, duration, confidence], axis=1)\n",
    "\n",
    "    # calculate length of segment (in ms)\n",
    "    length_of_track        = track_data[0][\"duration_ms\"] / 1000.\n",
    "    length_of_segment      = length_of_track / len(segments)\n",
    "    num_segments_for_n_sek = int(np.round(len_segment_sec / length_of_segment))\n",
    "    \n",
    "    # select a random lstm-input-segment from the aggregated feature data\n",
    "    offset  = np.random.randint(0, sl - num_segments_for_n_sek)\n",
    "    segment = sequential_features[offset:(offset+num_segments_for_n_sek),:]\n",
    "    \n",
    "    # re-scale segment length to desired length (in seconds)\n",
    "    x  = np.arange(segment.shape[0])\n",
    "    y  = np.arange(segment.shape[1])\n",
    "    xx = np.linspace(x.min(),x.max(),len_segment_frames)\n",
    "\n",
    "    newKernel = RectBivariateSpline(x,y,segment, kx=2,ky=2)\n",
    "    segment   = newKernel(xx,y)\n",
    "        \n",
    "    # track-based data\n",
    "    track_features = [track_data[0][\"acousticness\"],     # acoustic or not?\n",
    "                      track_data[0][\"danceability\"],     # danceable?\n",
    "                      track_data[0][\"energy\"],           # energetic or calm?\n",
    "                      track_data[0][\"instrumentalness\"], # is somebody singing?\n",
    "                      track_data[0][\"liveness\"],         # live or studio?\n",
    "                      track_data[0][\"speechiness\"],      # rap or singing?\n",
    "                      track_data[0][\"tempo\"],            # slow or fast?\n",
    "                      track_data[0][\"time_signature\"],   # 3/4, 4/4, 6/8, etc.\n",
    "                      track_data[0][\"valence\"]]          # happy or sad?\n",
    "    \n",
    "    if with_year:\n",
    "        track_features.append(int(m_data[\"year\"]))\n",
    "        \n",
    "    if with_popularity:\n",
    "        track_features.append(int(m_data[\"popularity\"]))\n",
    "        \n",
    "    \n",
    "    return segment, track_features\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now lets prepare the input features. We will be using the following parameters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "# number of input-vectors for the LSTM\n",
    "len_segment_frames = 24\n",
    "\n",
    "# number of seconds these vectors describe\n",
    "len_segment_sec    = 6."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "execute the function to prepare the data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "sequential_features.shape: (1060, 24, 29)\n",
      "trackbased_features.shape: (1060, 11)\n"
     ]
    }
   ],
   "source": [
    "sequential_features = []\n",
    "trackbased_features = []\n",
    "\n",
    "for i, (_, spotify_data) in enumerate(raw_track_data):\n",
    "    \n",
    "    _, _, _, f_sequential, f_trackbased = spotify_data\n",
    "    \n",
    "    seq_feat, track_feat = aggregate_features_sequential(f_sequential, \n",
    "                                                         f_trackbased, \n",
    "                                                         len_segment_frames,\n",
    "                                                         len_segment_sec,\n",
    "                                                         metadata.loc[i],\n",
    "                                                         with_year=True,\n",
    "                                                         with_popularity=True)\n",
    "    \n",
    "    sequential_features.append(seq_feat)\n",
    "    trackbased_features.append(track_feat)\n",
    "    \n",
    "sequential_features = np.asarray(sequential_features)\n",
    "trackbased_features = np.asarray(trackbased_features)\n",
    "\n",
    "print(\"sequential_features.shape:\", sequential_features.shape)\n",
    "print(\"trackbased_features.shape:\", trackbased_features.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Normalization\n",
    "\n",
    "As in the examples before, we need to normalize the data before using it with the neural networks"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "# standardize sequential_features\n",
    "rows, x, y = sequential_features.shape\n",
    "sequential_features  = sequential_features.reshape(rows, (x * y))\n",
    "sequential_features -= sequential_features.mean(axis=0)\n",
    "sequential_features /= sequential_features.std(axis=0)\n",
    "sequential_features  = sequential_features.reshape(rows, x, y)\n",
    "\n",
    "# standardize trackbased_features\n",
    "trackbased_features -= trackbased_features.mean(axis=0)\n",
    "trackbased_features /= trackbased_features.std(axis=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create Data-Pairs\n",
    "\n",
    "Also like before, we have to prepare the paired training examples. We continue with the last approach presented in this part, which is based on the mutual similarities and includes identities. Each pair contains the sequential and track-based inputs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "def create_pairs_with_sims_and_identity_segments(sequential_features, trackbased_features, metadata, num_pairs_per_track, sims):\n",
    "    \n",
    "    data_pairs_seq   = []\n",
    "    data_pairs_track = []\n",
    "    labels           = []\n",
    "    \n",
    "    for row_id, q_track in metadata.sample(frac=1).iterrows():\n",
    "        \n",
    "        query_segment      = sequential_features[[metadata.loc[row_id].featurespace_index]][0]\n",
    "        query_track_vector = trackbased_features[[metadata.loc[row_id].featurespace_index]][0]\n",
    "        \n",
    "        data_pairs_seq.append([query_segment, query_segment])\n",
    "        data_pairs_track.append([query_track_vector, query_track_vector])\n",
    "        labels.append(1)\n",
    "        \n",
    "        for _ in range(num_pairs_per_track):\n",
    "            \n",
    "            # search similar and dissimilar examples\n",
    "            pos_example = metadata[metadata.playlist == q_track.playlist].sample(1)\n",
    "            neg_example = metadata[metadata.playlist != q_track.playlist].sample(1)\n",
    "\n",
    "            # create feature pairs\n",
    "            data_pairs_seq.append([query_segment, sequential_features[[pos_example.featurespace_index]][0]])\n",
    "            data_pairs_track.append([query_track_vector, trackbased_features[[pos_example.featurespace_index]][0]])\n",
    "            labels.append(np.max([0, sims.iloc[row_id, pos_example.index].values[0] - 0.1]))\n",
    "\n",
    "            data_pairs_seq.append([query_segment, sequential_features[[neg_example.featurespace_index]][0]])\n",
    "            data_pairs_track.append([query_track_vector, trackbased_features[[neg_example.featurespace_index]][0]])\n",
    "            labels.append(np.max([0, sims.iloc[row_id, neg_example.index].values[0] - 0.1]))\n",
    "\n",
    "    return np.array(data_pairs_seq), np.array(data_pairs_track), np.asarray(labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Execute the function to prepare the results:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_pairs_seq, data_pairs_track, labels = create_pairs_with_sims_and_identity_segments(sequential_features,\n",
    "                                                                                        trackbased_features,\n",
    "                                                                                        metadata_aligned, \n",
    "                                                                                        10,\n",
    "                                                                                        tag_based_track_similarities)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The Siamese Network Architecture using LSTMs\n",
    "\n",
    "The network architecture consists again of shared \"twin\" layers. But, now two layers are shared. First the fully connected layers which train on the track-based features. Second, a bi-directional LSTM which trains on the re-scaled sequential data provided by the Spotify API. The network thus now takes four inputs. The outputs of the fully connected layer and the LSTM are finally joined and the Eucledian distance of each twin is calculated which is the final output of the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "def create_siamese_network_with_lstm(data_pairs_seq, data_pairs_track):\n",
    "\n",
    "    # sequential input\n",
    "    input_seq_left  = Input(shape=data_pairs_seq[:, 0].shape[1:])\n",
    "    input_seq_right = Input(shape=data_pairs_seq[:, 0].shape[1:])\n",
    "\n",
    "    # track-based input\n",
    "    input_track_left  = Input(shape=data_pairs_track[:, 0].shape[1:])\n",
    "    input_track_right = Input(shape=data_pairs_track[:, 0].shape[1:])\n",
    "\n",
    "    # shared bi-directional LSTM layer for the sequential features\n",
    "    bdlstm = Bidirectional(LSTM(29, return_sequences=False, activation=\"selu\"))\n",
    "\n",
    "    # shared fully connected layers for the track-based features\n",
    "    shared_fc_1 = Dense(11, activation=\"selu\")\n",
    "    shared_fc_2 = Dense(11, activation=\"selu\")   \n",
    "\n",
    "    # left twin\n",
    "    seq_resp_left   = bdlstm(input_seq_left)\n",
    "    track_resp_left = shared_fc_1(shared_fc_2(input_track_left))\n",
    "    left_twin       = concatenate([seq_resp_left, track_resp_left], axis=1)\n",
    "\n",
    "    # right twin\n",
    "    seq_resp_right   = bdlstm(input_seq_right)\n",
    "    track_resp_right = shared_fc_1(shared_fc_2(input_track_right))\n",
    "    right_twin       = concatenate([seq_resp_right, track_resp_right], axis=1)\n",
    "\n",
    "    # calc difference\n",
    "    distance = Lambda(tut_func.euclidean_distance,\n",
    "                      output_shape=lambda x: x[0])([left_twin, right_twin])\n",
    "\n",
    "    return Model([input_seq_left, input_seq_right, input_track_left, input_track_right], distance)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Prepare the model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "# define the model\n",
    "model_rnn = create_siamese_network_with_lstm(data_pairs_seq, data_pairs_track)\n",
    "\n",
    "# define the optimizer\n",
    "opt = Nadam(lr=0.001)\n",
    "\n",
    "# compile the model\n",
    "model_rnn.compile(loss      = tut_func.contrastive_loss, \n",
    "                  optimizer = opt)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Train the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW4AAAD8CAYAAABXe05zAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3XmYXFWd//H3qare0xupJJ2FAAlhh4QRJD8DoglhETWAj3dYnB/6QyLP4xJkFEWdgXmQwd88Awwuo7+MyjaoHCVI2AYiiBCUsEkSYoQnsga6QzqddHeW3qrO74+q6q7eqzuprnvrfl7PE7q67q1b39OXfPrk1D33GOccIiISHJFCFyAiImOj4BYRCRgFt4hIwCi4RUQCRsEtIhIwCm4RkYBRcIuIBIyCW0QkYBTcIiIBE8vTcTUdU0Rk7EwuO+UruHnvvffG9bp4PE5zc/MBriYYwtx2CHf71fZwth362j9jxoycX6OhEhGRgMmpx+153ptAO5AAeqy1J+WzKBERGd5Yhko+aq0N779nRER8Im9j3CISLs45Ojo6SCaTGJPTZ2wAbNu2jc7OzjxW5h/OOSKRCOXl5WP6GQ2Ua3A74DHP8xzw/6y1K8f9jiJSlDo6OigpKSEWG1t/MBaLEY1G81SV//T09NDR0UFFRcW4j2FyWUjB87wZ1tr3PM+bCqwBvmytfWrAPsuB5QDW2g90dXWNq6BYLEZPT8+4Xht0YW47hLv9xdD2bdu2UVZWVugyAqGzs5Np06YBfee+tLQUcrwcMKfgzuZ53nXAbmvtv4+wm9PlgGMX5rZDuNtfDG3fu3cvlZWVY35dMfzSGqvsn9WAywFzCu5RLwf0PK/K87zqzGPgTOCVcVc8DOccyQfvofPPzx7oQ4uIFJVcruOeBqz1PG898BzwkLX2fw50IcYY3GO/pfPFPx7oQ4tISMybN6/QJUyIUT9FsNa+DsyfgFqgpo7krp0T8lYiIkHlr5mTNbUkd7UUugoRCTjnHNdffz2LFy9myZIl3H///UDqA9QLLriApUuXsnjxYtatW0cikeDKK6/s3XflSv9fNOev67hr6kg2vVvoKkRkPyV/9V+4d97IbV9jyOUiCXPwYUQuvDynYz788MNs2rSJNWvW0NLSwsc+9jEWLlzIfffdx+mnn86KFStIJBLs27ePTZs20dTUxBNPPAFAa2trTu9RSL7qcZuaevW4RWS/Pffcc5x33nlEo1GmTJnCwoULWb9+PQsWLMBay0033cTmzZuZNGkSs2fP5u233+Y73/kOv//976muri50+aPyXY/b7WnHdXdjSkoKXY2IjFOuPWPIz+WAw/XgFy5cyL333svjjz/OihUruOKKK/j0pz/NmjVrePLJJ7n99tt54IEHuPnmmw9oPQear3rc1NSlvrbvKmwdIhJoCxcuZPXq1SQSCXbs2MG6detYsGABW7duJR6Pc8kll3DhhReyceNGWlpaSCaTnHvuuXz9619n48aNhS5/VL7qcZuautQKDG274KAphS5HRALqnHPO4cUXX2Tp0qUYY/j2t7/N1KlTsdbyk5/8hFgsRlVVFbfeeiuNjY1cddVVJJNJAK655poCVz+6Mc+czNG4Zk66N14j+a9fI/Llf8KccHIeyvK3Ypg9tz/C3P5iaLtmTuYu7zMnJ1R6qMS16lpuEZHh+DK4adMYt4jIcHwV3KakFFNZBe3+v45SRKRQfBXcAJHag9TjFhEZgf+Cu/4gnIJbRGRY/gvu2oNAH06KiAzLf8Fdp6ESEZGR+DO49+7G9XQXuhQRKWIj3bv7nXfeYfHixRNYzdj4MLjrUw/adGWJiMhQfDXlHSBSNzn1oG0nHBQvbDEiMi4/fWEbb+zsyGlfk+NtXQ+rL+fzJ00bdvsNN9zAzJkz+exnPwvATTfdhDGGZ599ltbWVnp6erj66qs566yzcqoro6Ojg2uuuYYNGzYQjUa59tprWbRoEa+++ipXXXUVXV1dOOdYuXIlDQ0NfOELX6CxsZFkMsmKFStYtmzZmN4vF74L7mjdQakHGucWkTFYtmwZ1157bW9wP/DAA9x9991cfvnlVFdX09LSwic+8QnOPPNMjMlpZjkAt99+OwCPP/44W7Zs4aKLLuLpp5/mrrvu4rLLLuOCCy6gq6uLRCLBE088QUNDA3fddRcAbW1tB7qZgA+DO5IObte2K7dJ+yLiOyP1jAc6UPcqOe6442hubqapqYkdO3ZQW1vL1KlTue6661i3bh3GGJqamti+fTtTp07N+bjPP/88n/vc5wA4/PDDmTVrFq+//jof+MAH+P73v09jYyPnnHMOc+bM4aijjuL666/nhhtu4IwzzuCUU07Z73YNxX9j3LXqcYvI+Jx77rk89NBDrF69mmXLlrFq1Sp27NjBI488wpo1a4jH43R2do7pmMMN45x//vncdtttlJeXc8kll7B27Vrmzp3LI488wlFHHcWNN97ILbfcciCaNYjvgtuUlUF5hYJbRMZs2bJl3H///Tz00EOce+65tLe3E4/HKSkp4ZlnnmHr1q1jPuYpp5zCfffdB8Df/vY33n33XebOnctbb73FIYccwmWXXcbSpUvZvHkzTU1NVFRU8KlPfYorrrgib/f29t1QCQA19QpuERmzI488kj179tDQ0MC0adO44IILuPTSSznnnHM49thjOfzww8d8zEsvvZRvfvObLFmyhGg0yi233EJZWRmrV69m1apVxGIxpk6dyle/+lXWr1/Pd7/7XYwxlJSUcOONN+ahlT67Hzek7k277eufh2iU6NduOMBl+Vsx3JN5f4S5/cXQdt2PO3fFdT/ujJo69bhFRIbhy6ESU1OHe9X/676JSLBt3ryZr3zlK/2eKysr48EHHyxQRbnxZXBTUwd72nE93ZiYVnsXCYI8Dbvm1dFHH82aNWsm/H3392flz6GS2sxKOJr2LhIUkUgkdGPV49HT00Mksn/R68sed+9q7+27NO1dJCDKy8vp6Oigs7NzTDMTy8rKxnxtdVA554hEIpSXl+/XcXwZ3FRr7UmRoDHGUFFRMebXFcMVNRPNn0MlmdXeFdwiIoP4NLgzt3ZVcIuIDOTL4O6d9q4lzEREBsl5jNvzvCjwAvCutfbj+SspTZNwRESGNJYe9wpgc74KGaSmTmPcIiJDyCm4Pc+bBZwL/DS/5WRRj1tEZEi59rj/A7gaSOaxln5MTV3qOm4REeln1DFuz/M+DrxvrX3R87yPjLDfcmA5gLWWeHx8E2disRjxeJzdDTPZs7udyXV1mJg/Lzc/0DJtD6swt19tD2fbYXztH/W2rp7n3Qj8A9ADlAM1wCpr7WdGeNl+3da1ubmZ5B/+B/ff/0nk327D1E8e17GCJuwTEcLcfrU9nG2H8d3WddSurLX2GuAagHSP+2ujhPYB0TvtvW0XhCS4RURy4cvruIHe2ZP6gFJEpL8xDR5ba58EnsxLJQP1TnvfqdXeRUSyqMctIhIwvg1uU1YOZVrtXURkIN8GNwA1tQpuEZEBfB7cmvYuIjKQv4O7tl53CBQRGcDXwa1p7yIig/k6uKmug93tOC1AKiLSy9/BnbkkcLdWexcRyfB1cBtdyy0iMoivg5va9NqTrQpuEZEMfwe3VnsXERkkEMGtoRIRkT6+Du7UtPdyBbeISBZfBzegtSdFRAYIRHC7Ns2eFBHJCERwq8ctItLH98FtFNwiIv34PripqYM9mvYuIpIRjOB2TtPeRUTSfB/cpiY9e1LDJSIiQACCW5NwRET6C0xwa9q7iEhKYIJbPW4RkRTfB7cpr4DSMt0hUEQkzffBDaRu76oet4gIEJTgrqnDae1JEREgKMFdrdmTIiIZgQhuTXsXEekTiOCmpg52t+ESiUJXIiJScMEI7tr0tPd2TXsXEQlEcGu1dxGRPoEIbk3CERHpExttB8/zyoGngLL0/r+x1l6b78L6yZr2bib0jUVE/CeXHncnsNhaOx9YAJzted7C/JY1QG+PW0uYiYiM2uO21jpgd/rbkvQfl8+iBilLT3vXUImIyOjBDeB5XhR4ETgc+JG1dl1eqxrAGKO1J0VE0oxzuXeePc+rA+4DvmytfWXAtuXAcgBr7Qe6urrGVVAsFqNniGXKWr5xOaaikvrrbh3XcYNguLaHRZjbr7aHs+3Q1/7S0lIgt4/xcupxZ1hrd3me9yRwNvDKgG0rgZXpb11zc/NYDt0rHo8z1GsTlZNge9OQ24rFcG0PizC3X20PZ9uhr/0zZszI+TWjfjjped6UdE8bz/MqgDOAv467ynEyNbpDoIgI5Nbjng7ckR7njgDWWvtgfssaQta0dxONTvjbi4j4RS5XlWwATpyAWkbWu9p7W+r+3CIiIRWMmZNo2ruISEZggrt3Ek6rJuGISLgFJ7hrtdq7iAgEKbgzPW4tYSYiIRec4C6rgNJSjXGLSOgFJriNMVp7UkSEAAU3kFrtXR9OikjIBSu4azV7UkQkUMGt1d5FRAIW3Klp7+24pFZ7F5HwCl5wu2Rq2ruISEgFKrh7p723arhERMIrUMFNTfrmUhrnFpEQC1hwa9q7iEggg1s9bhEJs2AFd3kFlGjau4iEW6CCu2+1d82eFJHwClRwA6lp7+pxi0iIBTK4NVQiImEWuODWtHcRCbvABTc1ddDepmnvIhJawQxuTXsXkRALXHCbWs2eFJFwC1xwU61JOCISbsELbk17F5GQC2xw6w6BIhJWwQvuikqIlWioRERCy1fB7Zyjqyc54j7GGK09KSKh5pvg7uxJ8vnf/o1fvLR19J017V1EQsw3wV0Wi1BbHuO5t3IIZM2eFJEQ801wAyxoqOSVpnb2do88K9LoDoEiEmL+Cu7pVSSSjk3b9o28Y7WmvYtIeMVG28HzvIOBO4EGIAmstNbemo9ijp5SQVkswstNezh51qThd6zNTHtv77s8UEQkJHLpcfcA/2itPRpYCHzR87xj8lFMSTTCgpk1vNy4Z8T9jJYwE5EQGzW4rbWN1tqX0o/bgc3AzHwVdPLsOra2ddG8t3v4nRTcIhJiYxrj9jzvUOBEYF1eqgFOPjh1E6kRe9290971AaWIhM+oY9wZnudNAu4FrrTWDrqnqud5y4HlANZa4vH4uAqaFo1yUGUJf21JcOEpQx8jWVHOdqAq0U3VON/Hj2Kx2Lh/bsUgzO1X28PZdhhf+3MKbs/zSkiF9t3W2lVD7WOtXQmsTH/rmpubx1RIRjwe54SpFTz3Vgvvb99OxJhB+zjnIFbCnsZ32TfO9/GjeDzOeH9uxSDM7Vfbw9l26Gv/jBkzcn7NqEMlnucZ4GfAZmvtzftRX87mT6+itTPBmzs7h9zet9q7xrhFJHxy6XEvAv4B2Oh53svp575lrX04X0XNb6gE4OWmPcw5qHzonTTtXURCatTgttauBQaPV+TR5MoSZteWsr5xDxccM3nonWrqoCW8/7wSkfDy1czJbPOnV7Hp/X10DnO3QFNbD+3qcYtI+Pg2uBc0VNGddGzePsz09+o6aG/FJUe+DayISLHxbXAfN62SWGSE67lr6iCZhD3tE1uYiEiB+Ta4y2MRjppSyctNIwQ3QKsm4YhIuPg2uCF1m9c3dnayq6Nn0Dbdr0REwsrfwT29CoANTXsHb6zVau8iEk6+Du459eVMKo3w56HGudXjFpGQ8nVwRyOGExqqWN+4JzXNPVtFFcRiCm4RCR1fBzfAidOr2LGvh61tXf2e75v2rg8nRSRcfB/cvdPfhxouqda0dxEJH98H97RJpUyvLmH9UJcF1tZrqEREQsf3wQ2pWZQbt+2lO9F/nDu12ntrgaoSESmMQAT3/OlVdPQ4XmseMP29pg7ad2nau4iESiCC+/hplUQMg2dRatq7iIRQIIJ7UmmUeZPLB39AqWu5RSSEAhHckJpFuaWlg92did7nzIxDAHB/3VCoskREJlxwgruhiqSDjdv6pr+bmbPh0Hm4px8bPEFHRKRIBSa4j4hXUBEbPP3dnLYU3n0L3txSoMpERCZWYII7FjEcN61y0PXc5uQPQ2kZbu1jBapMRGRiBSa4ARZMr6RpdzdN7X3T301FJeakU3HPPYXr7ChgdSIiEyNYwd2Qus3rwMsCzalLoWMf7oVnClGWiMiEClRwz6wpZXJljJcbB9yf+/CjoWGmhktEJBQCFdzGGE6cXsWGbXtIJF2/582pZ8KWzbjGdwpYoYhI/gUquAHmN1SxpyvJlpb+49nmf30UolHc2t8VqDIRkYkRwOBO3eZ1/cDLAmvqYP4HcX96AtfTXYjSREQmROCCu7Y8xpz6siFXf4+ceia0t8KG5wtQmYjIxAhccENq+vurzfvY1z3groDHLoD6OMmn1xSmMBGRCRDI4J7fUEVPEja93//qEhOJYhYtgU0v4Vq2F6g6EZH8CmRwHzO1gtKoGXI5M/OhJeAc7o+PF6AyEZH8C2Rwl0YjHDO1ctB9SwDMlAY4ej5u7e+0wIKIFKVABjfAgoZKtrZ10bx38BUk5rQzYcf78Nf1BahMRCS/ghvc01PT3wdeFghgFiyEqmpd0y0iRSk22g6e5/0c+DjwvrX2uPyXlJtD6sqoLY/yctNelsyt67fNlJRgFn4E94dHcLvbMJNqClSliMiBl0uP+3bg7DzXMWYRY5jfUMX6pj0kh1hEwZy6FHp6cM8+OfHFiYjk0ajBba19CmiZgFrGbEFDJa0dCV5rHnw7VzPrUDjsCNzaNVodR0SKSmDHuAEWHlzNpNII9pXmIbebU9Or47zx2gRXJiKSP6OOcefK87zlwHIAay3xeHx8BcViOb82DnzmpC5+8se3aOwq5fgZ/ceyk2efx3b7M8peeJqaDy4aVz0TaSxtL0Zhbr/aHs62w/jaf8CC21q7EliZ/tY1Nw/dCx5NPB5nLK/96MFl/LI8yn8+tYXrz5g9aLs5aRH7nlpD5ycvwZRXjKumiTLWthebMLdfbQ9n26Gv/TNmzMj5NYEeKgEoj0X49LGT2bBtLxuGuPGUOXUpdO7DvbC2ANWJiBx4owa353m/BP4EHOl53lbP8y7Lf1ljc9a8OiZXxPjv9c2DP4icezQ0zMKt1Y2nRKQ4jDpUYq29aCIK2R+l0Qje8ZP58XPbePG9PZw0c1LvNmMM5rSluF/fhmt8BzP94AJWKiKy/wI/VJJxxtw6GiaVcPf67YN63WZhZnUc9bpFJPiKJrhjEcPfHx/n9Z2d/Omd9n7bUqvjnIL7o1bHEZHgK5rgBjj90Bpm1ZTyiw3N/RYTBoicthR2t8H65wpUnYjIgVFUwR2NGC46Ic47rV08/VZb/43HLICD4iQ1XCIiAVdUwQ3wodnVHFZfxq82NtOT1es2kSjmQ2fApj/jdmh1HBEJrqIL7ogxXHxCnMb2bn7/emu/bWbREgDcM7rdq4gEV9EFN8DJMycxb3I592xspjvRtwqOiU9LrY7z9KO4fXtHOIKIiH8VZXAbY/jM/Cls39vDY1v697ojn7wYWnfhfnNbgaoTEdk/RRncAPMbKjluagW/fqWZzp6sXvfcozBLl+GeehT3lz8XsEIRkfEp2uA2xnDx/Cns7Ejw8Gs7+29bdjE0zCR5xw81ZCIigVO0wQ1w7NRKTpxexb1/aWFvd6L3eVNaRuSzK2DnDg2ZiEjgFHVwA1wyP057Z4IH/jqg1z33KMyZGjIRkeAp+uCeN7mCU2ZN4rebW2jvTPTbZpZdAg2zSN7xAw2ZiEhgFH1wA1x8Qpx93Ul+u7n/0pmmpJTI51bAzhbcr39eoOpERMYmFMF9aH05px5SzYOvtrCro6ffNjPnSMyZ5+Gefgy3SUMmIuJ/oQhugAtPiNOVcNy7acegbamrTGaRvPMHuL2DV9EREfGT0AT3rJoyPnpYLY+8tosde/vf2rXfkImuMhERnwtNcAP8/fGTcTju2ThErzt7yOSVlwpQnYhIbkIV3NMmlXL2vHoe3bKLO/78PsmBK+UsuximH0zyzh9qyEREfCtUwQ3wf/5uKmfPq2PVX1r497Xv9Z8Onxky2aWrTETEv0IX3NGI4YqTp3HpiVN45u12/unxd2jNutLEHHYE5qzzcWvX4F55sYCViogMLXTBDan7mFxwzGSuPm0Gb+zs4OpH3+Ldtq6+7Z+8KD1k8iMNmYiI74QyuDMWza7h+iWz2ded5BuPvsmm91OzJ/sNmdifFbhKEZH+Qh3cAEdNqeDfzjqEmvIY//z4O/zhjdT9u81hR2DOPh/3zO9wGzVkIiL+EfrgBmioLuX/nnkIR8bLufmPjdiNzTjnMJ/QVSYi4j8K7rTqsij/svhgTj+0hrs3NPODZ5voicSIfO5KaNtJ8l++TPI3t+He/htuwGWEIiITKVboAvykJBrhqx+aTkN1Cfds3MH2Pd1848NzqPrSd0j+/mHc71bjHr0Pps3EnHwa5oOnYaYfXOiyRSRkFNwDGGO4+IQpNEwq5YfPNvLNx97inz8yn6nHn4Tb3YZ76U+455/GPXQP7sFfwazDUgF+0qmYKQ2FLl9EQkDBPYzFc2qJV8b43lPv8vVH3+T8Yw6irjxGzeGLqDvudKo726nZtI7YC0/hVt2JW3UnHHYE5oMfxpy0CFM3udBNEJEipeAewQkNVXzvrEP41z9s5baXtg+xx+FUzjmCmiOgtns3Na3bqHl5O7XP30NNzSTKKsspjcUoKUn/KS2htKyEkrJSysrLKCkvo6S8nNKKckoqK6goK6ezs5NorIRoxGCMmfA2i4j/KbhHMbu2jB9/Yg57upK0diZo7eihtTNBW8eAx53lbJ9Uz5b6ObR1JUmQFbqJ9J+Ood6hB9id/tP/l0PEJYg6R9QlibgkERxRkkSdSz9OfTWAwaU+aTapT5xN5nkDhsxzfd8DRNIPMs+nHhsixvU+NlnHJOv1vccxZsA2k7Wt9z/0/Q7KKiDr+GAoKYnR09OTVaPp9z6ZlxljsupIPR8xmTpNvxoj6TeOmMy+fe+fqTdTXOa4mY2929M1Zn7SqcfZbck615kDJBOQSGJcEhIJTDKRepxMQDLruWTqubLSUroTCSLRKCYaJRKLYqKx9OPUV5P1NRKLpd8rXafp+3mRblPvc70nua+92e3KNlRXofeQ9J3f7H17v886R2NRuzdGW9uefq/N/v8x+0dssmvMOj8D+zgDa8sca6zG2neKmtT9//NNwZ0DYwyTyqJMKosys6Z01P2dc+ztTtKVcHQnHF3JJF1dCbo79tG1r5Pujk66Ojvp7uyiu7Obrq5uurp7MJEIHR2dJJKOZNKRSDoSzpF0jkQSEumvSedIOEi4VIA4B8n0V8eA5wY8n3SAMf33JRVKQ33PoO2p/5OTJvPrApwxffsbQ3LQX5C+713WJjdgP4fBmb7nM99njpH9Hr01DXqc+ppM/xVPmr7jJE10tDM3zOP9FWHUC7hGvdo089u/GL1T6AIOmLrEPu743yfm/X0U3HlgjKGqNErVoC2VI74uHo/T3Nycr7IOGOdc6rcBLvWbwDlwyXTCJ1M9StK/GTL7MGCf3t8omeMkqa+vZ2dLS//nM48zr8P1P0a/42del3Xs3uMArqt3u0s6XPq1Lr3d4XCJvmM5HC6Z/nWRfi/jXP/2Z72XyXrOxUow0RIoKcFFY7hYCcSyvkazvpbEqKurp2XbNpLd3bjuLlx3N8nuLujuIdndhevpJtmd+pN63NP7vul37G06mRrp+xH1bs+6lNVlNva7vNVlvyj9Fo7UL06XPhUm62eU/Ys2/R6Dur9Z/eTs7ntaWXkFHR0d6br7zmn6zPSeX5c+b2S3rbd5rq+urGYM+vXrhvhm0OW9buin++2StTHrYaysFPBJcHuedzZwKxAFfmqt/V5eqxJfM33/Nj+gMwFi8TgmVnbgDjgCv316UFtfS3eie/Qdi1BQOix+MupfO8/zosCPgHOAY4CLPM87Jt+FiYjI0HLpL30Q2GKtfd1a2wX8CliW37JERGQ4uQyVzKT/pwdbgVMG7uR53nJgOYC1lng8Pr6CYrFxvzbowtx2CHf71fZwth3G1/5cgnuo4cBBw/bW2pXAysz28Y5ZhXm8K8xth3C3X20PZ9uhr/0zZszI+TW5DJVsBbJvyDELeG+MtYmIyAGSS4/7eWCe53mHAe8CFwIX57UqEREZ1qg9bmttD/Al4FFgc+opuynfhYmIyNByuo7bWvsw8HCeaxERkRyYPC0KoJUGRETGLqe5YflaAceM94/neS/uz+uD/CfMbQ97+9X2wtfhk/bnREuXiYgEjIJbRCRg/BjcK0ffpWiFue0Q7var7eE15vbn68NJERHJEz/2uEVEZAS+WUgh7Pf89jzvTaCd1DInPdbakwpbUf54nvdz4OPA+9ba49LPHQTcAxwKvAl41tqdhaoxn4Zp/3XA5fStX/et9PyJouJ53sHAnUADqUWaVlprbw3D+R+h7dcxxnPvix637vnd66PW2gXFHNpptwNnD3jum8Dj1tp5wOPp74vV7QxuP8At6fO/oBhDO60H+Edr7dHAQuCL6b/rYTj/w7UdxnjufRHc6J7foWKtfQpoGfD0MuCO9OM7gPMmtKgJNEz7Q8Fa22itfSn9uJ3UbTRmEoLzP0Lbx8wvwT3UPb/H1aAAc8Bjnue9mL63edhMs9Y2Qup/cGBqgesphC95nrfB87yfe55XX+hi8s3zvENJLdC4jpCd/wFthzGee78E91AzhsJ2ucsia+3fkRou+qLneR8udEEyoX4MzAUWAI3ATYUtJ788z5sE3Atcaa1tK3Q9E2mIto/53PsluEN/z29r7Xvpr+8D95EaPgqTbZ7nTQdIf32/wPVMKGvtNmttwlqbBP6LIj7/nueVkAquu621q9JPh+L8D9X28Zx7vwR37z2/Pc8rJXXP79UFrmnCeJ5X5XledeYxcCbwSmGrmnCrgUvTjy8F7i9gLRMuE1pp51Ok59/zPAP8DNhsrb05a1PRn//h2j6ec++bCTie530M+A9SlwP+3Fp7Q4FLmjCe580h1cuG1CWavyjm9nue90vgI0Ac2AZcC/wWsMBs4G3g09baovwAb5j2f4TUP5UdqcvhvpAZ8y0mnuedCjwNbCR1SRzAt0iN9Rb1+R+h7RcxxnPvm+Cc9oHlAAAAN0lEQVQWEZHc+GWoREREcqTgFhEJGAW3iEjAKLhFRAJGwS0iEjAKbhGRgFFwi4gEjIJbRCRg/j/oo7Wlzdy6kQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f986f70fef0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "model_rnn.fit([data_pairs_seq[:, 0],  data_pairs_seq[:, 1], \n",
    "               data_pairs_track[:,0], data_pairs_track[:,1]], \n",
    "              labels, \n",
    "              batch_size       = 24, \n",
    "              verbose          = 0, \n",
    "              epochs           = 25,\n",
    "              callbacks        = [tut_func.PlotLosses()], \n",
    "              validation_split = 0.1);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Evaluate by Example\n",
    "\n",
    "Due to the changed input, we also have to adapt the evaluation function to display example result-lists:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def similar_rnn(model, query_idx):\n",
    "    \n",
    "    print(metadata.iloc[query_idx])\n",
    "    \n",
    "    # calclulate predicted distances between query track and all others\n",
    "    res = [model.predict([sequential_features[[query_idx]], sequential_features[[i]], \n",
    "                          trackbased_features[[query_idx]], trackbased_features[[i]]]) \\\n",
    "           for i in range(feature_data.shape[0])]\n",
    "\n",
    "    # reshape\n",
    "    res = np.array(res)\n",
    "    res = res.reshape(res.shape[0])\n",
    "\n",
    "    # get sorted indexes in ascending order (smallest distance to query track first)\n",
    "    si = np.argsort(res)\n",
    "    \n",
    "    # output filter\n",
    "    display_cols = [\"artist_name\", \"title\", \"album_name\", \"year\",\"playlist\"]\n",
    "    \n",
    "    return metadata.loc[si, display_cols][:10]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's check the results for some individual tracks by supplying the index to our dataset:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "track_id                                         637McrDyDQ9CffkFuZGRpq\n",
      "artist_name                                                       KSHMR\n",
      "title                                                           Kolkata\n",
      "album_name                                                      Kolkata\n",
      "label                                                            Dharma\n",
      "duration                                                         206250\n",
      "popularity                                                           54\n",
      "year                                                               2017\n",
      "genres                [big room, deep big room, edm, electro house, ...\n",
      "playlist                                                      clubbeats\n",
      "featurespace_index                                                   33\n",
      "Name: 33, dtype: object\n"
     ]
    }
   ],
   "source": [
    "similar_rnn(model_rnn, 33)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  },
  "notify_time": "5",
  "toc": {
   "toc_cell": false,
   "toc_number_sections": true,
   "toc_threshold": 6,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}