Project 1 - Closing Price Stock Regressor - Version 1.0.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Project 1 - Closing Price Stock Regressor - Version 1.0\n",
    "\n",
    "    Introduction\n",
    "\n",
    "This Jupyter notebook attempts (keyword, attempts) to predict the daily closing price of an inputted stock ticker, given its historical data, some added features through feature engineering, and the current day's opening price. I am going to do my best to explain each detail so that anyone (newbie or professional) can follow along. This model as it stands right now is not very good at predicting the closing price, but that isn't really my intent with this project, but more just to display what I've learned so far, and for others to get started as well if they're interested in a skeleton project to get started. There's always room for improvement but I'm hoping that this will help get someone started on their way if they are looking to get into the field of machine learning.\n",
    "\n",
    "Day traders typically start their mornings by creating a 'watchlist' of stocks before the market's opening bell. The theory behind this project is that, if you know (roughly) where a stock is about to open today pre-market, and have some of its previous days historical data, you might be able to train a regression model to gauge a better idea of where the stock will close that day, giving actionable insight into whether to buy the stock or not.\n",
    "\n",
    "This project is only meant to act as a demonstration of some of my Python programming and machine learning knowledge to date (August, 2019 as of this writing). Everything I've compiled in this project I have pieced together from various sources and it's all things that I have learned from my own research and self-teaching. I have many years of technical analysis knowledge of the stock and Forex markets, but I do not have a degree in Computer Science or any formal education in the field (which may or may not become evident to whomever is reading this). I welcome advice and criticisms to make things more efficient and better so that I can develop as a programmer. If you see anything I've missed the point on, please let me know. I have spent a fair amount of time commenting each section of code to make it easy for a newbie coder to read along and understand what's happening. Experienced coders will be able to just look at each code section and see what's going on, so skim or read, the choice is yours. This should also help to isolate a section in case someone runs into an issue, they will be able to point to where it happened. \n",
    "\n",
    "    DISCLAIMER: This project is merely a compilation of various Python and machine learning skills that I have picked up \n",
    "    over the past few months and years, and acting as a test for me to see if I'm able to CLEARLY explain the various \n",
    "    details and tools used throughout the project so that maybe one day I can show a potential employer my skillset and \n",
    "    secure a job in this field. This tool is not actually meant to be used for stock advisement, it is for academic\n",
    "    and future development purposes only.\n",
    "    \n",
    "    You've been warned. Now enjoy!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The Environment and Installing the Dependencies\n",
    "\n",
    "First thing's first, if you want to run this project, you need to have the appropriate dependencies installed. Everything listed below are the versions of each dependency as of this writing. I'll explain how to install them all at the bottom of the list. Now I can't speak for other versions, but you need to be running ... Python 3.6 on Windows 10 ..., as that's what I'm running on my machine. Here are the dependencies you'll need:\n",
    "\n",
    "    DateTime==4.3\n",
    "    pandas==0.23.0\n",
    "    pandas-datareader==0.7.0\n",
    "    seaborn==0.8.1\n",
    "    matplotlib==2.1.2\n",
    "    numpy==1.14.5\n",
    "    bs4==0.0.1\n",
    "    requests==2.22.0\n",
    "    feature-selector==1.0.0\n",
    "    featuretools==0.9.1\n",
    "    dask==2.2.0\n",
    "    dask-ml==1.0.0\n",
    "    sklearn==0.0\n",
    "    TPOT==0.10.2\n",
    "    ta-lib==0.4.18\n",
    "\n",
    "Again, this is all in Python 3.6 and Windows 10. All of the other modules used in this project (like \"os\" and \"warnings\" should be included in your Python package already. If you run into something I've missed, let me know, but this should be a complete list. There are two ways to install all of these packages:\n",
    "\n",
    "\n",
    "1. Install them all at once using the Pip installer by typing the following command into your command prompt:\n",
    "\n",
    "        pip install DateTime==4.3 pandas==0.23.0 pandas-datareader==0.7.0 seaborn==0.8.1 matplotlib==2.1.2 numpy==1.14.5 bs4==0.0.1 requests==2.22.0 feature-selector==1.0.0 featuretools==0.9.1 dask==2.2.0 dask-ml==1.0.0 sklearn TPOT==0.10.2 TA-Lib\n",
    "    \n",
    "   Or\n",
    "            \n",
    "        pip3 install DateTime==4.3 pandas==0.23.0 pandas-datareader==0.7.0 seaborn==0.8.1 matplotlib==2.1.2 numpy==1.14.5 bs4==0.0.1 requests==2.22.0 feature-selector==1.0.0 featuretools==0.9.1 dask==2.2.0 dask-ml==1.0.0 sklearn TPOT==0.10.2 TA-Lib\n",
    "\n",
    "(Depending on how you have installed your Pip installer. Notice the 3 after pip)\n",
    "\n",
    "\n",
    "2. Install them all from the provided \"requirements.txt\" file by typing in the following command into your command prompt:\n",
    "\n",
    "        pip install -r requirements.txt\n",
    "        \n",
    "   Or\n",
    "           \n",
    "        pip3 install -r requirements.txt\n",
    "\n",
    "(One of those two ways should create the environment needed to run this project. Otherwise each module will need to be installed manually as needed)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### References and Resources Before We Get Started\n",
    "\n",
    "Before I forget and start going down the code wormhole, I wanted to take some time and reference some various websites, other's projects and the tools that I'm using in the following code. I want to thank everyone involved in my learning process and I urge you to check them out:\n",
    "\n",
    "1. The motivation and skeleton for my project - https://gogul09.github.io/software/regression-example-boston-housing-prices\n",
    "(That article taught me more in one reading than any other)\n",
    "\n",
    "\n",
    "1. TPOT (automated machine learning) - https://epistasislab.github.io/tpot/\n",
    "2. FeatureTools (feature engineering) - https://www.featuretools.com/\n",
    "3. Feature-Selector (take a guess :D) - https://github.com/WillKoehrsen/feature-selector\n",
    "4. Dask (parallel computing) - https://docs.dask.org/en/latest/\n",
    "5. TA-Lib (technical indicators) - https://mrjbq7.github.io/ta-lib/index.html\n",
    "\n",
    "\n",
    "1. Medium (Great resource for learning. Sign up for emails based on what you read!) - https://medium.com/topic/artificial-intelligence\n",
    "2. Towards Data Science (Another awesome resource) - https://towardsdatascience.com/\n",
    "3. Machine Learning Mastery (see above) - https://machinelearningmastery.com/\n",
    "4. Stack Overflow (duh!) - https://stackoverflow.com/\n",
    "5. Upwork (in a real pinch, pay someone to teach you something) - www.upwork.com\n",
    "6. Udemy (I received great value from this course) - https://www.udemy.com/the-data-science-course-complete-data-science-bootcamp/\n",
    "\n",
    "\n",
    "Without further adu, let's get started."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The Code\n",
    "\n",
    "Keep in mind, I have already run every cell in this notebook. If you want to re-run it, you are more than welcome to, however I have made references to values and numbers pertaining to this run only, like prices and MSE values and such. My breakdowns of each step will then vary if you're looking up a different stock, so just keep that in mind if you're re-running it for the first time. Adjust as you need.\n",
    "\n",
    "ALSO, if you do want to start this script over again, make sure you click \"Kernel > Restart and Clear Output\" to make sure you're starting fresh!!!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1. GET THE STOCK'S HISTORICAL DATA"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Which stock ticker would you like to predict? CLVS\n",
      "Getting the historical data for:  CLVS\n"
     ]
    }
   ],
   "source": [
    "# In-notebook plotting. Take this out if running from a .py file\n",
    "%matplotlib inline\n",
    "\n",
    "# Display Future Warnings that repeat \"needlessly\" only once (for now)\n",
    "import warnings\n",
    "warnings.filterwarnings(\"once\")\n",
    "\n",
    "# Let's begin. Ask for which stock ticker you want to predict.\n",
    "# A list of stocks that are able to be run so far are:\n",
    "\n",
    "# CLVS\n",
    "# RKDA\n",
    "# AAPL\n",
    "# AMZN\n",
    "\n",
    "# ...so start with one of them. More on this later. I used CLVS for this\n",
    "# notebook. A smaller cap stock with no long bias over its history.\n",
    "ticker_input = input('Which stock ticker would you like to predict? ') \n",
    "print('Getting the historical data for: ',ticker_input)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\windowshopr\\AppData\\Local\\Programs\\Python\\Python36\\lib\\importlib\\_bootstrap.py:219: ImportWarning: can't resolve package from __spec__ or __package__, falling back on __name__ and __path__\n",
      "  return f(*args, **kwds)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "            High   Low  Open  Close   Volume  Adj Close\n",
      "Date                                                   \n",
      "2011-11-16 13.39 12.29 13.05  12.56  1912700      12.56\n",
      "2011-11-17 12.70 12.44 12.56  12.69   298700      12.69\n",
      "2011-11-18 12.77 12.50 12.70  12.58    77300      12.58\n",
      "2011-11-21 12.59 12.50 12.51  12.55   100300      12.55\n",
      "2011-11-22 12.59 12.45 12.50  12.54    65700      12.54\n",
      "Done!\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\windowshopr\\AppData\\Local\\Programs\\Python\\Python36\\lib\\site-packages\\pandas_datareader\\data.py:310: ResourceWarning: unclosed <socket.socket fd=2628, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.8.8.67', 54601), raddr=('216.115.100.123', 443)>\n",
      "  session=session).read()\n"
     ]
    }
   ],
   "source": [
    "# Import first group of dependencies, and then\n",
    "# download the historical daily data from Yahoo Finance\n",
    "# using Pandas Datareader. A different source could be\n",
    "# used as I've read that getting historical data using\n",
    "# this method is temperamental (and deprecated), but works for now\n",
    "from datetime import datetime\n",
    "from pandas_datareader import data as web\n",
    "import pandas as pd\n",
    "pd.options.display.float_format = '{:,.2f}'.format\n",
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt\n",
    "import random\n",
    "import os\n",
    "import numpy as np\n",
    "import time\n",
    "\n",
    "# Downloading historical data as a Pandas dataframe.\n",
    "# Think of a dataframe as an Excel file, with rows\n",
    "# and columns of data, sometimes with a header row.\n",
    "ex = 'yahoo'\n",
    "start = datetime(2000, 1, 1) # Edit this to whatever you want, but the older the better\n",
    "end = datetime.now() # Today\n",
    "dataset = web.DataReader(ticker_input, ex, start, end)\n",
    "\n",
    "# Now let's look at the top 5 rows of the dataset\n",
    "print(dataset.head(5))\n",
    "\n",
    "print('Done!')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2. ADJUST THE HISTORICAL DATASET AND ADD TECHNICAL INDICATORS"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# We should have a Date index column, which isn't exactly a part of \n",
    "# the dataframe itself. The rest of the columns are the stock's\n",
    "# historical data per day.\n",
    "\n",
    "# Now that we have the dataframe, let's modify it for our model.\n",
    "# You'll notice the data has a Close and an Adjusted Close column.\n",
    "# We're going to use the Adj Close for various reasons, so let's\n",
    "# get rid of the Close column:\n",
    "dataset = dataset.drop(['Close'],axis=1)\n",
    "\n",
    "# Now let's shift the daily data so that for the current day,\n",
    "# we have new columns containing the previous day's OHLC data.\n",
    "dataset['PrevOpen'] = dataset['Open'].shift(1)\n",
    "dataset['PrevHigh'] = dataset['High'].shift(1)\n",
    "dataset['PrevLow'] = dataset['Low'].shift(1)\n",
    "dataset['PrevAdjClose'] = dataset['Adj Close'].shift(1)\n",
    "dataset['PrevVol'] = dataset['Volume'].shift(1)\n",
    "\n",
    "# Now that we have that data, we need to drop the High, Low and \n",
    "# Volume columns because we won't know what those values will be\n",
    "# until the trading day is over. We'll only know (roughly) where the \n",
    "# stock will open, and then all of what happened yesterday.\n",
    "dataset = dataset.drop(['High'],axis=1)\n",
    "dataset = dataset.drop(['Low'],axis=1)\n",
    "dataset = dataset.drop(['Volume'],axis=1)\n",
    "\n",
    "# Now just for fun, let's add in a couple moving averages based on the \n",
    "# opening prices. We could add in way more indicators here in the future.\n",
    "# The model can determine which ones are important. You'll see that later.\n",
    "dataset['9MA'] = dataset['Open'].rolling(window=9).mean()\n",
    "dataset['20MA'] = dataset['Open'].rolling(window=20).mean()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "            Open  Adj Close  PrevOpen  PrevHigh  PrevLow  PrevAdjClose  \\\n",
      "Date                                                                     \n",
      "2011-11-16 13.05      12.56       nan       nan      nan           nan   \n",
      "2011-11-17 12.56      12.69     13.05     13.39    12.29         12.56   \n",
      "2011-11-18 12.70      12.58     12.56     12.70    12.44         12.69   \n",
      "2011-11-21 12.51      12.55     12.70     12.77    12.50         12.58   \n",
      "2011-11-22 12.50      12.54     12.51     12.59    12.50         12.55   \n",
      "\n",
      "                PrevVol  9MA  20MA  UpperBB ...   ROC  RSI  ATR  BETA  \\\n",
      "Date                                        ...                         \n",
      "2011-11-16          nan  nan   nan      nan ...   nan  nan  nan   nan   \n",
      "2011-11-17 1,912,700.00  nan   nan      nan ...   nan  nan  nan   nan   \n",
      "2011-11-18   298,700.00  nan   nan      nan ...   nan  nan  nan   nan   \n",
      "2011-11-21    77,300.00  nan   nan      nan ...   nan  nan  nan   nan   \n",
      "2011-11-22   100,300.00  nan   nan      nan ...   nan  nan  nan   nan   \n",
      "\n",
      "            LINEARREG  LINEARREG_ANGLE  LINEARREG_SLOPE  STDDEV  TSF  VAR  \n",
      "Date                                                                       \n",
      "2011-11-16        nan              nan              nan     nan  nan  nan  \n",
      "2011-11-17        nan              nan              nan     nan  nan  nan  \n",
      "2011-11-18        nan              nan              nan     nan  nan  nan  \n",
      "2011-11-21        nan              nan              nan     nan  nan  nan  \n",
      "2011-11-22        nan              nan              nan     nan  nan  nan  \n",
      "\n",
      "[5 rows x 28 columns]\n",
      "Done!\n"
     ]
    }
   ],
   "source": [
    "# This next section I've added as part of an upgrade process to my original\n",
    "# code. We're going to include the TA-Lib module, which is a great tool to\n",
    "# calculate techincal indicators based on previous stock data.\n",
    "import talib\n",
    "\n",
    "# I'm not going to get too detailed with the technical indicators, but you can\n",
    "# check out https://mrjbq7.github.io/ta-lib/doc_index.html to learn more about them,\n",
    "# but for now, let's just add a couple to our dataframe and let the model decide\n",
    "# which ones are important enough to keep for training. Some would argue that too many\n",
    "# adds noise to the data, and they'd be right, but we have a step to determine feature\n",
    "# importance in a little while to combat this. We'll also deal with some of these\n",
    "# indicators correlating to one another in a second.\n",
    "\n",
    "# Overlap Studies\n",
    "# Bollinger Bands\n",
    "upperband, middleband, lowerband = talib.BBANDS(dataset['PrevAdjClose'], timeperiod=20, nbdevup=2, nbdevdn=2, matype=0)\n",
    "dataset['UpperBB'] = upperband\n",
    "dataset['MidBB'] = middleband\n",
    "dataset['LowerBB'] = lowerband\n",
    "\n",
    "# SAR - Parabolic SAR\n",
    "sar = talib.SAR(dataset['PrevHigh'], dataset['PrevLow'], acceleration=0.02, maximum=0.2)\n",
    "dataset['SAR'] = sar\n",
    "\n",
    "\n",
    "# Momentum Indicators\n",
    "# ADX - Average Directional Movement Index\n",
    "real = talib.ADX(dataset['PrevHigh'], dataset['PrevLow'], dataset['PrevAdjClose'], timeperiod=14)\n",
    "dataset['ADX'] = real\n",
    "\n",
    "# MACD - Moving Average Convergence/Divergence\n",
    "macd, macdsignal, macdhist = talib.MACD(dataset['PrevAdjClose'], fastperiod=12, slowperiod=26, signalperiod=9)\n",
    "dataset['MACD'] = macd\n",
    "dataset['MACDSIGNAL'] = macdsignal\n",
    "dataset['MACDHIST'] = macdhist\n",
    "\n",
    "# MOM - Momentum\n",
    "real = talib.MOM(dataset['PrevAdjClose'], timeperiod=10)\n",
    "dataset['MOM'] = real\n",
    "\n",
    "# ROC - Rate of change : ((price/prevPrice)-1)*100\n",
    "real = talib.ROC(dataset['PrevAdjClose'], timeperiod=10)\n",
    "dataset['ROC'] = real\n",
    "\n",
    "# RSI - Relative Strength Index\n",
    "real = talib.RSI(dataset['PrevAdjClose'], timeperiod=14)\n",
    "dataset['RSI'] = real\n",
    "\n",
    "\n",
    "# Volatility Indicator Functions\n",
    "# ATR - Average True Range\n",
    "real = talib.ATR(dataset['PrevHigh'], dataset['PrevLow'], dataset['PrevAdjClose'], timeperiod=14)\n",
    "dataset['ATR'] = real\n",
    "\n",
    "\n",
    "# Statistic Functions\n",
    "# BETA - Beta\n",
    "real = talib.BETA(dataset['PrevHigh'], dataset['PrevLow'], timeperiod=5)\n",
    "dataset['BETA'] = real\n",
    "\n",
    "# LINEARREG - Linear Regression\n",
    "real = talib.LINEARREG(dataset['PrevAdjClose'], timeperiod=14)\n",
    "dataset['LINEARREG'] = real\n",
    "\n",
    "# LINEARREG_ANGLE - Linear Regression Angle\n",
    "real = talib.LINEARREG_ANGLE(dataset['PrevAdjClose'], timeperiod=14)\n",
    "dataset['LINEARREG_ANGLE'] = real\n",
    "\n",
    "# LINEARREG_SLOPE - Linear Regression Slope\n",
    "real = talib.LINEARREG_SLOPE(dataset['PrevAdjClose'], timeperiod=14)\n",
    "dataset['LINEARREG_SLOPE'] = real\n",
    "\n",
    "# STDDEV - Standard Deviation\n",
    "real = talib.STDDEV(dataset['PrevAdjClose'], timeperiod=5, nbdev=1)\n",
    "dataset['STDDEV'] = real\n",
    "\n",
    "# TSF - Time Series Forecast\n",
    "real = talib.TSF(dataset['PrevAdjClose'], timeperiod=14)\n",
    "dataset['TSF'] = real\n",
    "\n",
    "# VAR - Variance\n",
    "real = talib.VAR(dataset['PrevAdjClose'], timeperiod=5, nbdev=1)\n",
    "dataset['VAR'] = real\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "# Phewf! Now print the new head of the dataframe and check it out so far\n",
    "print(dataset.head(5))\n",
    "print('Done!')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3. ADD THE STOCK INDUSTRY'S ETF HISTORICAL DATA TO THE DATASET"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[INFO] Now getting the Industry ETFs historical data...\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\windowshopr\\AppData\\Local\\Programs\\Python\\Python36\\lib\\importlib\\_bootstrap.py:219: ImportWarning: can't resolve package from __spec__ or __package__, falling back on __name__ and __path__\n",
      "  return f(*args, **kwds)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[INFO] Industry is : Biotechnology\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\windowshopr\\AppData\\Local\\Programs\\Python\\Python36\\lib\\site-packages\\pandas_datareader\\data.py:310: ResourceWarning: unclosed <socket.socket fd=2596, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.8.8.67', 54603), raddr=('216.115.100.123', 443)>\n",
      "  session=session).read()\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "            PrevOpen  PrevHigh  PrevLow  PrevAdjClose      PrevVol  9MA  20MA  \\\n",
      "Date                                                                            \n",
      "2011-11-16       nan       nan      nan           nan          nan  nan   nan   \n",
      "2011-11-17     13.05     13.39    12.29         12.56 1,912,700.00  nan   nan   \n",
      "2011-11-18     12.56     12.70    12.44         12.69   298,700.00  nan   nan   \n",
      "2011-11-21     12.70     12.77    12.50         12.58    77,300.00  nan   nan   \n",
      "2011-11-22     12.51     12.59    12.50         12.55   100,300.00  nan   nan   \n",
      "\n",
      "            UpperBB  MidBB  LowerBB    ...      STDDEV  TSF  VAR  PrevIndOpen  \\\n",
      "Date                                   ...                                      \n",
      "2011-11-16      nan    nan      nan    ...         nan  nan  nan        31.45   \n",
      "2011-11-17      nan    nan      nan    ...         nan  nan  nan        31.48   \n",
      "2011-11-18      nan    nan      nan    ...         nan  nan  nan        31.20   \n",
      "2011-11-21      nan    nan      nan    ...         nan  nan  nan        30.97   \n",
      "2011-11-22      nan    nan      nan    ...         nan  nan  nan        32.15   \n",
      "\n",
      "            PrevIndHigh  PrevIndLow  PrevIndClose   PrevIndVol  Open  \\\n",
      "Date                                                                   \n",
      "2011-11-16        31.85       31.39         31.26 1,151,400.00 13.05   \n",
      "2011-11-17        31.79       31.14         30.81 2,376,000.00 12.56   \n",
      "2011-11-18        31.41       30.73         30.54 1,682,100.00 12.70   \n",
      "2011-11-21        31.21       30.72         30.33 1,926,600.00 12.51   \n",
      "2011-11-22        32.41       31.99         31.77 3,968,400.00 12.50   \n",
      "\n",
      "            Adj Close  \n",
      "Date                   \n",
      "2011-11-16      12.56  \n",
      "2011-11-17      12.69  \n",
      "2011-11-18      12.58  \n",
      "2011-11-21      12.55  \n",
      "2011-11-22      12.54  \n",
      "\n",
      "[5 rows x 33 columns]\n",
      "Done.\n"
     ]
    }
   ],
   "source": [
    "# From here, I thought it would be important to include some historical data\n",
    "# about the Industry that the stock is in. It has been suggested that a large\n",
    "# part of a stock's price movement is based on the industry that it's in. How\n",
    "# I decided to tackle this was to find an ETF ticker symbol that best represents\n",
    "# that Industry's price movements, and then download its historical price data\n",
    "# like we did before.\n",
    "\n",
    "# Scrape the stock's Yahoo Finance > Profile page for the Industry that it's in.\n",
    "print('[INFO] Now getting the Industry ETFs historical data...')\n",
    "from bs4 import BeautifulSoup\n",
    "import requests\n",
    "headers = requests.utils.default_headers() \n",
    "headers['User-Agent'] = 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36'\n",
    "# Go to the URL, parse the page data, and find the Industry\n",
    "url = 'https://finance.yahoo.com/quote/' + ticker_input + '/profile'\n",
    "page = requests.get(url)\n",
    "soup = BeautifulSoup(page.text, 'html.parser')\n",
    "table = soup.find('p', {'class' :'D(ib) Va(t)'})\n",
    "industry = table.findAll('span')\n",
    "indust = industry[3].text\n",
    "print('[INFO] Industry is :',indust)\n",
    "\n",
    "# Then get the historical data for that industry's ETF. This list\n",
    "# will need to be added to in the future based on the industry's\n",
    "# that are shown on a stock's Profile tab over at Yahoo Finance.\n",
    "# If you're trying to search for a stock who's industry and ETF is \n",
    "# not in this below list, you'll need to add the appropriate code \n",
    "# snippet below as an 'elif' to incorporate it. That's why the \n",
    "# small list of suggested stocks at the beginning.\n",
    "if indust == \"Biotechnology\":\n",
    "    etf_ticker = \"IBB\"\n",
    "elif indust == \"Specialty Retail\":\n",
    "    etf_ticker = \"XRT\"\n",
    "elif indust == \"Oil & Gas E&P\":\n",
    "    etf_ticker = \"XOP\"\n",
    "elif indust == 'Agricultural Inputs':\n",
    "    etf_ticker = 'DBA'\n",
    "elif indust == 'Consumer Electronics':\n",
    "    etf_ticker = 'VGT'\n",
    "else:\n",
    "      print(\"[INFO] The stock symbol you have entered does not have its Industry's ETF ticker symbol entered in the list. Go to the stock's Yahoo Finance page, click the Profile tab, and copy what's listed as the Industry (ex. Specialty Retail) and then add it to the list in the code like the others. Then you have to find a ticker symbol for the ETF of that industry (or the best one you can find that accurately represents that industry), add it to the code's list, then start the program over.\")\n",
    "\n",
    "# Once we have the industry's ETF from Yahoo Finance, get its \n",
    "# historical data like before\n",
    "ex = 'yahoo'\n",
    "etf_df = web.DataReader(etf_ticker, ex, start, end)\n",
    "\n",
    "# Then from there, shift the previous day's data forward one day\n",
    "# to put them into the row of the current day's\n",
    "dataset['PrevIndOpen'] = etf_df['Open'].shift(1)\n",
    "dataset['PrevIndHigh'] = etf_df['High'].shift(1)\n",
    "dataset['PrevIndLow'] = etf_df['Low'].shift(1)\n",
    "dataset['PrevIndClose'] = etf_df['Adj Close'].shift(1)\n",
    "dataset['PrevIndVol'] = etf_df['Volume'].shift(1)\n",
    "\n",
    "# Reshape the dataframe to put Open and Adj Close to the far \n",
    "# right so when we export the predictions dataset, the predictions\n",
    "# column will be right next to them for easier analysis\n",
    "opencolumn = dataset['Open']\n",
    "adjclosecolumn = dataset['Adj Close']\n",
    "\n",
    "dataset = dataset.drop(['Open'],axis=1)\n",
    "dataset = dataset.drop(['Adj Close'],axis=1)\n",
    "\n",
    "dataset['Open'] = opencolumn\n",
    "dataset['Adj Close'] = adjclosecolumn\n",
    "\n",
    "# Let's preview the new dataset so far. Note the Not a Number (nan)\n",
    "# values. We'll handle these in a second.\n",
    "print(dataset.head(5))\n",
    "\n",
    "# Now save our initial dataset to a .csv file for later review if we want\n",
    "if not os.path.exists(\"./historical data/\"):\n",
    "    os.makedirs(\"./historical data/\")\n",
    "dataset.to_csv('./historical data/' + ticker_input + '_historical_data.csv')\n",
    "\n",
    "print('Done.')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 4. EXPLORE AND PREPROCESS THE DATASET"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Let's change the dataframe's variable to df, an industry standard.\n",
    "df = pd.DataFrame(dataset)\n",
    "df.columns = dataset.columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[INFO] Dataset names : Index(['PrevOpen', 'PrevHigh', 'PrevLow', 'PrevAdjClose', 'PrevVol', '9MA',\n",
      "       '20MA', 'UpperBB', 'MidBB', 'LowerBB', 'SAR', 'ADX', 'MACD',\n",
      "       'MACDSIGNAL', 'MACDHIST', 'MOM', 'ROC', 'RSI', 'ATR', 'BETA',\n",
      "       'LINEARREG', 'LINEARREG_ANGLE', 'LINEARREG_SLOPE', 'STDDEV', 'TSF',\n",
      "       'VAR', 'PrevIndOpen', 'PrevIndHigh', 'PrevIndLow', 'PrevIndClose',\n",
      "       'PrevIndVol', 'Open', 'Adj Close'],\n",
      "      dtype='object')\n"
     ]
    }
   ],
   "source": [
    "# Print the \"feature\" (or column) names.\n",
    "print(\"[INFO] Dataset names : {}\".format(df.columns))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[INFO] Dataset shape (Rows, Columns) : (1948, 33)\n"
     ]
    }
   ],
   "source": [
    "# Show the \"shape\" of the dataset.\n",
    "print(\"[INFO] Dataset shape (Rows, Columns) : {}\".format(df.shape))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[INFO] df type : <class 'pandas.core.frame.DataFrame'>\n"
     ]
    }
   ],
   "source": [
    "# Print the dataframe's type (in case you need to know for later)\n",
    "print(\"[INFO] df type : {}\".format(type(df)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[INFO] Checking if any columns with NaN values...\n",
      "PrevOpen            True\n",
      "PrevHigh            True\n",
      "PrevLow             True\n",
      "PrevAdjClose        True\n",
      "PrevVol             True\n",
      "9MA                 True\n",
      "20MA                True\n",
      "UpperBB             True\n",
      "MidBB               True\n",
      "LowerBB             True\n",
      "SAR                 True\n",
      "ADX                 True\n",
      "MACD                True\n",
      "MACDSIGNAL          True\n",
      "MACDHIST            True\n",
      "MOM                 True\n",
      "ROC                 True\n",
      "RSI                 True\n",
      "ATR                 True\n",
      "BETA                True\n",
      "LINEARREG           True\n",
      "LINEARREG_ANGLE     True\n",
      "LINEARREG_SLOPE     True\n",
      "STDDEV              True\n",
      "TSF                 True\n",
      "VAR                 True\n",
      "PrevIndOpen        False\n",
      "PrevIndHigh        False\n",
      "PrevIndLow         False\n",
      "PrevIndClose       False\n",
      "PrevIndVol         False\n",
      "Open               False\n",
      "Adj Close          False\n",
      "dtype: bool\n"
     ]
    }
   ],
   "source": [
    "# Now check if any columns have blank values\n",
    "print('[INFO] Checking if any columns with NaN values...')\n",
    "print(pd.isnull(df).any())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice anything? Remember that we shifted the data down one day to put the\n",
    "previous day's data in the current day's row? Well since we don't know the\n",
    "previous day's data from the day before the start date of the dataset, \n",
    "Pandas has no choice but to put 'nan' in there to fill the empty space.\n",
    "We also experience this when creating our 9 and 20 period moving averages\n",
    "and other indicators because we need the previous day's of data before the value for\n",
    "them would start to show up.\n",
    "\n",
    "There are various ways to deal with 'nan' values in a dataset:\n",
    "1. Replace the missing value with a large negative number (e.g. -999), but this is an old way of doing things.\n",
    "2. Replace the missing value with mean of the column.\n",
    "3. Replace the missing value with median of the column.\n",
    "\n",
    "For this application, our nan values are only in the top few rows due to \n",
    "our shifting, so we can just drop these rows from our dataset because 30\n",
    "days max missing from a dataset out of many years of stock data won't make too much\n",
    "difference. If your stock's ETF ticker symbol doesn't date back as far as\n",
    "your stock's historical data, you will have more nan values to deal. The\n",
    "newbie way to do this for now is just dropping every row that has a nan\n",
    "values, so you could employ the commented out code in place of this one,\n",
    "that choice will be up to you, but we don't want any nan values in our data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "PrevOpen           False\n",
      "PrevHigh           False\n",
      "PrevLow            False\n",
      "PrevAdjClose       False\n",
      "PrevVol            False\n",
      "9MA                False\n",
      "20MA               False\n",
      "UpperBB            False\n",
      "MidBB              False\n",
      "LowerBB            False\n",
      "SAR                False\n",
      "ADX                False\n",
      "MACD               False\n",
      "MACDSIGNAL         False\n",
      "MACDHIST           False\n",
      "MOM                False\n",
      "ROC                False\n",
      "RSI                False\n",
      "ATR                False\n",
      "BETA               False\n",
      "LINEARREG          False\n",
      "LINEARREG_ANGLE    False\n",
      "LINEARREG_SLOPE    False\n",
      "STDDEV             False\n",
      "TSF                False\n",
      "VAR                False\n",
      "PrevIndOpen        False\n",
      "PrevIndHigh        False\n",
      "PrevIndLow         False\n",
      "PrevIndClose       False\n",
      "PrevIndVol         False\n",
      "Open               False\n",
      "Adj Close          False\n",
      "dtype: bool\n"
     ]
    }
   ],
   "source": [
    "# Drop all Nan rows of the dataset to get rid of\n",
    "# the ones caused by the shifting of data\n",
    "df = df.dropna(how='any')\n",
    "# df = df.iloc[20:] # to drop the first X amount of rows instead\n",
    "\n",
    "# Now check if we have any nan values. All Falses? Great!\n",
    "print(pd.isnull(df).any())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[INFO] Dataset types:\n",
      "PrevOpen           float64\n",
      "PrevHigh           float64\n",
      "PrevLow            float64\n",
      "PrevAdjClose       float64\n",
      "PrevVol            float64\n",
      "9MA                float64\n",
      "20MA               float64\n",
      "UpperBB            float64\n",
      "MidBB              float64\n",
      "LowerBB            float64\n",
      "SAR                float64\n",
      "ADX                float64\n",
      "MACD               float64\n",
      "MACDSIGNAL         float64\n",
      "MACDHIST           float64\n",
      "MOM                float64\n",
      "ROC                float64\n",
      "RSI                float64\n",
      "ATR                float64\n",
      "BETA               float64\n",
      "LINEARREG          float64\n",
      "LINEARREG_ANGLE    float64\n",
      "LINEARREG_SLOPE    float64\n",
      "STDDEV             float64\n",
      "TSF                float64\n",
      "VAR                float64\n",
      "PrevIndOpen        float64\n",
      "PrevIndHigh        float64\n",
      "PrevIndLow         float64\n",
      "PrevIndClose       float64\n",
      "PrevIndVol         float64\n",
      "Open               float64\n",
      "Adj Close          float64\n",
      "dtype: object\n"
     ]
    }
   ],
   "source": [
    "# Now let's check the dtypes of the dataset\n",
    "print('[INFO] Dataset types:')\n",
    "print(df.dtypes)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[INFO] Print a statistical summary of dataset:\n",
      "       PrevOpen  PrevHigh  PrevLow  PrevAdjClose       PrevVol      9MA  \\\n",
      "count  1,914.00  1,914.00 1,914.00      1,914.00      1,914.00 1,914.00   \n",
      "mean      45.13     46.35    43.87         45.13  1,166,180.56    45.14   \n",
      "std       24.77     25.36    24.12         24.77  1,698,372.15    24.52   \n",
      "min        5.40      5.77     5.20          5.34      7,500.00     6.95   \n",
      "25%       21.27     21.84    20.52         21.22    357,525.00    21.12   \n",
      "50%       43.62     44.82    42.48         43.80    779,450.00    44.32   \n",
      "75%       64.05     65.59    62.36         64.25  1,377,150.00    64.48   \n",
      "max      115.85    116.75   111.47        114.64 30,660,400.00   106.71   \n",
      "\n",
      "          20MA  UpperBB    MidBB  LowerBB    ...       STDDEV      TSF  \\\n",
      "count 1,914.00 1,914.00 1,914.00 1,914.00    ...     1,914.00 1,914.00   \n",
      "mean     45.15    52.05    45.16    38.26    ...         1.44    45.13   \n",
      "std      24.21    28.20    24.21    22.08    ...         1.97    25.19   \n",
      "min       8.98    13.51     9.15   -14.12    ...         0.06     0.33   \n",
      "25%      21.01    25.26    21.02    17.37    ...         0.55    21.09   \n",
      "50%      44.95    49.98    45.01    36.27    ...         1.04    43.57   \n",
      "75%      64.20    73.99    64.28    56.00    ...         1.80    63.89   \n",
      "max      99.69   147.63    99.82    92.02    ...        35.00   120.09   \n",
      "\n",
      "           VAR  PrevIndOpen  PrevIndHigh  PrevIndLow  PrevIndClose  \\\n",
      "count 1,914.00     1,914.00     1,914.00    1,914.00      1,914.00   \n",
      "mean      5.97        88.82        89.63       87.88         88.28   \n",
      "std      50.03        24.83        25.07       24.56         24.86   \n",
      "min       0.00        34.75        35.69       34.75         34.94   \n",
      "25%       0.31        72.24        73.15       71.25         71.18   \n",
      "50%       1.09        95.98        96.84       95.05         95.55   \n",
      "75%       3.25       107.73       108.91      106.68        107.52   \n",
      "max   1,225.23       133.09       133.60      132.17        131.67   \n",
      "\n",
      "         PrevIndVol     Open  Adj Close  \n",
      "count      1,914.00 1,914.00   1,914.00  \n",
      "mean   3,657,323.98    45.12      45.12  \n",
      "std    2,992,208.07    24.77      24.78  \n",
      "min      291,300.00     5.31       5.20  \n",
      "25%    1,841,550.00    21.27      21.22  \n",
      "50%    2,833,200.00    43.62      43.80  \n",
      "75%    4,371,450.00    64.05      64.25  \n",
      "max   35,551,800.00   115.85     114.64  \n",
      "\n",
      "[8 rows x 33 columns]\n"
     ]
    }
   ],
   "source": [
    "# All numbers? Good. Machine learning algorithms want numbers\n",
    "# to deal with. If we had categorical (word) values, we'd have\n",
    "# to one-hot encode them. Homework; research what that is :D\n",
    "\n",
    "# Print a statistical summary of the dataset for reference\n",
    "print('[INFO] Print a statistical summary of dataset:')\n",
    "print(df.describe(include='all'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[INFO] Printing the Pearson Correlation tables...\n",
      "PEARSON CORRELATION\n",
      "                 PrevOpen  PrevHigh  PrevLow  PrevAdjClose  PrevVol   9MA  \\\n",
      "PrevOpen             1.00      1.00     1.00          1.00    -0.07  0.99   \n",
      "PrevHigh             1.00      1.00     1.00          1.00    -0.06  0.99   \n",
      "PrevLow              1.00      1.00     1.00          1.00    -0.08  0.99   \n",
      "PrevAdjClose         1.00      1.00     1.00          1.00    -0.07  0.99   \n",
      "PrevVol             -0.07     -0.06    -0.08         -0.07     1.00 -0.05   \n",
      "9MA                  0.99      0.99     0.99          0.99    -0.05  1.00   \n",
      "20MA                 0.97      0.97     0.97          0.96    -0.04  0.99   \n",
      "UpperBB              0.92      0.92     0.92          0.92     0.01  0.94   \n",
      "MidBB                0.96      0.96     0.96          0.96    -0.03  0.98   \n",
      "LowerBB              0.94      0.93     0.94          0.93    -0.09  0.95   \n",
      "SAR                  0.93      0.93     0.93          0.93    -0.02  0.96   \n",
      "ADX                 -0.16     -0.16    -0.16         -0.16     0.09 -0.17   \n",
      "MACD                 0.32      0.32     0.33          0.33    -0.07  0.27   \n",
      "MACDSIGNAL           0.34      0.33     0.34          0.34    -0.05  0.32   \n",
      "MACDHIST             0.03      0.04     0.04          0.04    -0.09 -0.08   \n",
      "MOM                  0.17      0.18     0.18          0.19    -0.13  0.06   \n",
      "ROC                  0.15      0.16     0.16          0.17    -0.08  0.06   \n",
      "RSI                  0.21      0.22     0.22          0.23    -0.07  0.13   \n",
      "ATR                  0.82      0.83     0.81          0.82     0.14  0.85   \n",
      "BETA                 0.04      0.04     0.04          0.04     0.03  0.06   \n",
      "LINEARREG            0.99      0.99     0.99          0.99    -0.06  0.99   \n",
      "LINEARREG_ANGLE      0.17      0.18     0.18          0.18    -0.05  0.09   \n",
      "LINEARREG_SLOPE      0.19      0.19     0.19          0.20    -0.09  0.11   \n",
      "STDDEV               0.32      0.34     0.32          0.33     0.35  0.36   \n",
      "TSF                  0.99      0.99     0.99          0.99    -0.06  0.99   \n",
      "VAR                  0.04      0.05     0.04          0.04     0.30  0.10   \n",
      "PrevIndOpen          0.43      0.42     0.42          0.42     0.31  0.43   \n",
      "PrevIndHigh          0.42      0.42     0.42          0.42     0.31  0.43   \n",
      "PrevIndLow           0.42      0.42     0.42          0.42     0.31  0.43   \n",
      "PrevIndClose         0.42      0.42     0.42          0.42     0.31  0.43   \n",
      "PrevIndVol           0.26      0.26     0.25          0.25     0.09  0.27   \n",
      "Open                 0.99      1.00     1.00          1.00    -0.07  0.98   \n",
      "Adj Close            0.99      0.99     0.99          0.99    -0.07  0.98   \n",
      "\n",
      "                 20MA  UpperBB  MidBB  LowerBB    ...      STDDEV   TSF   VAR  \\\n",
      "PrevOpen         0.97     0.92   0.96     0.94    ...        0.32  0.99  0.04   \n",
      "PrevHigh         0.97     0.92   0.96     0.93    ...        0.34  0.99  0.05   \n",
      "PrevLow          0.97     0.92   0.96     0.94    ...        0.32  0.99  0.04   \n",
      "PrevAdjClose     0.96     0.92   0.96     0.93    ...        0.33  0.99  0.04   \n",
      "PrevVol         -0.04     0.01  -0.03    -0.09    ...        0.35 -0.06  0.30   \n",
      "9MA              0.99     0.94   0.98     0.95    ...        0.36  0.99  0.10   \n",
      "20MA             1.00     0.97   1.00     0.96    ...        0.37  0.96  0.12   \n",
      "UpperBB          0.97     1.00   0.97     0.85    ...        0.42  0.91  0.19   \n",
      "MidBB            1.00     0.97   1.00     0.95    ...        0.37  0.95  0.13   \n",
      "LowerBB          0.96     0.85   0.95     1.00    ...        0.28  0.93  0.04   \n",
      "SAR              0.98     0.96   0.98     0.92    ...        0.38  0.93  0.15   \n",
      "ADX             -0.19    -0.12  -0.19    -0.28    ...       -0.06 -0.15 -0.02   \n",
      "MACD             0.14     0.07   0.13     0.20    ...        0.03  0.37 -0.09   \n",
      "MACDSIGNAL       0.22     0.15   0.21     0.26    ...        0.09  0.38 -0.00   \n",
      "MACDHIST        -0.19    -0.22  -0.19    -0.14    ...       -0.16  0.05 -0.28   \n",
      "MOM             -0.06    -0.11  -0.07    -0.02    ...       -0.13  0.19 -0.28   \n",
      "ROC             -0.04    -0.04  -0.05    -0.06    ...        0.02  0.17 -0.08   \n",
      "RSI              0.03     0.01   0.03     0.04    ...        0.03  0.23 -0.05   \n",
      "ATR              0.87     0.93   0.87     0.73    ...        0.55  0.81  0.31   \n",
      "BETA             0.06     0.06   0.06     0.06    ...        0.02  0.05  0.03   \n",
      "LINEARREG        0.96     0.92   0.96     0.94    ...        0.35  1.00  0.08   \n",
      "LINEARREG_ANGLE -0.04    -0.05  -0.04    -0.03    ...        0.01  0.22 -0.07   \n",
      "LINEARREG_SLOPE -0.04    -0.10  -0.05     0.02    ...       -0.06  0.25 -0.18   \n",
      "STDDEV           0.37     0.42   0.37     0.28    ...        1.00  0.35  0.86   \n",
      "TSF              0.96     0.91   0.95     0.93    ...        0.35  1.00  0.08   \n",
      "VAR              0.12     0.19   0.13     0.04    ...        0.86  0.08  1.00   \n",
      "PrevIndOpen      0.45     0.43   0.45     0.44    ...        0.16  0.42  0.05   \n",
      "PrevIndHigh      0.45     0.43   0.45     0.43    ...        0.16  0.41  0.05   \n",
      "PrevIndLow       0.45     0.43   0.45     0.43    ...        0.15  0.41  0.04   \n",
      "PrevIndClose     0.44     0.43   0.44     0.43    ...        0.16  0.41  0.05   \n",
      "PrevIndVol       0.28     0.29   0.28     0.24    ...        0.20  0.25  0.07   \n",
      "Open             0.96     0.92   0.96     0.93    ...        0.33  0.99  0.04   \n",
      "Adj Close        0.96     0.91   0.96     0.93    ...        0.33  0.98  0.04   \n",
      "\n",
      "                 PrevIndOpen  PrevIndHigh  PrevIndLow  PrevIndClose  \\\n",
      "PrevOpen                0.43         0.42        0.42          0.42   \n",
      "PrevHigh                0.42         0.42        0.42          0.42   \n",
      "PrevLow                 0.42         0.42        0.42          0.42   \n",
      "PrevAdjClose            0.42         0.42        0.42          0.42   \n",
      "PrevVol                 0.31         0.31        0.31          0.31   \n",
      "9MA                     0.43         0.43        0.43          0.43   \n",
      "20MA                    0.45         0.45        0.45          0.44   \n",
      "UpperBB                 0.43         0.43        0.43          0.43   \n",
      "MidBB                   0.45         0.45        0.45          0.44   \n",
      "LowerBB                 0.44         0.43        0.43          0.43   \n",
      "SAR                     0.46         0.45        0.45          0.45   \n",
      "ADX                    -0.27        -0.27       -0.27         -0.27   \n",
      "MACD                   -0.10        -0.11       -0.10         -0.11   \n",
      "MACDSIGNAL             -0.11        -0.11       -0.11         -0.11   \n",
      "MACDHIST               -0.00        -0.00        0.00          0.00   \n",
      "MOM                    -0.05        -0.05       -0.04         -0.04   \n",
      "ROC                    -0.10        -0.10       -0.10         -0.10   \n",
      "RSI                    -0.19        -0.19       -0.18         -0.19   \n",
      "ATR                     0.42         0.42        0.41          0.41   \n",
      "BETA                    0.06         0.06        0.06          0.06   \n",
      "LINEARREG               0.42         0.42        0.42          0.41   \n",
      "LINEARREG_ANGLE        -0.04        -0.04       -0.03         -0.04   \n",
      "LINEARREG_SLOPE        -0.05        -0.05       -0.05         -0.05   \n",
      "STDDEV                  0.16         0.16        0.15          0.16   \n",
      "TSF                     0.42         0.41        0.41          0.41   \n",
      "VAR                     0.05         0.05        0.04          0.05   \n",
      "PrevIndOpen             1.00         1.00        1.00          1.00   \n",
      "PrevIndHigh             1.00         1.00        1.00          1.00   \n",
      "PrevIndLow              1.00         1.00        1.00          1.00   \n",
      "PrevIndClose            1.00         1.00        1.00          1.00   \n",
      "PrevIndVol              0.24         0.25        0.22          0.23   \n",
      "Open                    0.42         0.42        0.42          0.42   \n",
      "Adj Close               0.42         0.42        0.42          0.41   \n",
      "\n",
      "                 PrevIndVol  Open  Adj Close  \n",
      "PrevOpen               0.26  0.99       0.99  \n",
      "PrevHigh               0.26  1.00       0.99  \n",
      "PrevLow                0.25  1.00       0.99  \n",
      "PrevAdjClose           0.25  1.00       0.99  \n",
      "PrevVol                0.09 -0.07      -0.07  \n",
      "9MA                    0.27  0.98       0.98  \n",
      "20MA                   0.28  0.96       0.96  \n",
      "UpperBB                0.29  0.92       0.91  \n",
      "MidBB                  0.28  0.96       0.96  \n",
      "LowerBB                0.24  0.93       0.93  \n",
      "SAR                    0.30  0.93       0.92  \n",
      "ADX                   -0.09 -0.16      -0.16  \n",
      "MACD                  -0.02  0.33       0.32  \n",
      "MACDSIGNAL             0.01  0.33       0.33  \n",
      "MACDHIST              -0.08  0.04       0.04  \n",
      "MOM                   -0.09  0.19       0.19  \n",
      "ROC                   -0.09  0.17       0.17  \n",
      "RSI                   -0.10  0.23       0.23  \n",
      "ATR                    0.39  0.81       0.81  \n",
      "BETA                   0.07  0.04       0.04  \n",
      "LINEARREG              0.26  0.99       0.98  \n",
      "LINEARREG_ANGLE       -0.10  0.18       0.18  \n",
      "LINEARREG_SLOPE       -0.07  0.20       0.19  \n",
      "STDDEV                 0.20  0.33       0.33  \n",
      "TSF                    0.25  0.99       0.98  \n",
      "VAR                    0.07  0.04       0.04  \n",
      "PrevIndOpen            0.24  0.42       0.42  \n",
      "PrevIndHigh            0.25  0.42       0.42  \n",
      "PrevIndLow             0.22  0.42       0.42  \n",
      "PrevIndClose           0.23  0.42       0.41  \n",
      "PrevIndVol             1.00  0.25       0.25  \n",
      "Open                   0.25  1.00       1.00  \n",
      "Adj Close              0.25  1.00       1.00  \n",
      "\n",
      "[33 rows x 33 columns]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAb8AAAFQCAYAAAAvAXAlAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzsnXmcHUXVv5/vTHYChCTsIIEQ2WLYwiYgIMjyqoCKQkQWXxREEBGBVxYVcUPkByoiiCwRRRCRTQSULQLKFkhIWARCCBCIsmchIcnMnN8fVTfp3Nxb3TP3ZmbuzHnm05+5XaerurrvcrqqziIzw3Ecx3F6E01d3QHHcRzH6Wxc+TmO4zi9Dld+juM4Tq/DlZ/jOI7T63Dl5ziO4/Q6XPk5juM4vQ5Xfo7jOM4KR9IVkl6X9GQVuST9QtI0SVMkbZORHSHp+bgdUY/+uPJzHMdxOoPxwL4J+X7AqLgdDVwMIGko8F1gB2B74LuSVqu1M678HMdxnBWOmd0HvJ045ADgKgs8BAyRtDawD3Cnmb1tZu8Ad5JWooXokcpPUqukyZKelPQnSYPq1O6qkq6S9ELcrpK0aj3adhzH6eWsC7yS2Z8Zy6qV10SfWhvopiwws60AJF0NfAU4vySUJEBm1tbOdi8HnjSzw2M73wMuAz5bl14XYPGb07s0Ht2eW345Kb/78V8l5bbo/bR8zptJuQanZzts4fyknLbWpHjzXb6Rrg9c2WdkUt5M+i26bEBaPrptYFK+SEkxw9OXyBd+s31S/rVj70s3AFz8yI+T8l+M/V5S3j/nUzxm8cKkfKcpP0rKj9n+tKT8jMFz0x2oA3m/LpPeGZ6UH/jEWTX3oe8ao3I+LWna83vTb/WRxxCmK0tcamaXtuN0lfpqifKa6JEjvzLuBzaWNELSM5J+BTwOrC9pb0kPSno8jhAHS9pP0nWlypJ2l/QXSRsD2wLfz7R9NjBW0sh43H2SbpT0tKRLJDXFNpY7TyyfIel7sXyqpE077a44juPk0dZaeDOzS81sbGZrj+KDMKJbP7O/HvBaorwmerTyk9SHsIg6NRZtQphT3hp4DzgT2MvMtgEmAicR5pN3lLRSrHMw8Edgc2CymS15ro6vJwNbxKLtgW8CHwJGAp+WNLzKeUq8GcsvBk6u4+U7juPUhrUV32rnFuDwaPW5IzDbzGYBfwP2lrRaNHTZO5bVRE9VfgMlTSYompcJ05UAL8WFVIAdCQrtn/HYI4ANzKwFuAP4ZFSeHwduJgy9Kw21s+WPmNn0qBSvAXapdp5M/Rvi/8eAEZUuRtLRkiZKmnjZVde04zY4juPUQFtb8S0HSdcADwKbSJop6ShJX5H0lXjIbcB0YBrwG+CrAGb2NmHG7dG4nR3LaqLHr/mVCMt8vJctIlgQjatQ/4/AcQTLpEfNbK6kp4CtJTWV1grjtOaWwDOEoXi5cizNV1c7D0BpcaOVKu9HnD64FLp+zc9xnN5D+80iUm1V/Q0syY3wu1tJdgVwRd06Q88d+RXhIWDnuJaHpEGSPhhlE4BtgC8TFCFmNg2YRJjCLHEm8HiUAWwvacOoFA8GHsg5j+M4TveltaX41mD01JFfLmb2hqQjgWsk9Y/FZwLPmVmrpFuBIwnTlCWOAi6UNI0wonswlpV4EDiHsOZ3H3CjmbVVO88KubAVTF81d3UX0uRYc3YHVNF4bSl5X8q8K+yTNzfQlH7mrcezft5TdZ48z2K21vPXSpEBUZul3+dcM8ymnO9aZ3zWG+D71FF6pPIzs8EVymYAo8vK7gG2q9LG8cDxZWXvAF9InHq+mR1coa2K5zGzEZnXE4HdE207juN0LnWc9uxu9Ejl5ziO49SBAoYsjYorvzphZhMIa4WO4zg9gnoavHQ3XPk5juM4lfGRn+M4jtPraF3c1T1YYbjycxzHcSrj0549B0mthHBnfQjO6UeYWU405ELtzgDGmlk6MnOD8751c3+e7mAenoPlmPHn/dzkyvNs6Bcvyjmg61lotTkrtNboKlGP3/wmpfuwuKaQ051ED5727I1O7gvMbCszGw0sImR8WEKMK9cb74vjOM6ydG5sz06lt//I1yXjQ7XGJQ2VdJOkKZIekjQmlk+VNCQq2rcklVIk/U7SXiv4mh3HcYpRx9ie3Y1eq/zqnPGhGt8DJpnZGOB04KpY/k9gZ0I2iOnArrF8R0I4tPK+emBrx3E6HWtbXHhrNHrdmh9LMz5AGPldDqxD9YwPAP2AB82sRVIp48P1hIwPpybOtQvwGQhRXiQNi5nf7wc+ArxESGV0tKR1gbfNbF55Ix7Y2nGcLqEBR3RF6Y3Kr+4ZHxLnqpaB+L7YxgeAM4BPAQcRlKLjOE73oAHX8orSG5VfER4CLpK0sZlNkzQIWM/MniNEcbmcTMaHBPcBhwLfl7Q7IXHtHGBOTHLbz8ymS3qAkMj2+OpNdQ/61moLtKKtLfPa7wZf5tzA1jlj+7w72Jw3N5DzHuZZoxah1vWUvqrtfWrOuced8THIC2zd1AhzON3AOnpF4cqvAh3M+AAwRVryrb0OOAu4UtIUYH7Z8Q8DJbv8+4EfE1IgOY7jdA+6wcPiiqLXKb8VmPFhRJVTHlCljcMyr/9FLzY+chynm+Jrfo7jOE6vowGT1BbFlZ/jOI5TGR/5OY7jOL0NMzd4cRzHcXobPXjkJ7NGsLddcayIQNeSxhOc4n+dKTsQONrM/idRb14lg5wsH1l3z6pvWF+lgzoXCUqd58pw5+RLk/KPb/3VpLxPTvurql9SPtvSQZlnt76flA9vHpSUj2xK3n4AJi5+Iykf2JS+hjlt6T7ObVmQlK/WZ6WkPO8ef7R5jaT8OdLnB5jV9l5S3pxjvzW7NX2OnzE0KT+jablYEMtwyuJ0/eYcd44867MiKiGvjVsHpl0hJrXUHiN/wsy7agqfveDeyworiIF7fKkRQnUvwS0MV0yg62uAQ8rKDonljuM4jYHH9uw11CvQ9V3AppLWjuWDgL2Am+L+SZKejNuJnX6VjuM4RWhtKb41GK78IvUMdG1hlfgG4HOxfH/gXjObK2lb4IvADoQYol+WtPUKv0DHcZz24imNejSlQNcTgZcJocugeqDryYRILRuYWQtQCnTdhxDo+uZYJzv1mZ3y3AW40czei0Gsb2BpVoeKZLM6zHrv1Rov13EcpyA9eNrTrT1XXKDrfwJrS9oS+DBLFWG7F4WzWR1SBi+O4zh1pQGVWlFc+RWj3YGuzczieuBvgdvMrGTidx8wXtI5BEX4KeAwCnL347+qx/V0mDxrzr9OyulfazrvV9uct5LypsGrJeW2IJVkA2jO/8hvss0Xk/IL+m6RlA9pSV/jd/qkLVZ3HfiBpLwtx1JxXUtbm548fo+k/MQv/i0pB7j9sV8m5f+3/beT8r7NQ5LyPjkWr3c8dmFSfsYOZyXlx66a/pxZW+2Gi5YT2Hq3N4cl5edOSl9jp9CA05lF8WnPApjZG4RA1tfEINUPAZtGWStwK2G98NayqtcAWwLXZtp6HBgPPEIIbn2ZmU1asVfgFCVP8TlOr6IHG7z0+pHfigp0HcsnUWGa08zOB84v0hfHcZwuw6c9HcdxnF5HD572dOXnOI7jVMZHfo7jOE6vw5Wf4ziO0+vowbGfXfkVQNL6wFXAWoSYtpea2c8lDSW4N4wAZgCfM7N3JB0JXEmICHN3bONTBIf2z5rZ9bFsdeA14PhsEOwUtigdFLlm2tIpTPKCJue5MtDcNynWgHTQZvr2T8sXpgMu59Vf1JZvtTa1f9qEfZ3W9DlmzH89Kc8LUN4nR74oJ3h33j1uKuCKmudS8u+2Oen6OT+qh+QEB887f7+ca2juW9uIpogrRN4xecG1bVFOgPHOWI9rqa8Vp6R9gZ8DzQRL93PK5BcAJV+cQcAaZjYkykpJCABeNrP9a+mLuzoUowX4ppltRoj2cpykzYFvAXeb2Sjg7rhfYiqQdYo/BHiirN3PEtwmKjnPO47jdC11DG8mqRm4iOAWtjkwLv6OLj2d2TdiooGtgAsJA4YSpSQEW9Wq+MCVXyHMbFb0zyNGcHkGWBc4gODETvx/YKba/cD2kvpKGgxsDEwua3oc8E1gPUnrrsBLcBzHaT/1DW+2PTDNzKab2SKC//MBiePHsQIz4bjyayeSRgBbExzU1zSzWRAUJJBNlGaE7A77EN7gW8raWR9Yy8weAa4jBMV2HMfpPpgV3rIxiON2dFlr6wKvZPZnxrLlkLQBsCFwT6Z4QGz3oZgftSZc+bWDOIL7M3CimaUXNQLXEqY7K+XyO4Sg9ErHVZ36zH6oLrv6+vZ33HEcpyO0Y+RnZpea2djMVp75utIiaLWFz0OA62MErRIfMLOxwOeBn0kaWculucFLQST1JSi+q82sNA/9X0lrm9msmLtvGUsGM3tE0mjCXPVzMWB2iXHAmpIOjfvrSBplZs+Xnzsb2HrRK0/0XPMrx3G6F/V1dZgJrJ/ZX49g8FeJQwgJA5ZgZq/F/9MlTSDMwL3Q0c648iuAgta6HHgmhiYrcQshvdE58f/NFaqfBixjoilpE2AlM1s3U/Y9whv+/VRfbM6bHbmEurGq0lZ4eYGp8ywNNXDlpNzeT1tzts6YkpQDNI8YU1X24qTfssOOX0/Wn6W0RWuf5vQ9mrc4bbE7uzVt5bd6n3QUvIU5xgf/e8h1XHnHCVXlP7v9WE7c7+JkGzY3/T635PTh3ZxrXHlIun7b7LTF7Jqt6UmtloVpi9l6kBfYujnH4jXvHneGD561pq2/28mjwChJGwKvEn7vPl9+UPx9XA14MFO2GjDfzBZKGg7sDJxbS2dc+RVjZ0Lmhakxnx/A6QSld52kowi5AD9bXtHMbq/Q3jjgxrKyPxOmP5PKz6mNlOIDchVfTyCl+IBcxef0IuqoYM2sRdLxwN8Irg5XmNlTks4GJppZyS5iHHCtLesPsxnwa0lthOW6c8zs6Vr648qvAGb2ANXz8O1Z4fjxhMwN5eVHxpfLLdyZ2RSC+a/jOE73oM6+hGZ2G3BbWdl3yvbPqlDvX8CH6tkXV36O4zhOZdp6romBKz/HcRynMh7b03Ecx+l11NfgpVvhys9xHMepjI/8eh+SDPi9mR0W9/sAs4CHzewTkvYHNi8PzBqPnWdmg2M0mGeAZwkGM+8BXzSzZyXtTnCNeJFgvfQ68HkzS9pwa/BqdbrCjjHbFiXlTXn9ywksnefKkOcq0fyB0enz9xuYFL++8N10fWCN/iOS8qE5D8sD+6RdIQY3p+9Ra05A5OacoM6W4yZQBK08rKY+DGlOvw9z56XvQdPKw5Pyt5rS96hP/9pGNEUCW+fRqnQbXf1dB3r0mp9HeKnOe8BoSaVv6ccIvikAmNktlRRfBV6IgVi3JMT/PD0juz/KxhB8YI6r2ILjOE5XUMfA1t0NV35pbgc+Hl8vE2RV0pGSfhlfbyjpQUmPSkr56a0CvFNeGJ3oV64kcxzH6TLarPjWYLjyS3MtcIikAcAYQjDrSvwcuNjMtgP+UyYbKWmypBeAk4BshJhdo9P8y8BewBV17b3jOE4NWFtb4a3RcOWXIDqejyCM+m5LHLozS0eFvyuTlaY9RwInEmN0RkrTnusTkt9WDNezTGDrP9xQ6RDHcZz609pafGsw3OAln1uA84DdgdQqf5Fx/y0EJVdN9ueKDWcDW7/0eOPNLziO05g04HRmUXzkl88VwNlmNjVxzD8JQVoBDk0ctwvVo5CnZI7jOJ1PfZPZdit85JeDmc0krOml+DrwB0lfZ/nR28i4ridgEfCljGzXjGx2maxyfxbOry5sy5l6aCoQyT6njdmt6YwEtmBuuv2FtWVlyHNl0CppE/i2N15Kyl9/r4Crw+C0ifqw1vTT8pB+6awMqzXluAHkuJv0VfqZtm3KQ2l5gUmM5OcQaM7pw2o52UFmLhyQlG+e8zl6WQuT8tbFK/65P88dYk7O9zHP7adTLCx78MjPlV8VzGy5XygzmwBMiK/HE4NXm9mLwE6ZQ8+J5TOAir9ksa1V69Vfx3GcutOALgxFceXnOI7jVMZHfo7jOE5vw1oaz4qzKK78HMdxnMr4yM9xHMfpdfian5OHpDOAzwOtQBtwjJk9LGl14DXgeDP7deb4GcBcgn/gO8DhZpY2RYR8i85a6+Z82Ic3D0rXb875SOUEtm4eMSZdPycwdZ41Z9PqGyTla6w0JH1+YEZz+j62krbim/Xu20l5H6XrD2xKW0rOacoJCr3ZNkl589IQth1mQdvipHy2pa2GNxjQUtP5R1naWrRPv9oiCVqBAVGetefqrTnXmKd4OsO9oAeP/NzPrw5I2gn4BLBNDFK9F/BKFH8WeIgQJaacPeLxE4AzO6GrjuM4hbE2K7w1Gq786sPawJtmthDAzN40s9eibBzwTWA9SetWqf8gUE3mOI7TNbS0Ft8aDFd+9eHvwPqSnpP0K0m7AUhaH1jLzB4BrgMOrlJ/X+Cmzumq4zhOQTyrg5PCzOYB2wJHA28Af5R0JCHk2XXxsGtZfurzXkmvE6ZJ/1Ct/WUCW1/3l3p333EcpzI9WPm5wUudMLNWwtrdBElTgSMIU5lrSirF+1xH0igzez7u70FImjseOJuQ8qhS20sCWy/89z8a71PmOE5DYkUsexoUH/nVAUmbSBqVKdqK8GCxkpmta2YjzGwE8GOWBsAGwMwWEFIdHS5paGf12XEcJ5cePPJTT9bsnYWkbYELgSFACzCN4N4wz8y+lTluDHCtmW0eXR3GmtmbUXYh8LqZpTLBM3L4Nl36hh0w+INJ+Y1zn0nKF7WlzbvX6J92NXh9YTrwdF5g6jxXhhnP508rn7Ptt5PyvDfoRaXN/IfQNylf1dKuEOu2pk3sr7C0K8MqTWk3AYDp77+elI9badOkfD5pM/1JLW8l5a8sTLuLbDZwraT8tv9MSsrrgZR+H7YdPiopf3PRnJr78Pwbj6U7kcOcoz5W+PdmlcvvrOlcnY1Pe9YBM3sM+HCB46YAm8fXI8pkX1shnXMcx+kg1uJO7o7jOE5vo+fqPld+juM4TmUa0Xm9KK78HMdxnMq48nMcx3F6HT7t6XQXruwzskvPf+biN5LyC/pukZRP7Z82CJuldEDkNfqPSMsHp9vPC0p9xtgzWNnSHkDfeixpkEvb7LQl5NTdzknKN/n0wqS85dV5SfmA/dKBq2//ftqKsIgF+O/7pgOEb/HZ+Ul500brJeXH/mRRUn5eUzrA+Vdb09e41fCu/R4BrNG8UlL+k+bhndST6vTkaU/388tBkkn6XWa/j6Q3JN1adtzNkh6sUP9wSU9KekrS05JOjuXjJb0o6YkYFu2qROxPp5PIU3yO05uwFiu8NRr+Tc/nPWC0pNKj5sdg2ZwvkoYA2wBDJG2YKd+P4MC+t5ltEY+Znal6ipltCWwCTCKEO0vnq3Ecx+ks2tqxNRiu/IpxO/Dx+HoccE2Z/DPAXwjxO7MRXE4DTi5leDCz983sN+WNW+AC4D/AfnXuu+M4ToewtuJbESTtK+lZSdMkfauC/Mg4szY5bl/KyI6Q9Hzcjqj12lz5FeNa4BBJA4AxwMNl8pJCvIZlg1ePBh5rx3keB5YLjZENbH3L/Ont6rjjOE6HqePIT1IzcBHhAX9zYJykzSsc+kcz2ypul8W6Q4HvAjsA2wPflbRaLZfmyq8AMTLLCIJiuy0rk7QmsDHwgJk9B7RIGt3BU1W01jCzS81srJmN3X/QRh1s2nEcp33UeeS3PTDNzKab2SLCoOKAgl3ZB7jTzN42s3eAOwmp4DqMK7/i3AKcx/JTngcDqwEvxnidI1g69fkUIdVRUbYG0sExHcdxOglrKb4VYF3glcz+TCon8f6MpCmSro85UdtTtzDu6lCcK4DZZjZV0u6Z8nHAvmb2IEA0eLkTOJOQxeFcSZ8ws/9I6g8cY2a/yDasEAH3a4SM8HekOtGcGzZ5xTKwKW2PM6Ql7aqwTmv/pLxPc7r9oTkJo4e1pu9PK+mg0O805d/fPFeGplXXSMov6ZP+2n3p+vQz6aK21ZPybTaekZQPU/o9mEf6PQTom+My8tqtaXeNVVafmpQPJR2Yurkp7QqxrlZOymcseicpb8v5njVVnqRpVxsDlP4c5H3XO8PGpOhaHoTlGUJO0xKXxnRsSw6pdIqy/b8A15jZQklfAX4LfLRg3Xbhyq8gZjYT+Hm2TNII4APAQ5njXpQ0R9IOZnZbnBa9Kyo4IyjREj+V9G1gUGxjjzgd4DiO0+W0R/ll845WYSawfmZ/PUL2m2wb2XQevwF+kqm7e1ndCcV7tzyu/HIws8EVyiaw9MYvN/Q2s20yr68ErqxwzJH16qPjOM4KweqapehRYFScHXuVsDz0+ewBktY2s1lxd3+WLgP9DfhRxshlb4I1fYdx5ec4juNUpD0jv9y2zFokHU9QZM3AFWb2lKSzgYlmdgtwgqT9CXlR3waOjHXflvR9ggIFONvM0kkdc3Dl5ziO41TE2uqbn9bMbqPMYt7MvpN5fRpVRnRmdgXLLhvVhCs/x3EcpyJtrQ2VnL1duPJzHMdxKlLPac/uRo9QfpIM+L2ZHRb3+wCzgIfN7BOZ424G1jCzncrqHw6cSjCnFWEu+jxJ44HdgDnAQIJF5mlm9mqsNwMYa2Zvxv3dCeHMPiHpyCg7XtImwK+BIUB/4H7gzyy1ZNqYsAC8AJhiZodXu9bLBlS37lWO+bUVsAzOa2PO4veT8u/0SRurzpifdhOYl9P+wD45rhb9lrNPWoZZ76aXCQ4YNiYph/ysDHmuDBdPPDcpb32+PIDQsrRN/ldS/tYVTyflf5+bzsyxwYC0KwXABX1WTcovPTXnPo74YFJ86+cuTsrnr5QO9nDzrPQ9OnDtsUl5ZzCzJZ154soBQ5PyInpp13b0pxL1nvbsTvQI5Ucm+LSZLSAdfHqepA3N7MVYng0+/VoMYXZYpuopZnZ9dFU4kRB8enQ7XRJ+AVxgZjfHc37IzKYSFn6RNIGgNCe2/9Idx3FWDAWyWzUsPSnCS3cOPr02wU+l1Fbaw9dxHKcbYG0qvDUaPUn5dVXw6XtLEciBy6rUuQC4R9Ltkr4RR6GFyQa2fnbui+2p6jiO02HaWlV4azR6jPLrwuDTe5QikANfqlQhOrpvBvyJEKXgoRjqrBDZwNabrLxhfgXHcZw64CO/xqHbBp82s9fM7AozO4DgwNlR5es4jtMpmKnw1mj0FIOXEt0i+HQ5kvYF7jazxZLWAoZRZpBTlNFtA6vK8t7MItZhfXIWuB9pWZCU7zrwA0l5X6UDS89uTbc/uDk9YF6tqfr9AeiTc/4h9E3KATb5dDpoc15g6jxrzuZROyTltjB9j4bu/Z+kfMjl6fpF2NIGpQ8YOjwpbhq2flLevyn9PnwoJ0B6np3GnLa0vVqeZXSeVXQR8gJfb9k6oOZz1Iq7OjQI3Tj49N7AzyWV7PhPMbP0L5TjOE4X09aAI7qi9Ajl11XBp81sRLVzmtl4YHx8fRJwUqKd3VPncRzH6QoacTqzKD1C+TmO4zj1pxGtOIviys9xHMepSCNacRbFlZ/jOI5TEV/zcxzHcXodvubnVEVSKzCVcC9fBA4zs3ejbAvgQmA9gnP8VcAPzELEvBhX9PvASlF+q5mdnDrfosRnsTWnr0WslvPaWK3PSjnnSJtv57karN4nHZi6Naf9uTmGuAOb0oGxV7V0/wBaXp2XlC9qSweGzgtMnefK0Gf07kl5y6vpKEAj+qXbf3nRO0k5wOK8X47WlqS47ZmHkvIxA9ZOnz/Hl2GH1TdJyt9pnZ9uIIemAq4Oed+FZqVdYlLfdch356gHHtvTSbEgRngZTcg8fByApIEEp/tzzOyDwJbAh4GvRvlo4JfAF8xsM4LT+/Qu6L/jOE5F2kyFt0bDlV99eZClbhWfB/5pZn8HMLP5wPHAt6L8VOCHZvbvKG8xs191cn8dx3Gq0tamwluj4cqvTkhqBvYkjPYAtqAsYLaZvQAMlrQK7Q+o7TiO06n4yM9JMTBmdHgLGEoImwZhDa/ajHm7ZtKzWR0emfd8x3vqOI7TDnpybE9XfrWzIGZ02ADoR1zzIwTMXiZdtKSNgHlmNpd2BNTOZnXYfvCo+vXccRwnQU8e+bm1Z50ws9mSTgBulnQxcDVwuqS9zOyuaADzC+DcWOWnwA2SHjCz5yQ1ASea2fmp8wxPmGPmBaUuMi3fnNNGnxwLtXUtbU25qDkdEHlhTiTd5hwru745/ZvTlA6I/CRz2ad1leQxA/bbJinfZuMZSflbVzydlOcFps6z5uyzzxeT8tbTHk/KBzSn30NIfw4BeOftdB+mpa8h730cljYmZaUcq96n35uZlOdRxNozj3UGDEvKh+Xc484YufRgY08f+dUTM5sEPAEcYmYLgAOAMyU9S3CHeJRg4VnKP3gicI2kZ4AnCRkjnC4kT/E5Tm+ita2p8NZo+MivRsqDapvZJzOvpxKS11areytw6wrrnOM4Tg304IxGrvwcx3Gcylgdpne7K678HMdxnIq09eBFP1d+juM4TkXafOTnOI7j9DZ82tMphKRPATcAmwF9gd9F0QeA2XF7E/gS8AzwLME3cCJwlJktzjvHF36zfXVhU47F1eJ00GcAckzMX/nqo0n5yeP3SDc/IB0YW0PWSspt9utJeduUdMDkps3Sbgp7HHJFUg5w+/fnJOXDlHan+PvcN5LyIZenA0/nBabOc2W48fELk/Kjx56SlAN84eLE5xD4zDF3JOVvtKSDg9+5WzrAeP9DP5mUr3dU+nOw3prrJeV51MOpe/Gi9DWOvHBsUk6OW1A9aO3Byq/x7FO7N+OABwiuDlNjwOutCCHPTon7e8VjX4iyDxGyPnyua7rsOI5TmbZ2bI2GK786IWkwsDNwFHBI0Xpm1go8wtKA2I7jON0CV35OEQ4E7jCz54C3JaXn1yKSBgA7AOl5IsdxnE7GUOGt0XDlVz/GAdfG19fG/RQjMwGxX44RXyqSDWx9+e3pRKiO4zj1ok3Ft0bDlV8dkDQM+ChwmaQZwCnAwZJSH4nSmt/GwI6S9q92YDaw9VH7fbieXXccx6lKGyq8FUHSvpKelTRN0rfT2BRSAAAgAElEQVQqyE+S9LSkKZLulrRBRtYqaXLcbimv215c+dWHg4CrzGwDMxthZusDLwK75FU0s1mEBLenreA+Oo7jtIvWdmx5xJynFwH7AZsD4yRtXnbYJGCsmY0BrmdpIgCIGXTiVnWwUBSZ9WAX/k5C0gTgHDO7I1N2ArCZmR0raTxwq5ldH2Uj4v7ouC9gMnC8md2fOteXRhxU9Q2rx6Kz5cRxb8n5vAxS2ny7HtHwU7Tl9D8vK8QLrXNzzzFIaQ+hATn34NWW/HOkmNv6fvr8OVkZNu07NCm/dOJPc/vw1bH/l5TPbE27MuR9Voc1DUjK8z5ni3M+B4NI168HeZ/Fvjljj/mFVEqaK2ZcX9MX7vq1Dy2sIA6adXXyXJJ2As4ys33i/mkAZvbjKsdvDfzSzHaO+/PKYynXgvv51QEz271C2S8yr48sk80gZHIv7Ruw5QrroOM4Tgeo89BoXeCVzP5MgrFfNY4Cbs/sD5A0EWghDDZuqqUzrvwcx3GcirRnNknS0cDRmaJLzezS7CEVqlXUr5K+QEgGvlum+ANm9lpMCn6PpKlm9kI7urgMrvwcx3GcirTHijMquksTh8wE1s/srwe8Vn6QpL2AM4DdzGxhpv3X4v/pcalpa6DDys8NXhzHcZyKtKLCWwEeBUZJ2lBSP0IwkGWsNuM636+B/c3s9Uz5alKIGyhpOCGgyNO1XJuP/BzHcZyK1NN/z8xaJB0P/A1oBq4ws6cknQ1MNLNbgJ8Cg4E/RU+xl6Nl52bAryW1EQZt55hZTcqvIa09K1n9SDoLmGdm50Xryo8BG5nZwvikMNHMRkRLy1JQ6RLnm9lVsZ2tgceBfc3sb5n2W4GphAeGF4HDzOzdsvaWCVItaXfg5nh8iZPN7C5JawIXADsC7wCLgHPN7MbUtS9+/fkufcP22/b4pPz2x36ZlNuCtKWjzX0rKdfKw9L1F85PyvMYvfPXc4/5fd8NkvK+zWkrvQua0paGW9qgpHxxzg/S8BwjwWRwdOCEY5MGxwD8auJPkvLfbP2dpHxgzmLS5pZ+H7eZcl5SfurY05Py4wa9k+5ADm11CGz9xLvpz/KBk9P3kLb8Fbm+a29WU0fHr/uFwr83R776+4Zyde/J056twP9Wkb2Q8RfZqqT4IqXg1OURWko+JqOBt4HjytujcpDq+8vOdVd0bbgJuM/MNjKzbQlTALWFmnccx6kj1o6t0ejJyu9nwDekHKesDFEpHQQcCewd425W4kEqBKJuR5DqjwKLzOySTN2XzCyda8ZxHKcT8fBmjcnLhBHcYRVkIzNhciZL2jWW7wy8GM1nJwD/U14xRinYk7KF2iirFKR617JzjQS2IEytOo7jdFs8q0Pj8iNCnM3y6yyf9iwtcqSCUw/MBKIeCtyZkaWCVJdPey5nmivpIklPSKqYKTYb2Pqyq66tdIjjOE7daVXxrdHo0daeZjYtKqXcRLFxRPcZYH9JZxAcModJWtnM5hLX/CStCtxKWPMrRXF5IcrWBiZI2j9aLlXjqXiuUj+PKxnlVLmOJf4zXW3w4jhO76ERR3RF6ekjP4AfAicXOG4v4AkzWz8Gp94A+DMhT98SzGw2cAJwsqS+ZbKiQarvIYTqOTZTljbxcxzH6WR68rRno478Bkmamdk/v9qB0Y/kcSCbXLY0TVniiigvdzP4M3As8LuyNidJeoJgoVluF34TcFZmHXHXsnP9wMyul3QgcIGkU4E3gPeAdLRg4Bdjv1dVVo8nmbw2mnOO+L/tv52U/7ttTlLeYumvUV5g6mal+7egbXFSPm6lTZNygC0+mzbDf+3WhUn5paeOSZ9g6PC0vLUlLX/n7aT4M8ek8yYXcX/Kc2X48qSzk/LWaRVn+JfwxYOvScr32TJ9/slKu8ycMK9vUp53D9LZyorx5jI/Ycvz2nbfT8qLKJxvvPz7dvRoeXryNFNDKj8zS/7CVQgk/enM6xnAwILnuYVo2FLuV2hmn8zspoJUr1ql7VkE5ek4jtMtaUQrzqI0pPJzHMdxVjyNOJ1ZFFd+juM4TkVqzyjYfXHl5ziO41TEpz0dx3GcXodPezrdhv4J86vO8FuZ3bogKe/bPCQpz7Oiezen/SHNaVul1dQvKZ9t7yfl9y7+Dzv0XSN5TNNG6RCsq6w+NSlnxAfT7Q9bPylve+ahpLx12otJ+Rst85Lyoc0rJeWQH5g6z5qzeePtkvLXWy/POX+6j9MX/Tcp32TQOkl5HipgBmk5tpIDciIv9u0GppbdoAsrjNzfS0nLfVMknSXp5Ph6vKRXs7mWJM2Ir0dIWlAW3uvwTDtbSzJJ+5S13xqPfVLSXyQNqdDe05KuKvnaSdpd0uyyc+0VZWtK+oOk6ZIek/SgpE8lrnmQpKslTY19eEDS4Gr3I5YfLenfcXtE0i4Z2QRJz8YoLv+UtElZeam/1+e9H86KJU/xOU5vog0rvDUa9Rr5lTIoXFxBVsp4UIlsBoW/ZcoXlOpI+i0hmsoPs+3FiCx3EqK3XB1l95vZJ7InyGRQ+K2ZfT6WbQDsn7ierwP/NbMPxeM3Aao6iEn6BHAMsIuZvSlpG+AmSdub2X/iYYea2URJRxNyVu2fLU/0xXEcp0voyQYv9Zop62kZFNYGXs0c/6yZpTyX/w84xczejMc/DpSUdjn3ARvn9NlxHKfL6ckRXuql/HpaBoUrgP+L06M/kDQq5/gtgMfKyibG8nI+SUiKW+LqTH9/WqnxbGDrf857vug1OI7j1ERPTmlUT4OXHxGU1F/LyqtNe5ZnUDgMuCHulzIojCAolUoZFEYB11fIoFA+7UnZ/kXALoTRYMVVdzObLGkjYG9CzM9HJe1kZs9UOr4KYtn14qslLQBmAF/LlOdOe2YDW/9y/eKZlR3HcWqhEdfyilI3A0Ezmwa0N4PCd6JxzIXAfpJWjoeU1vw2APpROWv6xsCOklJrdxAyKCyJ62lmxxFGk6vnXM88M7vBzL4K/J4KI9MMTwPblpVtE8tLHBpTGh1oZq/k9NlxHKfL6cmZ3Ovt6vBDlh/5VaKUQWGJlWc0bDmQTBBpM5st6QTgZknLGNOY2SxJpQwKqfRB9wA/knSsmZXaSGZQkLQz8LSZvSOpH7A5YWq2GucCP5G0r5m9JWkrwlrmDqnzdIQxi6svPTbnfAQXpkOiAtBX6dn7nzUNTcr7tKRdFQ5pSrsirDwkff658/on5TMXVls6DmwwIB0U+hst6YDIAMf+ZFFSPpS1kvJbP1fJLmwp/ZvSQZfHDFg7Ke+bE9z7zt2ak/Kv/it9DwE2b00H984LTJ3nynDH5EuS8sfGpBO1/PvGk5Lynx92V1Ke902pxxrXajkaY/PFabccywnyXg8acS2vKEWUX2/MoDASuDga5TQRFPqfo2y5+2Fm50taF/iXJAPmAl+IwavzKE2HArxpZnsVqOM4jrPCaW3IMV0xcpVfb8ygYGZXAVdVkVW8H3FUWfGR3sx2b0+54zhOd6C3j/wcx3GcXkhPNnjp1covRpb5SVnxi2ZWNfqL4zhOb6Hnqr5ervzM7G8sG1nGcRzHifi0p+M4jtPr6NUGL05xJA0D7o67axFC470R928k+EC2Eh6ojjGzhyVNIIRTK1l8/sDMqga43mnKj1ZAz4uz7zaVIrYt5Y7HUlHjwBbMTcrbZr+elDetPDwp33zhe0l5Hq985JTcY85rSttwNTelXSHmr7RRUv6h1rQ7x+Kc36NhaW8O+h/6yaR80IP/SjcAbDPl3KR8ny2/k5TnZWXIc2XYdsp5SfnJY09Pyo8b9G5SXg/M0q4Ik2en3YZ2fOKHSTltK35c5mt+TiHM7C2gFJD7LGCemZ0naSeCi8g2ZrZQ0nCC834JD27tOE63o+eqPld+ncXaBB++hQClANiO4zjdmZ488uuM/KcO/B1YX9Jzkn4labcyeTa49bDyytnA1pdddW252HEcZ4XQk7M6+MivEzCzeZK2BXYF9gD+KOlbZjY+HpKc9swGtl78+vM991HMcZxuRV42+kbGlV8nEfMPTgAmSJoKHAGM78o+OY7jpHBrT6cmYib4NjMrJePbCnipI20ds/1pVWV5c9hFPsjNOcFyT1mctlA7Y4ezkvJ+Oe2v2Zq+irea0tfwslI5h2GUpYM2bzYwHZQa4Kutc5LydZckJ6nMzbPS1pR579IOq2+SlK+UEzx8vaMeSsoXD8z/nJyaY005WekA4dMX/TcpzwtMnWfNed7EtFX0sWNPTco7gwH902a5d+5wRlJeZKrxihlVDccL0YjTmUVx5dc5DAYulDQEaAGmAUd3bZccx3HStJmP/Jx2YmZnZV4/Bny4ynG7d1KXHMdx2kW9VZ+kfYGfA83AZWZ2Tpm8PyGpwLbAW8DBMUECkk4DjiL4Sp8QI3R1GLf2dBzHcSrShhXe8ohJzC8C9iPkSB0nafOyw44C3jGzjYELiLGX43GHAFsA+wK/iu11GFd+juM4TkWsHX8F2B6YZmbTzWwRcC1wQNkxBwC/ja+vB/aMeVUPAK41s4Vm9iJh6Wj7Wq7NlZ/jOI5TkRas8Jb1R45buV3DusArmf2ZsaziMWbWAswGhhWs2y58zc9xHMepSHv8/LL+yFWoZOpdfoJqxxSp2y66pfKT1ApMJfTvGeAIM5tfh3ZnAGOLhheTNAK41cxGx/1dCDE6V4mHnB/f8E7jjMHpwNC1Yjm2zS8uGJKUH7tq2sS9uW/6BC0L09P4ffq3JuWti9OTGX36vZOUf+eJfyflAFsNH5mUz1iUPseBa49Nyue0pQNjv9Oa/io8/d7MpHy9NddLygctWCUpBzhuUPoaT5jXNynfZNA6SfnPD7sr5/zpwNR5rgwXT0wH5u4MrhuTDv69w5AVH3w7jzq7OswE1s/srwe8VuWYmZL6AKsCbxes2y6667TnAjPbKiqdRcBXskIFOrXvktYC/gB8xcw2BXYBjpH08c7sh+M4TmdhZoW3AjwKjJK0oaR+BAOWW8qOuYUQAATgIOAeC43fAhwiqb+kDYFRwCO1XFt3VX5Z7gc2ljRC0jOSfgU8ToiVubekByU9LulPkgZL2k/SdaXKknaX9Jdsg5m2fiPpKUl/lzQwyraV9ISkB4Fs/p7jgPFm9jgsCU59KvCtWG+8pEsk3R9jeH4iljdL+qmkRyVNkXRMpl8TJF0v6d+Sro4Lu47jON2Celp7xjW84wkJxJ8BrjOzpySdLWn/eNjlwDBJ04CTiL+vZvYUcB3wNHAHcFyMmtVhurXyi8Pe/QhToACbAFeZ2dbAe8CZwF5mtg0wkXCz7gR2lFRKGHYw8McKzY8CLjKzLYB3gc/E8isJPiQ7lR2/BfBYWdnEWF5iBLAb8HHgEkkDCKa7s81sO2A74MvxyQVga+BEgtnvRsDOVe7DkoXka99OT2k5juPUi1as8FYEM7vNzD5oZiPN7Iex7Dtmdkt8/b6ZfdbMNjaz7c1seqbuD2O9Tczs9lqvrbsqv4GSJhOUy8uEpwGAl8ysFJtpR4LS+Gc89ghgg/h0cQfwyag8Pw7cXOEcL5rZ5Pj6MWCEpFWBIWb2j1j+u8zxovICa7bsOjMrhTGbDmwK7A0cHvv4MMFyaVQ8/hEzm2lmbcBkgvJc/gRml5rZWDMbe8jQ9HqN4zhOvajnyK+70S0NXohrftmCOCOYTdMt4E4zG1eh/h8J05RvA4+aWSUrkWwQyFZgINUVHMBTwFiWnaPeljAML1Fet2Sl9LXyaASSdq/Qh+76fjiO0wspuJbXkHTXkV8RHgJ2lrQxgKRBkj4YZROAbYAvU3nKsyJm9i4wO1p1AhyaEV8EHCmplKl9GCH6QNZs7LOSmiSNJExjPkuY3z5WUt9Y74OZKdlOxdrytzyacjZrU3LrDdS8NlK7M3GXU6txRN7nrCfQhCW37oDn8+uGmNkbko4Eronx4CCsAT5nZq2SbgWOZKnlUFG+CFwhaT5BcZXON0vSF4DfSFqZMKL7mZlljWmeBf4BrEmwCn1f0mWE6czHo0HLG8CB7eyT4zhOp9MoD1sdoVsqPzMbXKFsBjC6rOweghFJpTaOJ1gWZctGxJdvZtsys/Myrx8DtsxUOysju6/a+SL/NLNvlJ2zDTg9blkmxC3bX8dxnG5DI67lFaVbKj/HcRyn62ktshbSoLjyqxNmdmRX98FxHKee+LSn4ziO0+vwZLaO4zhOr6Pnqr4GVH7dMeh19Nk72cw+UWs/aqEzpudrPUWt7g411++Eb3NTxQD0xVFO/bz28+RmtbuctOW0kRepTznvQ89daVrK+w0QzbAnG7w0ostMtwt67TiO0xPpyRFeGl1JdJeg1xWRtKekSZKmSroiRiTfXtINUX6ApAWS+kkaIGl6XpuO4zidRau1Fd4ajYZVft0s6HWl/g0AxgMHm9mHCNO0xxKU89bxsF2BJwm+gzsQYn86juN0C/KiDTVa5KEsjaj8umPQ60psEtt5Lu7/FvhI7MM0SZsB2xOS436EoAjvr9SQZ3VwHKcrqHM+v25Fwxm80D2DXlcitZp9P2HUuhi4izBCbAZOrnRwzBZ/KcALo/dpvE+Z4zgNSSOu5RWlEZVfER4CLpK0sZlNkzQIWC+OwiYQRovtDnotabakXczsAZYNel2JfxNGjBub2TTgMELcT4D7gKsI07RvxCDZaxEyR6T7kZhaz7PAA2jKMbPLayNvqiDPkjDPWrNWS8Tc9nPk00Zvxqin/l1TH1b0D0ajtw+1O0/Xw2K1q2nOkRf5Pq9oGnFEV5Qeqfw6O+h1ZE9J2TnJz8bj/xSnWB8FLomyhwnBr++L+1OA120Ff9LyFJ9DzYrPcXoSrT3Y6aThlF93DHptZhMIU6OV2Lq8wMwWAP0z+0dXqes4jtNleIQXx3Ecp9fRiFacRXHl5ziO41TER36O4zhOr8NHfo7jOE6vw0d+3ZjuFuiaEKnlZWAjM5udkd8E/MHMrqtS/8h4vmRG90nvDK/eh5w+Li5gOd2U81l/aGBrUr7bm8OS8uacJ8nmnC9ba04w4DlNaQPy1VtbkvJth49KygHWaF4pKR+g9NdqZsucpDzP1aA5J3TtOgPS78HiRel71LdA7Isn3k2f402lgzHk3aPVcj6Hk2cPTbffP/0+XzfmO0l5U857UCQodZ4rw7gnzk7Kb/jQt3PPkccHa6zfiGHLitKIEV7K6VaBrs3sPeDvwIGZPqwK7EJQjo7jOA2BhzdrHLpLoOtrgEMy+58C7jCz+ZKGSrpJ0hRJD0kas6JuhuM4Ti2YtRXeGo0eo/y6WaDrO4BtY+QWCIrwmvj6e8AkMxsDnE6I9OI4jtPt8JRG3ZtuF+jazBYBtwAHSRoObEWYCoUw/fm7eNw9wLDYVlWyga3vmj+twC1xHMepHQ9s3b3proGuryGMNgXcbGaLM30pJ/nJyQa2/tPahzbep8xxnIakEUd0RekJI78iPATsLGljAEmDJJUMoSYA29CBQNfAbEm7xKLyQNf3EqZLj2PplCeEeJ6Hxn7sDrxpZmnzP8dxnC6gta2t8NZo9ISRXy5dEejazNok/ZkQ4Pq+jOgs4EpJU4D57T3ngU+cVV2YY+ZfDy7eKh2G9NxJFybltmhBWj73raRcg1dL13//vaQ8mRYDePMjFbNKLcNPmqu7m0C+O8eVA9Jm+lu2DkjKF+VY2Q9Le6Mw8sKxSfn84x5INwAcODltpv/adt9PyvvmDCg2X/x+Ur7jEz9Myu/c4YykfIch76Y7UAfysjLkuTJ8emr6HnYGjWjFWZSGV37dMdB1Rv514OtlZW8DB1Tow3hCXj/HcZxuQSOu5RWl4ZWf4ziOs2LoyWt+rvwcx3GcivTkkV9vMXhxHMdx2klnGbzE4B93Sno+/l9ucV/SVjFQyVMxSMjBGdl4SS9Kmhy3rcrrl+PKz3Ecx6lIJzq5fwu428xGAXfH/XLmA4fHYCP7Aj+TNCQjPyWGutwq45ddlR6v/CR9SpJJ2jRxzHhJB8XXl0navMIxfSWdE59MnpT0iKT9omxGdGbvWtpaV/xWK9aW3tpytq5uvwBtNW65l5izNeVsufeo0EWm72Pt16jktqLP7wQ60cn9AOC38fVvycRGzvTlOTN7Pr5+DXgdWL2jJ+zxyg8YBzzAsrE2q2JmXzKzpyuIvg+sDYyOQbQ/Caxct146juN0M9rMCm81sqaZzQKI/9dIHSxpe6Af8EKm+IdxOvSCjEtbVXq08pM0GNgZOIqM8ouZHn4p6WlJfyVzoyVNkDS2rJ1BBCf4r5nZQgAz+2+l9ESSToojwyclnRjLVpL01xgE+8nSXHUMjP0PSY9J+puktet/FxzHcTpGe7I6ZMMwxm0Zp2BJd2V+G7Pbcq5fKeLv5O+AL9rSiNqnAZsS3NmGAv+X105Pt/Y8kJBN4TlJb0vaxsweJ2RZ2AT4ELAm8DRwRaKdjYGX8yKxSNqW4Pi+AyGM2cOS/gFsBLxmZh+Px60qqS9wIXBAdMI/GPgh8L81XK/jOE7daM+ILhuGsYp8r2oySf+VtLaZzYrK7fUqx60C/BU4MxO7uTRaBFgo6UogN1pFjx75EaY8r42vr437AB8BrjGz1jh3fE+dzrcLcKOZvWdm84AbgF0JmSb2kvQTSbvGJLebEJzn74zBts8E1qvUaPaJ6rKrrq10iOM4Tt1ps7bCW43cwtJoV0dQIcGApH7AjYRsPX8qk60d/4sw6Hky74Q9duQX0wl9FBgtyQiJlU3SqfGQ9kxSTwM+IGnlKoGvl5y2UmEceW4L/A/wY0l/J7yJT1VIh1Sp/pInqsWvP99zHW8cx+lWdKKf3znAdZKOImTn+SxAXIL6ipl9CfgcYeAyLIarBDgyWnZeLWl1wm/wZMqSmleiJ4/8DiI8IWxgZiPMbH3gRcLo7D7gEEnN8Ylhj1RDZjafkCrpF/HpA0lrS/pC2aH3AQfGwNkrEaZX75e0DjDfzH4PnEcIpP0ssLqknWJ7fSVtUadrdxzHqZnOsvY0s7fMbE8zGxX/vx3LJ0bFh5n93sz6ZtwZlrg0mNlHzexDZjbazL4QZ97qd3GNtBGyNexbVnYCcDHh6eCXhLW+m+J2UKbethXa6wecSxgFPgk8DOwTZTOA4fH1SVH+JHBiLNsHmEJ4InkUGBvLtyIozCeAp4Avd+A6j67xPjV0/e7Qh66u3x360NX1u0MfGr1+b9sUb5oTkTQV2N/MXuzqvhRB0kQzS4fp78H1u0Mfurp+d+hDV9fvDn1o9Pq9jZ487dluJN0JTG0Uxec4juN0jB5r8NIRzOxjXd0Hx3EcZ8XjI7/Gp6pfTS+p3x360NX1u0Mfurp+d+hDo9fvVfian+M4jtPr8JGf4ziO0+tw5ec4juP0Olz5OY7jOL0OV369EEkfjZkqnE5G0viu7kM1JO3c1X3IQ9KnM6+Xy/bdm5C0pqTLJd0e9zeP4cGcArjBSwMi6YPAKcAGZNxVzOyjBetfBewIvAXcH7cHzOyd+vc2ty8jCQHHD7GQJzHv+OHAccA7hEwcPyUED38B+KaZTetgP4YAx5nZDwsc2wfYj5BCBeAZQvaQlgJ1HzezbTrSx0wb5wLTzeySsvJvAGuZWdV0LpKaCTES1419flLSJ4DTgYFmtnWB8x8ArGdmF8X9h1maVPRUM7u+QBvXmdnn4uufZPss6e9mtneVekvuXz3uZWznw8AIlv0uXVWw7gvAQ4Tv0H1WORdopXoXkogvbGYnFGjjduBK4Awz2zJ+LieZ2YeK9KG3435+jcmfgEuA3wDtTq9uZocDxJijBwEXAeuQ83mQNJdlv7CK+wrN2ipFzh/jqR4MfB4YA/yYpRk38vgDMBEYBTxC+PL/nKAALwN2zzn3+sC3Cdd7U2zv+8BhwDUF+r4OcC8wC5hEuPZPAP9P0h4WsoSkGCRpa6oHQX88rw/xfJUeFH5OCKOXymV2ObA+4d79QtJLwE7At8zspgLnBjiVZZND9yfkUVuJ8H7kKj/C+1fiY2V9TmXnVpXXHULS74CRhNCDpe+SAYWUH7A5IYXZrsB5kjYFnjCzT+XUm9iB7pYz3Myuk3QagJm1SGr370FvxZVfY9JiZhd3tHIMyL0rIZ/hm4Q4p/fn1TOzmjLXS/oyQcmtB1wHfAm42cy+145m1jSz02PqkpfM7Kex/N+SjitQ/yrgH8CfgX0JT+1PAWPM7D8F6v8IuNjMfpYtlHQCQYkfUbHWUtYF/h+Vf7iNkIkkDzNbPoeMmbXF+5JiLOFa2yQNILz/Gxe89hL9zOyVzP4DZvYW8FYM6F6E1JRTSjYwPjw0AQPKHyQKPjxkGQtsbh2fAmsFFsf/bcB/qZKLLouZ/Ta7L2nlUFwgIPNS3ovZayy2sSMwux31ezWu/BqTv0j6KiEt0sJSocVI6AX4GWGa8BLgXjOb0d4OSNqSoEAhTPdMKVDtIuBB4PNmNjG2094fnVYIvxKS3iyTFUkqNtTMzoqv/ybpv8B2ZrYwUSfLjmZ2ZHmhmf1C0rMF6k8rOj2dYL6kUWb2fLZQ0ihgQU7dRSXFaWbvS3qunYoPYJm1NjM7PrObGrVlGZRRYgMzSkzAwES9/wDnV3gNxR8esjwJrEUYyXeEOYR8necDv4kPAYWRNJqQlXxo2NUbwOFm9lSB6icR8uCNlPRPwr0/qD3n7834ml8DIqlS7FEzs43a0cYWhNxYuxCmoJ41s8MK1v068GVCsl4IqZsuNbMLc+oNJ+TpGgesSRj9HWkh3VTRfr9LyIQhgvK9ryQCdjGzpBGEpCcIU6Ol0cK92f28BwhJk6qti6Vk7TkmD0n7ARcCPwAei8VjgdMImURuS9SdT8hMAuGaR8b90tT1mALnvxqYYGa/KSs/BtjdzHKnsCVNIL3mlUwzVi8k3UvIrvIIyz5I7l+w/gGE79D2wCLgX4SHwbsL1v8XYc3u3ri/O/AjM/twwfp9CImxRfgOLy5Sz3Hl1yuRtD+7EnoAABr+SURBVAqwM7AbQYEMBx4ys7wpu1L9KcBOZvZe3F8JeLDID2emjfUI60bjgEHAjWZ2eoF6u6XkZvaPnPozCCPEitOOeQ8QkqYDJ1cSAeea2cic+nub2d8z+30J63evmlnudFmm3miC0VNp7e8p4KdmNjWn3gYpuZm9VODcaxDWSxcCpWnGbQlrfwea2X/z2ugokrYDXimNViUdDnwGeAk4qx2zH6X2Kn6e8j5HFdrZlGAEdSKwhpmlRq/Zek+Y2ZZ5ZVXqfpZgtDRX0pmEPKE/6MDUb6/ElV8DEt0UTgI+YGZHx+muTczs1oL1pwAPxO0+M5vZzvNPJUwVvh/3BwCPdtTKLFqvjmvn2l+XIOnKlNzMvphT/xLgQjN7StKqhGngVsK018lmlmt0U9be4HDa8CBS4PiqlpTtRdJHgVIC5qfM7J521B1FsNTdmDBteLKZvVqg3uPAXmb2tqSPANcCXyOM3jYzs3ZP+8UHglFmdlf8bjWb2dyCdf8czz2N+H0CHi59NwrUv5HwAPG7WPQFQr7PAwvUnWJmYyTtQlhvPg843cx2KHLuXo91g6SCvrVvA/5IsLh7Mu4PBCZ3oJ2VgcEdqHcSIQHvWXGbTEzcm1Pv06mt4LlHESwKzycYztwOzIv9GdvB+zkSOKN0P1fwe/dU5vWJwE3x9VoEM/Wi7XwVeJngrvIWYeTz1QL1Cp8j0cZHM683LH+PC7ZxP2HqfBPCCPaGgvWeyLy+iDDaK+135DvwZUKC6Rcyn6+721F/O4Ky7Oi9XA34BUEBPk5Yj1+tYN1J8f+PCevodXl/e8vmBi+NyUgzO1jSOAAzW1DAym8JVRbZjzCzJ4vUN7PzJf2DMHUq4ItmNqlA1esJinJyqSvZZlm6hpjiSoLF5irAwwQF8inC9O1FBLPzXDrqbiHpJGC2mV1eVv41wo/gzyrXXMKizOuPEdxWMLP/FH0L4xTXhwnra9Nj2UbAzyUNNbMfJKqvqoyjeDlmVuQ9OI8wxQbBajbra3cmxd7HlW3pmuFP44iuCM2S+ljwqdwTODoj68jv2XGE9bqHAczs+TitW5TJwHFxFArBkvgSy1l7k3QQcKsF39pcn74qvCrp18BewE8k9ccDlxTGlV9jskjSQJaaOI8ks1hfgEuBk2zZRfZLCT+oVZH0V4Jf3E1m9hhLjS2K8hmCwhkD3AxcY+13Sh9sZpfG/nzFzP4Uy++U9NNEPWKdWt0t/pdlf+xLXEoYQeQpv3cVnMpfJTw8HBX71Ye0lWOWw4AtLTO1ZmbTJX2OMAJOKj+Cn2A1V4siiivla1f0IazcTSFr8YlVX7e6DvhHtPRdQHTRkbQxHTPzX2hmi0oPHvF9aM9a0MVAX+BXcf+wWPalnHqHAr+SdAfBv/TvZtZeH73PEdx1zjOzd+MD3SntbKPX4sqvMfkucAewfrS82xk4sh31VyopPgAzm1DQP+tSgpHKzyTdQ/jS3mZmi9LVlpznRuDGeK4DCI7hwwjWbkUNDLLuDHMSsmrU6m5hla7XzBYWHH0fQ5jmWoswVVxyM9gT+Gs7OrHcmlKcAci7By+Z2f8WPU+101d5XWm/GrNY1k0h67aQclk4kDDluzZBYZTO10RY+2sv/5B0OkH5fiy2/Zd21N/OljVOuSdaFCcxs09Fw7NPEUZ+l0sqPRDel669pI35ChFm9pG0D3C/ZYypnDSu/BoQM7szThPtSHhS/rqZlfu8pZgu6dssu8heyX2i/Lw3AzfHUef+BIfuSyTdRvjS3lnw/O8TntLnAB8ABrSj75tGgx0R/JtK/oUCirh6rENwtzhfUsndom87zo+kNa3MojG2lYuZPUd4Wi8v/xvwt4JdmClpTyszp48GKHn+ahUVtEJcz8+bWZFAARtJuiW2VXpdanvDAvWxGlwZzOyhCmXPdbC5bxFG31MJDya3ESIFFaVV0kgzewGWTD8XGsGZ2Rzgt8Bv40PgQcCFceo61/1Hy7sc/V5SrsuRE3BrzwYlrtvsQnhKfiCOqorWXQ34XqwPwULtLDN7twP9GEP4Ao8xs+acY/cgTDluD9wFXFsafbXjfDWb6mfaare7RTStPwH4Jsua+Z8LXGRlkTsq1K9HTMctCNPGDxCmno1geLEzcIAlHKQljS6t7UrairDm+TnCw88NRX44a3U3ybSzBmHNbYt4DU8T7mFVlw9JM1l2xFh+7qqyRJv9CHFajeArV2gmI9bdk7AOPZ2g/DcgrIHfm6y4bBurERTfOILBzZ/N7MQC9Wp2OerN+MivAZH0K4KJeMks/hhJexV8aqfSIruk86jsv1bp/GsSfjAPIUw//QlImvhH7ibEnnyA4BN2eFQmpX7l/vC3R7lVQhk/MTObGY19XiV8FwYXOP9Vsc7ZBB87I/jYfdfMbi/QhZpjOlpwkxhNUFxbEH507wOOqTQdWsYiSd8h/NC+RbAcVntGYinlpoKZIeJxfwDGEwyYRFhLfUTSoWb2zypVmwnvU81xPWM/Pk6IdPRCbHNDSccUfC8xs7uj20bJ0fzfBNeHvPOuTJjCHUe47lsIa7X3ZqZyc5th2VFmK3W6L70BH/k1IJKeAkaXviSSmoCpZrZFumayzZfN7AM5x5SMRTYlWPldm/iRqlQ/6USfN2qKbZQH1y5vIxlcWyvATyzT9nZm9mhH69eKQsaGQ8zs6sQxbQQjkaNKxkaSplv7ogPVIzPEQ8Cx5VbCcTT6/9s792BLquoOfwsQQ1GASikkYWBgCOhUAgpjKjBWDK+AAoE4wDAiSqCIVkBAfARTiZRYZXgkhECIRt4RwiMgD6V41ggUA0ICjGEYUEZmeBiiiMYAkxQCv/yx9pl7bnMe3X36dN++Z31VFKe7z969udzbq/d6/NY/qU+tmlXUyaFrvieAA7p+FvOAmyW9e/DIgXPm+Vv6Ge7mvgr/GRZWZjHPPP4ELnMIbkwvzZFxHBA7v7byAzxW1tkFzcF3VKOQ541xd+B04E71EFYeRh7jlmOOTQDM7DQ8SeKb+NqPwOsWh7G+plRAFuOybNcB15nZ8gHjemJm85lynf4Slxkb9P1vM9h4D5XVSokSx+HG50bchXwcnum3HOhr/PCM28OB76ZMw6sovluoojPEplnDByBpedoV9aPqnc1PMxnHT5FDmHoIeda4taS1o9xEXnJ0Fx6+KFJyFBA7v1ZiXmP3fvzhQ/p8P7AW+j9Azewd/abEi4e3ynn/G/GH5k3KqSySxm2G608ezJQA8k/xB/jpRWKOZvZAdnfQ61yPcSuA98rbvzwB/Gknu87MVihfT8FtcGO3BHgNj/MsUA6B8K54meEtqaalxOeJl6Wf/y/w/+d74YXSG+KJT7kMeIoPddxue+Jx2+vzZAumn+FInSHM7HFgd2V6SKbf0fv67bxSMkghCbMh6/ga/v/vGvyl5FD85XIZ5K57zM6ZZ+f3KINfgvrG7Qb8HXfGVvbzmc3Ezq+dfKnkuE5yRK8309xBfjzhYDFwupk9iMeNvpMj3nQNsBQvzu5oM26Jl2n8K170nZfXzewI3AgLf4jnybK7khHqxMyFiDdL9z1EXhS9Oo/hg+nGzcxezpsckmE7JSk5M7sQNz5bK6ckV1rHK/gO8Yr0MD0Uz3zMkypfRWeIvwNuN7PPMT1x6AwG1EqO4cH+a3gbos5LyQu4+MOBDKh7HLCDN2DzHPc9IP27E6fvZF4fQXqJHUD277izjk5/zdwu7Ekmdn4txMy2ZSpD7nEllY8G1rE+vms4FtgvR7ztB5J2LHqtz/fn4s1bF+I/h2V43dyaHGN/j6k6sU6m3A54Af1ApZG063ofnqDwL5LuKxoz65qrVPwqO67qOFiO+4/cGSLNcwAu09ed7XmWpCJ1do1QYcbrMkkLh50LqieMX4tIsZ4L8Tfk7+MPm53xN8Fj5HVDeebpuC1vLBt3MK/1OxDfAe6C7/wGFhmb2e14fOoypTq5lDl6FLCPpL3LrKVukvt2Eb7b3B54G7CvpAcHDuRNLqtp7ZQg387GvFt3x93c6X+3linjM/AlZFSqLDfpM/9JdSRtmLeG+iLejb1jfM/QgJZQY1jDcuB4Sfem492Bf5TUN2PUvKB9E0nXZs5/FHhB+ettJ5owfi3CzC4F1gCnddxOZmbAX+Exl4/3Hz1tng/iRmt/PG6Y123ZGX81rqF5K+7KvCtPAkyqZzoFV3fZAn/g/ATfRZ2R88H/BUlnWp96OeUol6iSZLwX40kkczSkONm8F2M/17PK7CBnCnmyTXPOMzRmNiopc/mT+M6zU36yAE/oulBJQm/A+NIxu8w8uwIX4650gP8Gjh7kgUiZsgdKeiFzfks8brtbnntPOmH8WoSZPSnpt4peGzBfIbdl17j9gDtUXIsQ875nW+H9A1/unlPSrTnGHyjp2/3KJqrIKC2LmW0z6q6nDWSyTW8C7gCOx+tEl0s6aMT5nx32EjEqZrYSb37888z5zXHRiPcMGd/Z/faM2Uk6reB6NsWfx3nizv/Rz7gOuhZMJ4xfizCzVZK273OtkPEr47bsGluqn6CZnYA/LB7H6+pOlEum1R63KotNSXn1JGepwgZ449NORuNK4DZ5p4IZTxXZpkPmr2Pn93g/AzfoWo/vjhSzM+/EsAiYS1cC4iDjaWY/BOZnf1/MGyOvLPoSPKlEtme7WGauzvEVdb21mOt0vknvsB8Zt+X55HRbdnEJHmfsdIF4Ds/WHNZM91hgV0kvp4SVa81srqS/p7cbsNfaRzY+I7Ib8CyeNfoAOdfdwcx+A4/1PQ88ksYfgGuN7iHpP6td7lgYOdvU+osVdGKY4+Z/zGxnSdNEqM1sZyD3fwewsZl9IBOzyyMS3+FGPMv4IfJ3ZvkWcIGZHa/p0mbnkq8rR0AYv7bxabzAeFUKlAvPPHyE1BonJ5fgIsaF3ZaJsv0E1++4OiWtMW+ldG1yIeU1IiMZnwrYEi/JWILLi92Mi3r31dPM8FXga9mEjrQr/mtcsWOms06NRNLrqdSjiMFYJ1bQIJ8FbjKzzotcRx/1E7jQe16OAS5OSVCQYnYFxm8l6U1C50P4S1wK7WlzgQFw0YuL8Ph/kINwe7YQcwmm+fiD/zElRfkC40u5LbvG34e7u5ZJ2iWt50pJvztk3FK8j+DyrnMb4AH/IzREGDt9f32mjM9OFDc+lZFcVkuAs/AkpDyi0E+ofwF3oXKPpqgi2zT9Dv5KSdbLzHYEPgysUQGR9lFICSJ/xpQ+6mO4sHbRmsVCMbvMuG8A50l6tMQ9N8KzjQFWSfrfonNMMmH8WkiKuVyNlyrkVljpGn81/rb7cUm/nf6I7h+UXp0Zvw/+9jkfL4peCBwl6a4h47YCXuv1cDGzhSqgE5rGFDY+VZDuu3+691w86eNiST/OMfYR9dG+HHRttmFm9+DlOU+aCww8iBfdzwf+TdIpjS4wJ2VidpnxK3EDthp3exaqlQzKE8avhVRQqvDvkhZ0P2zN7Pua3pSz31jDszXXMtVP8Hsq1k9wJEYxPhXc+zK8m8MtuLD3ioLjn6J39wwDzpQ0b/RV1o9NyaV9VNL+Ob7/aFfc8CvAOyQdZ95e6KHOtTGut6pShVuZitmtCyNI+tuc43vWTE5C1nDTRMyvhcjVI+7OlCpcDOQtbn417fY6XSHmkTPYLklmdoOkXSnQebwqMsbny0WNTwUcibv8dgBO6Ap15nX53Y1n2fYiVwfvmUIyVB/GY5/74Z0+vp5zeLfh2RPfvSPpVRvejb4KRpEX66ZMzK5b7KBQrDSojtj5tZQRSxVKuS27xp+Pt06pvX1PejB2XL3dv7y1qJsE635/lgD74pmrV+Nxq7kF5rgc78rxY1z4YFtJa83sbcDdebwQVVBBqUKpmJ2NIHZgZu+W9ISZ9SoNEvDz2DkOJ4xfC7GSCitp7MhuyxSn2BFXm3mFiFPkxrwH2y8lXZQ5/2k8G3bG92KzqZ6AR0lanc4V7Qm4EXAirrF6cafkIJUKzJP0zUHjq8JKyItlxtceszOzCyQda2b9usVvjndpOXJca5gNhPFrITaCwkoa/1ByW5a9f8QpSmLeDmgXSa9mzr8VT/SY8S8QZvY+XM7tELz/3VXAlyQN1Pwsea/rJC2qet6u+QvLi2XGj/S3YBXo7PaZ93ZJf1jVfLORMH4tpIJShVJuS/PebZ/C33QfBS5SS1RJZgrdiR5Frs1UzGwh7gJdhDfSvV5DdDELzl9LBmzRUgWrqKdemeQ1M/vIkHtHoXsOwvi1kApKFUq5LdN9f4W7vD4EPC3pxNL/IRNIyjLcW6mrRdf5LYA722b8OpjZenj95WJJRYq8h807Vtm7sqUKo8Ts+syXW2c3FeYDvAtXWVqajvfAQyADjWPgRLZnOymrsNLhQyXvO78rPf0ipjrJB/k5C7jZzD7L9CauZwJ/09iqCmBmH5N0efq8UNKyFHO+LXkh2kQZeTEkbVvVAnokrw0UZ5f0J2ncd/C/yefT8a/jcoVBDsL4tZNSpQoVuC27Za1eK2ZvAwBJ/2xmLwCn4SUbwpVFTpV0S6OLy8/JwOXp83n4A7vD0cA/VHivcf+SlSpV6DBqzM5G09md2zF8iZ/gJThBDsL4tZNT8T+WOWZ2BalUIce4y5jutpyPZ9zlZWcz6zTMNWCjdBxlBgVIRu5Nhs7M3t9E+UgJrM/nXsej8ucVz5flPjP7naKlCl2cje/YTjezwoITjKaze5eZ3Ybr3AqPvfbLAA0yRMyvZYxSqpBR1dgAeHCc8ZRgOGY2H8+cXIKXQCxoeElD6Y7DZWNyeWN0ZnYQvus6Px0/ALwzXf6CMl3Kx0VVpQpFYnaZcaMmr/0x8Pvp8B7VpIs6Gwjj10LKliqUfVAF1ZLS45ekf14DtgEWSFrT5LryYmZrgVW4oZiXPpOOt5M0tKWPmS3Du74/m46X42LpGwOXSNprHGvvsY6Ry3ZGFJwYKXktM9cHgCWSjhv65SDcni3leyVdZOG2bBjzjhib4XGiQ+TCzqvbYvgSuRq9DmHDjuFL3CvpReBFc53QsVKVvNiIMTsYMXnNzN6Lv0QtxnevUeaQkzB+7WQP4FNmtoYCpQrK0TIoGDsv4G7rLXA335MMEFieiVQkZvD2zJzHdx2+k/HT6eHXs1QByFuqMGpvzMLJa2a2A1Ou8hfxOKNJ2qPkGiaScHu2kFBYaTfmjU8X4Q+v7YG3AftKakXpSFeN27pTXcdSjs4UKVHrLkkXZM5/EvgDSUuqWu84qSBmV1hnt0te7hhJq9K5QvJyQRi/VhEKK7OPVNy+GH+TnyNpTsNLGoqZbZ45tR5wGN6q6eE8cmRm9i7gBnyX013v+Fbg4KwIwLioqFShVMyubPJaSnI5HC9wvzWt/8Iqaw8ngTB+LSIUVmY3ZrZNm3bvSdXlSODzuLTZVyWtLDjHnngndYDHJC0d9P2qKSMvlhlfujdm+m5pnV2b6qG4BM80vQyXl7u9zHyTRsT82kUorLQcM7tpyFf+qJaFjICZvQUvZv8McC9wkKQfFZxjT0lLJS1NCT+ru659RDXpU6rB3piJsslrSHoFuAK4IiXwHIq3hwrjl4PY+bWIKFVoP0nd5Vm8MPkBMgkX6WE8ozGz5/ASjXOAZ7LX8xiuKmoFq2LEUoVRe2NGe7CGCOPXIszsdaYauRqwER4viFKFlpB2GJ1msDsBNwNXSnqs0YUVwMwupX+GqpRD2DrjJpzWuSF7PE6s+d6YkbzWEGH8gqAhzDsKLMHFrk+TdF7DS6qNmbLzs4Z6Y0byWvNEzC8IaiYZvf1xwzcXOJcWFSebd6Pvi6Szc0yzXYp/Wtdn0nGdWYv3AF80s1KlCpSP2Y2qsxuMSOz8gqBGzOwyvJvDLcBVklY0vKTCmNmpAy5LQ3rhpTk+OOh6XbHPUeXFysbsQme3eWLnFwT1ciT+kNwBOKFLyao1cVtJX+53zcxOyjlHX+Nm3h2+LprqjRntwRomjF8Q1Iik9Zpew5g5Gc8CHUhK/DkM+E3gVkkrzOwA4C/wRK5aEl5orjdm6Ow2TBi/IAiqJO8W5iJgDl6req6ZPQ3sBpwi6YZxLa4HjfTGDJ3d5omYXxAElWFmz0jaOsf3VgA7SXoj7aJ+Bmwv6b/GvsipNURvzAkmdn5BEBTCzF6id51fp/Y0D6926ukk/Z+Z/bBOw5fuKzO7IZUq3FxweMTsWk7s/IIgqB2baogL05vi1qpwYmbnA5cWLVUIwYn2E8YvCILa6ads0qEuhZOQF5tcwu0ZBEHt9DNuKQv0cKAuea+ypQpBy4mdXxAEtWNmmwLH4aUONwF3AMfjPQGXSzpozPcPebEJJ4xfEAS1k5rI/gK4H9gLeDuwIXCipOU13D96Y044YfyCIKidTKnA+nipw9aSXmrg/lGqMIHMdrWJIAhmJt2lAq8Dq+syfD3uH+7OCSR2fkEQ1E7TpQJN3z9onjB+QRAEwcQRbs8gCGYEZraxmR1hZkXVVoKgMGH8giBoDDPb0MwONrNrgOeBvYGvN7ysYAIIt2cQBLVjZvvgnez3Bb4LXA2cJ2luk+sKJocwfkEQ1I6ZvYHX2B0laXU695Sk7ZpdWTAphLxZEARNsCsuY3anmT0FXAVEj7ugNmLnFwRBo5jZQtwFughYDlwv6RvNriqY7YTxC4JgRmBm6wH7AIslHd30eoLZTWR7BkFQO2b2sa7PCwEkvSHpNuDhxhYWTAyx8wuCoHbM7OGOlmb3517HQTAOYucXBEETWJ/PvY6DoHLC+AVB0ATq87nXcRBUTrg9gyCoHTNbC6zCd3nz0mfS8XaSNm5qbcFkEHV+QRA0wXuaXkAw2cTOLwiCIJg4YucXBEHtmNlqpsf2rOtYkubVv6pgkgjjFwRBEyzIHK8HHAZ8Dnik/uUEk0YYvyAIakfSi7BO1eVI4PO4tNn+klY2ubZgMgjjFwRB7ZjZW4Cjgc8A9wIHSfpRs6sKJolIeAmCoHbM7DngNeAc4JnsdUnfqn1RwUQRxi8Igtoxs0vpX8yuELYOxk0YvyAIgmDiiJhfEAS1Y2YnD7ou6ey61hJMJmH8giBogk0GXAt3VDB2wu0ZBMGMwsxOknRO0+sIZjdh/IIgmFGY2TOStm56HcHsJloaBUEw04h+fsHYCeMXBMFMI9xRwdiJhJcgCGrHzF6it5EzYKOalxNMIBHzC4IgCCaOcHsGQRAEE0cYvyAIgmDiCOMXBEEQTBxh/IIgCIKJI4xfEARBMHH8P0EEU3Rc3i3PAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1eb87ccae80>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1eb87dd7b00>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# It's good to know what these values are. You can see if there's\n",
    "# a huge outlier from human error that needs to be dealt with\n",
    "# in your dataset, etc.\n",
    "\n",
    "# Finding correlation between attributes and plotting heatmaps,\n",
    "# then saving them in our main directory for later if wanted.\n",
    "print('[INFO] Printing the Pearson Correlation tables...')\n",
    "print(\"PEARSON CORRELATION\")\n",
    "print(df.corr(method=\"pearson\"))\n",
    "sns.heatmap(df.corr(method=\"pearson\"))\n",
    "\n",
    "# Create the plots directory\n",
    "if not os.path.exists(\"./plots/\"):\n",
    "    os.makedirs(\"./plots/\")\n",
    "\n",
    "plt.savefig(\"./plots/heatmap_pearson.png\")\n",
    "plt.show() # comment this out if using .py file\n",
    "plt.clf()\n",
    "#plt.close() # uncomment this if using .py file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Generate a report with lots of statistical information and export to .txt file\n",
    "# for later reference if needed.\n",
    "\n",
    "# create directory to store stats summaries in\n",
    "if not os.path.exists(\"./statistical summaries/\"):\n",
    "    os.makedirs(\"./statistical summaries/\")\n",
    "\n",
    "file_report = 'statistical summaries/' + ticker_input + \".txt\"\n",
    "with open(file_report, \"w\") as f:\n",
    "    f.write(\"Features shape : {}\".format(df.drop(\"Adj Close\", axis=1).shape))\n",
    "    f.write(\"\\n\")\n",
    "\n",
    "    f.write(\"Target shape   : {}\".format(df[\"Adj Close\"].shape))\n",
    "    f.write(\"\\n\")\n",
    "\n",
    "    f.write(\"\\nColumn names\")\n",
    "    f.write(\"\\n\")\n",
    "    f.write(str(df.columns))\n",
    "    f.write(\"\\n\")\n",
    "\n",
    "    f.write(\"\\nStatistical summary\")\n",
    "    f.write(\"\\n\")\n",
    "    f.write(str(df.describe(include='all')))\n",
    "    f.write(\"\\n\")\n",
    "\n",
    "    f.write(\"\\nDatatypes\")\n",
    "    f.write(\"\\n\")\n",
    "    f.write(str(df.dtypes))\n",
    "    f.write(\"\\n\")\n",
    "\n",
    "    f.write(\"\\nPEARSON correlation\")\n",
    "    f.write(\"\\n\")\n",
    "    f.write(str(df.corr(method=\"pearson\")))\n",
    "    f.write(\"\\n\")\n",
    "\n",
    "    f.write(\"\\nMissing Values\")\n",
    "    f.write(\"\\n\")\n",
    "    f.write(str(pd.isnull(df).any()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "From here, we can see which features are highly correlated to\n",
    "each other. This is important because if some features are \n",
    "almost identically correlated to eachother, we won't need both\n",
    "of them in our datasets. We will deal with this more during \n",
    "the feature selection phase, but this gives a nice visual\n",
    "of how your feautres correlate to each other. The images created\n",
    "are saved in the project's main directory for your review.\n",
    "\n",
    "Let's create some more visualizations. You can review how to \n",
    "read these plots online and at the links given below :)\n",
    "Then save those plot images too. We won't show them here,\n",
    "just see the saved images in the working directory."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Done visualizing!\n"
     ]
    }
   ],
   "source": [
    "# Univariates\n",
    "sns.set(color_codes=True)\n",
    "colors = [\"y\", \"b\", \"g\", \"r\"]\n",
    "\n",
    "cols = list(df.columns.values)\n",
    "if not os.path.exists(\"./plots/univariate/box\"):\n",
    "    os.makedirs(\"./plots/univariate/box\")\n",
    "\n",
    "# Box plots\n",
    "# https://www.khanacademy.org/math/ap-statistics/summarizing-quantitative-data-ap/stats-box-whisker-plots/v/reading-box-and-whisker-plots\n",
    "for i, col in enumerate(cols):\n",
    "    sns.boxplot(df[col], color=random.choice(colors), orient=\"v\")\n",
    "    plt.savefig(\"./plots/univariate/box/box_\" + str(i) + \".png\")\n",
    "    plt.clf()\n",
    "    plt.close()\n",
    "\n",
    "if not os.path.exists(\"plots/univariate/density\"):\n",
    "    os.makedirs(\"./plots/univariate/density\")\n",
    "\n",
    "# Kernel Density plots\n",
    "# https://en.wikipedia.org/wiki/Kernel_density_estimation\n",
    "for i, col in enumerate(cols):\n",
    "    sns.distplot(df[col], color=random.choice(colors))\n",
    "    plt.savefig(\"./plots/univariate/density/density_\" + str(i) + \".png\")\n",
    "    plt.clf()\n",
    "    plt.close()\n",
    "# Commented out as I get \"LinAlgError: singular matrix\" error after I added in \n",
    "# all those technical indicators.\n",
    "\n",
    "# Bivariates\n",
    "if not os.path.exists(\"./plots/multivariate\"):\n",
    "    os.makedirs(\"./plots/multivariate\")\n",
    "\n",
    "# Scatter plots\n",
    "for i, col in enumerate(cols):\n",
    "    try:\n",
    "        if (i == len(cols) - 1):\n",
    "            pass\n",
    "        else: \n",
    "            sns.jointplot(x=col, y=\"Adj Close\", data=df);\n",
    "            plt.savefig(\"./plots/multivariate/target_vs_\" + str(i) + \".png\")\n",
    "            plt.clf()\n",
    "            plt.close()\n",
    "    except:\n",
    "        pass\n",
    "\n",
    "\n",
    "# Pairplot\n",
    "#sns.pairplot(df) \n",
    "#plt.savefig(\"plots/pairplot.png\")\n",
    "#plt.clf()\n",
    "#plt.close()\n",
    "# I've commented this out because trying to create a pairplot\n",
    "# image of a large dataset can take a while and make a big image file, so feel\n",
    "# free to add this back in if you want but expect it to be slow.\n",
    "\n",
    "# An alternative would be...\n",
    "#sns.pairplot(df.sample(1000)) \n",
    "#plt.savefig(\"plots/pairplot.png\")\n",
    "#plt.clf()\n",
    "#plt.close()\n",
    "\n",
    "print('Done visualizing!')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 5. AUTOMATED FEATURE ENGINEERING! WOOO!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will be using the Featuretools module for this step of the process.\n",
    "Feature Engineering, in a very basic sense, is the creation of new\n",
    "features, using the current features of your dataset. To do this\n",
    "part manually, data scientists would have to have some domain knowledge\n",
    "of the field they're working with. If you had a dataset of sales numbers\n",
    "and geographical locations, adding another column or feature of say, the ratio\n",
    "of sales per geographical location would show the relationshiop between sales and\n",
    "the location, giving our model more data to work with. Automatic Feature \n",
    "Engineering takes care of this step for us by creating lots of new features\n",
    "using the entire dataset to bring to light hidden relationships between the\n",
    "features, effectively mitigating the need to know \"everything\" about the \n",
    "field you're trying to figure out. More feature engineering can be found here:\n",
    "https://www.analyticsvidhya.com/blog/2018/08/guide-automated-feature-engineering-featuretools-python/\n",
    "\n",
    "In our example, if we have the 9 MA and 20 MA values, using Deep Feature Synthesis,\n",
    "it would create added columns like \"9MA > 20MA\", \"9MA < 20MA\", \"Adj Close > 9MA\",\n",
    "the list can go on and on, as we'll see in a minute."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "        Date  PrevOpen  PrevHigh  PrevLow  PrevAdjClose   PrevVol   9MA  20MA  \\\n",
      "0 2012-01-06     14.30     14.99    14.19         14.50 55,400.00 14.17 13.35   \n",
      "1 2012-01-09     15.00     15.00    14.32         14.52 97,500.00 14.23 13.46   \n",
      "2 2012-01-10     14.61     14.61    14.36         14.50 75,200.00 14.30 13.60   \n",
      "3 2012-01-11     14.51     14.98    14.32         14.81 65,500.00 14.38 13.68   \n",
      "4 2012-01-12     14.76     14.76    14.20         14.48 22,500.00 14.48 13.74   \n",
      "\n",
      "   UpperBB  MidBB    ...      STDDEV   TSF  VAR  PrevIndOpen  PrevIndHigh  \\\n",
      "0    15.03  13.24    ...        0.19 14.78 0.04        34.75        35.69   \n",
      "1    15.17  13.35    ...        0.24 14.86 0.06        35.53        35.87   \n",
      "2    15.20  13.49    ...        0.25 14.85 0.06        36.14        36.30   \n",
      "3    15.26  13.64    ...        0.25 14.77 0.06        36.64        36.97   \n",
      "4    15.34  13.71    ...        0.12 14.66 0.02        36.90        37.20   \n",
      "\n",
      "   PrevIndLow  PrevIndClose   PrevIndVol  Open  Adj Close  \n",
      "0       34.75         34.94   619,200.00 15.00      14.52  \n",
      "1       35.49         35.22 1,488,600.00 14.61      14.50  \n",
      "2       35.95         35.71 2,531,400.00 14.51      14.81  \n",
      "3       36.55         36.41 3,159,000.00 14.76      14.48  \n",
      "4       36.67         36.64 2,859,000.00 14.47      14.50  \n",
      "\n",
      "[5 rows x 34 columns]\n"
     ]
    }
   ],
   "source": [
    "# \"Reset\" the index column for Featuretools to use now \"Date\" column as the index, then we'll revert\n",
    "# the Date column back to an index after feature engineering is done (creating a new, bigger dataset).\n",
    "# But first, save the date column for later re-instating.\n",
    "\n",
    "# These if statements are a way of preventing over additions of indexes when re-running the \n",
    "# notebook. Just ignore it and move on.\n",
    "if \"level_0\" in df:\n",
    "    df = df.drop('level_0',axis=1)\n",
    "if \"index\" in df:\n",
    "    df = df.drop('index',axis=1)\n",
    "if 'Date' in df:\n",
    "    date_values = df['Date']\n",
    "df = df.reset_index()\n",
    "print(df.head())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\windowshopr\\AppData\\Local\\Programs\\Python\\Python36\\lib\\importlib\\_bootstrap.py:219: ImportWarning: can't resolve package from __spec__ or __package__, falling back on __name__ and __path__\n",
      "  return f(*args, **kwds)\n",
      "C:\\Users\\windowshopr\\AppData\\Local\\Programs\\Python\\Python36\\lib\\importlib\\_bootstrap.py:219: ImportWarning: can't resolve package from __spec__ or __package__, falling back on __name__ and __path__\n",
      "  return f(*args, **kwds)\n",
      "C:\\Users\\windowshopr\\AppData\\Local\\Programs\\Python\\Python36\\lib\\importlib\\_bootstrap.py:219: ImportWarning: can't resolve package from __spec__ or __package__, falling back on __name__ and __path__\n",
      "  return f(*args, **kwds)\n",
      "C:\\Users\\windowshopr\\AppData\\Local\\Programs\\Python\\Python36\\lib\\importlib\\_bootstrap.py:219: ImportWarning: can't resolve package from __spec__ or __package__, falling back on __name__ and __path__\n",
      "  return f(*args, **kwds)\n",
      "C:\\Users\\windowshopr\\AppData\\Local\\Programs\\Python\\Python36\\lib\\importlib\\_bootstrap.py:219: ImportWarning: can't resolve package from __spec__ or __package__, falling back on __name__ and __path__\n",
      "  return f(*args, **kwds)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Adding the engineered features to the dataframe. This may take a hot minute...\n",
      "Built 2016 features\n",
      "Elapsed: 03:47 | Remaining: 00:00 | Progress: 100%|████████████████████████████████████████| Calculated: 101/101 chunks\n"
     ]
    }
   ],
   "source": [
    "# Let's do it! Auto Feature Engineering using Feature Tools\n",
    "import featuretools as ft\n",
    "#print(ft.list_primitives().to_string()) # To get full list of primitives that could be used\n",
    "\n",
    "print('Adding the engineered features to the dataframe. This may take a hot minute...')\n",
    "\n",
    "# Define the function. We do this in case there's a runtime error, it won't\n",
    "# start the program over again, just stays within the function\n",
    "def feature_engineering_dataset(df):\n",
    "    es = ft.EntitySet(id = 'stockdata')\n",
    "\n",
    "# Now we make sure that the dataframe we feed to the featuretools doesn't include\n",
    "# our label. Notice the drop? Because we don't want to engineer features using the\n",
    "# label that hasn't happened yet, right?\n",
    "    adj_close = list(df['Adj Close'])\n",
    "    dataframe = df.drop('Adj Close',axis=1)\n",
    "    es.entity_from_dataframe(entity_id = 'data', dataframe = dataframe, index = 'Date')\n",
    "\n",
    "# Pesky warnings\n",
    "    warnings.filterwarnings(\"ignore\", category=RuntimeWarning) \n",
    "    warnings.filterwarnings(\"once\", category=ImportWarning)\n",
    "\n",
    "# Run deep feature synthesis with transformation primitives\n",
    "# Depending on how many features are being added to the dataset,\n",
    "# you may need to tweak the chunk_size and n_jobs variables below.\n",
    "# n_jobs should be tweaked for how many processors are on your\n",
    "# machine, and the chunk_size should be big enough to run quickly,\n",
    "# but small enough that each chunk can fit into memory. See the \n",
    "# Featuretools website for more information on this.\n",
    "    feature_matrix, feature_defs = ft.dfs(n_jobs=1,entityset = es, target_entity = 'data', \n",
    "                                           chunk_size=0.01,max_depth=2,verbose=True,\n",
    "                    #agg_primitives = ['avg_time_between',\n",
    "                     #   'trend','std',],    # review \"print(ft.list_primitives().to_string())\" for more\n",
    "                    trans_primitives = [\n",
    "                        'less_than_equal_to',\n",
    "                        'greater_than_equal_to'\n",
    "                    ])\n",
    "    warnings.filterwarnings(\"once\", category=RuntimeWarning) \n",
    "\n",
    "# Now re-add the Adj Close column because featuretools...\n",
    "    feature_matrix['Adj Close'] = adj_close\n",
    "\n",
    "# Return our new dataset!\n",
    "    return(feature_matrix)\n",
    "\n",
    "# Now run that defined function\n",
    "df = feature_engineering_dataset(df)\n",
    "\n",
    "# Now save a new .csv file with our new features in it for review later if we want\n",
    "if not os.path.exists(\"./historical data/\"):\n",
    "    os.makedirs(\"./historical data/\")\n",
    "dataset.to_csv('./historical data/' + ticker_input + '_with_feature_matrix.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "            PrevOpen  PrevHigh  PrevLow  PrevAdjClose   PrevVol   9MA  20MA  \\\n",
      "Date                                                                          \n",
      "2012-01-06     14.30     14.99    14.19         14.50 55,400.00 14.17 13.35   \n",
      "2012-01-09     15.00     15.00    14.32         14.52 97,500.00 14.23 13.46   \n",
      "2012-01-10     14.61     14.61    14.36         14.50 75,200.00 14.30 13.60   \n",
      "2012-01-11     14.51     14.98    14.32         14.81 65,500.00 14.38 13.68   \n",
      "2012-01-12     14.76     14.76    14.20         14.48 22,500.00 14.48 13.74   \n",
      "\n",
      "            UpperBB  MidBB  LowerBB    ...      LINEARREG >= PrevIndVol  \\\n",
      "Date                                   ...                                \n",
      "2012-01-06    15.03  13.24    11.45    ...                        False   \n",
      "2012-01-09    15.17  13.35    11.53    ...                        False   \n",
      "2012-01-10    15.20  13.49    11.78    ...                        False   \n",
      "2012-01-11    15.26  13.64    12.03    ...                        False   \n",
      "2012-01-12    15.34  13.71    12.09    ...                        False   \n",
      "\n",
      "            PrevHigh >= 20MA  MACDHIST >= MACDSIGNAL  PrevVol >= PrevAdjClose  \\\n",
      "Date                                                                            \n",
      "2012-01-06              True                   False                     True   \n",
      "2012-01-09              True                   False                     True   \n",
      "2012-01-10              True                   False                     True   \n",
      "2012-01-11              True                   False                     True   \n",
      "2012-01-12              True                   False                     True   \n",
      "\n",
      "            PrevLow >= PrevIndOpen  VAR >= LowerBB  PrevLow >= MidBB  \\\n",
      "Date                                                                   \n",
      "2012-01-06                   False           False              True   \n",
      "2012-01-09                   False           False              True   \n",
      "2012-01-10                   False           False              True   \n",
      "2012-01-11                   False           False              True   \n",
      "2012-01-12                   False           False              True   \n",
      "\n",
      "            MidBB >= LINEARREG_ANGLE  BETA >= RSI  Adj Close  \n",
      "Date                                                          \n",
      "2012-01-06                      True        False      14.52  \n",
      "2012-01-09                      True        False      14.50  \n",
      "2012-01-10                      True        False      14.81  \n",
      "2012-01-11                      True        False      14.48  \n",
      "2012-01-12                      True        False      14.50  \n",
      "\n",
      "[5 rows x 2017 columns]\n",
      "Behold our new dataset! Huzzah! (Remember, theres not really 5 rows, we just printed the head.)\n"
     ]
    }
   ],
   "source": [
    "print(df.head())\n",
    "print('Behold our new dataset! Huzzah! (Remember, theres not really 5 rows, we just printed the head.)')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 6. AUTOMATED FEATURE SELECTION"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, as you'll see, we have way more columns in our dataset than when we \n",
    "first started. Surely we don't need ALL of these columns before training\n",
    "our model do we? Nope!\n",
    "\n",
    "So what we'll do now is utilize the Feature-Selector module to help us\n",
    "perform automated feature selection. What's going to happen here is,\n",
    "it's going to find features that have more than 60% missing values,\n",
    "have a correlation threshold of more than 98% (remember our correlation\n",
    "heatmap from before?), and maybe most importantly, get rid of any \n",
    "features that don't add up to a max feature importance of 99%! This uses\n",
    "lightgbm to train a quick model to determine which features are the least\n",
    "important, then get rid of them, leaving us with an awesome dataset!\n",
    "\n",
    "https://github.com/WillKoehrsen/feature-selector"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "            PrevOpen  PrevHigh  PrevLow  PrevAdjClose   PrevVol   9MA  20MA  \\\n",
      "Date                                                                          \n",
      "2012-01-06     14.30     14.99    14.19         14.50 55,400.00 14.17 13.35   \n",
      "2012-01-09     15.00     15.00    14.32         14.52 97,500.00 14.23 13.46   \n",
      "2012-01-10     14.61     14.61    14.36         14.50 75,200.00 14.30 13.60   \n",
      "2012-01-11     14.51     14.98    14.32         14.81 65,500.00 14.38 13.68   \n",
      "2012-01-12     14.76     14.76    14.20         14.48 22,500.00 14.48 13.74   \n",
      "\n",
      "            UpperBB  MidBB  LowerBB    ...      LINEARREG >= PrevIndVol  \\\n",
      "Date                                   ...                                \n",
      "2012-01-06    15.03  13.24    11.45    ...                        False   \n",
      "2012-01-09    15.17  13.35    11.53    ...                        False   \n",
      "2012-01-10    15.20  13.49    11.78    ...                        False   \n",
      "2012-01-11    15.26  13.64    12.03    ...                        False   \n",
      "2012-01-12    15.34  13.71    12.09    ...                        False   \n",
      "\n",
      "            PrevHigh >= 20MA  MACDHIST >= MACDSIGNAL  PrevVol >= PrevAdjClose  \\\n",
      "Date                                                                            \n",
      "2012-01-06              True                   False                     True   \n",
      "2012-01-09              True                   False                     True   \n",
      "2012-01-10              True                   False                     True   \n",
      "2012-01-11              True                   False                     True   \n",
      "2012-01-12              True                   False                     True   \n",
      "\n",
      "            PrevLow >= PrevIndOpen  VAR >= LowerBB  PrevLow >= MidBB  \\\n",
      "Date                                                                   \n",
      "2012-01-06                   False           False              True   \n",
      "2012-01-09                   False           False              True   \n",
      "2012-01-10                   False           False              True   \n",
      "2012-01-11                   False           False              True   \n",
      "2012-01-12                   False           False              True   \n",
      "\n",
      "            MidBB >= LINEARREG_ANGLE  BETA >= RSI  Adj Close  \n",
      "Date                                                          \n",
      "2012-01-06                      True        False      14.52  \n",
      "2012-01-09                      True        False      14.50  \n",
      "2012-01-10                      True        False      14.81  \n",
      "2012-01-11                      True        False      14.48  \n",
      "2012-01-12                      True        False      14.50  \n",
      "\n",
      "[5 rows x 2017 columns]\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\windowshopr\\AppData\\Local\\Programs\\Python\\Python36\\lib\\site-packages\\lightgbm\\__init__.py:27: ResourceWarning: unclosed file <_io.TextIOWrapper name='C:\\\\Users\\\\windowshopr\\\\AppData\\\\Local\\\\Programs\\\\Python\\\\Python36\\\\lib\\\\site-packages\\\\lightgbm\\\\VERSION.txt' mode='r' encoding='cp1252'>\n",
      "  __version__ = open(os.path.join(dir_path, 'VERSION.txt')).read().strip()\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0 features with greater than 0.60 missing values.\n",
      "\n",
      "842 features with a single unique value.\n",
      "\n",
      "932 features with a correlation magnitude greater than 0.98.\n",
      "\n",
      "Training Gradient Boosting Model\n",
      "\n",
      "Training until validation scores don't improve for 100 rounds.\n",
      "Early stopping, best iteration is:\n",
      "[175]\tvalid_0's l2: 2.76672\n",
      "Training until validation scores don't improve for 100 rounds.\n",
      "Early stopping, best iteration is:\n",
      "[157]\tvalid_0's l2: 3.69196\n",
      "Training until validation scores don't improve for 100 rounds.\n",
      "Early stopping, best iteration is:\n",
      "[239]\tvalid_0's l2: 3.26372\n",
      "Training until validation scores don't improve for 100 rounds.\n",
      "Early stopping, best iteration is:\n",
      "[145]\tvalid_0's l2: 2.2825\n",
      "Training until validation scores don't improve for 100 rounds.\n",
      "Early stopping, best iteration is:\n",
      "[98]\tvalid_0's l2: 3.11058\n",
      "Training until validation scores don't improve for 100 rounds.\n",
      "Early stopping, best iteration is:\n",
      "[112]\tvalid_0's l2: 3.21604\n",
      "Training until validation scores don't improve for 100 rounds.\n",
      "Early stopping, best iteration is:\n",
      "[123]\tvalid_0's l2: 2.66594\n",
      "Training until validation scores don't improve for 100 rounds.\n",
      "Early stopping, best iteration is:\n",
      "[97]\tvalid_0's l2: 3.24964\n",
      "Training until validation scores don't improve for 100 rounds.\n",
      "Early stopping, best iteration is:\n",
      "[90]\tvalid_0's l2: 3.12764\n",
      "Training until validation scores don't improve for 100 rounds.\n",
      "Early stopping, best iteration is:\n",
      "[89]\tvalid_0's l2: 2.99976\n",
      "\n",
      "1820 features with zero importance after one-hot encoding.\n",
      "\n",
      "107 features required for cumulative importance of 0.99 after one hot encoding.\n",
      "1909 features do not contribute to cumulative importance of 0.99.\n",
      "\n",
      "1925 total features out of 2016 identified for removal after one-hot encoding.\n",
      "\n",
      "Total of 1925 features identified for removal\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['MOM <= MidBB', 'PrevLow >= PrevIndLow', 'PrevHigh <= PrevAdjClose', 'STDDEV >= LINEARREG', 'RSI >= PrevIndHigh', 'ADX <= Open', 'PrevIndClose >= TSF', '20MA <= LINEARREG', 'Open <= STDDEV', 'PrevLow >= UpperBB', 'LINEARREG <= PrevIndOpen', 'STDDEV <= LINEARREG', 'PrevLow >= ROC', 'ATR >= RSI', 'SAR >= PrevLow', 'MACDHIST <= PrevIndOpen', 'PrevIndHigh >= PrevIndLow', 'ADX <= PrevIndOpen', 'MOM >= STDDEV', 'MOM >= PrevVol', 'PrevIndLow >= PrevLow', 'PrevIndHigh', 'UpperBB <= STDDEV', 'LINEARREG_SLOPE >= LINEARREG_ANGLE', 'PrevVol <= TSF', 'PrevIndOpen >= PrevIndVol', 'PrevIndLow >= Open', 'Open >= RSI', 'LINEARREG >= PrevIndVol', 'RSI >= PrevLow', 'VAR >= PrevIndHigh', 'MACD >= LINEARREG_SLOPE', 'MidBB >= PrevIndClose', 'PrevLow <= ATR', 'MACD >= PrevIndVol', 'MACDSIGNAL >= VAR', 'PrevIndVol <= LINEARREG', '9MA <= STDDEV', 'MidBB <= BETA', 'PrevIndVol >= PrevIndLow', 'PrevIndOpen <= RSI', 'VAR >= PrevLow', 'PrevHigh <= ATR', 'MidBB <= PrevIndClose', 'UpperBB >= ATR', 'Open <= PrevIndLow', 'PrevIndClose >= PrevIndOpen', 'ROC >= MACDHIST', 'PrevIndOpen >= PrevOpen', 'STDDEV <= 20MA', 'UpperBB >= MidBB', 'PrevIndLow >= UpperBB', 'Open >= TSF', 'LowerBB >= PrevVol', 'PrevVol >= MACDSIGNAL', 'MACDSIGNAL <= PrevOpen', 'MACDHIST >= PrevIndHigh', 'VAR >= PrevIndClose', 'UpperBB <= LowerBB', 'LowerBB <= RSI', 'VAR >= TSF', 'PrevIndOpen <= LINEARREG_ANGLE', 'PrevVol <= PrevIndClose', 'ADX >= BETA', 'MACDSIGNAL >= PrevLow', 'ATR >= VAR', 'PrevIndClose <= PrevIndVol', 'PrevIndClose >= MACDHIST', 'PrevVol <= RSI', 'MidBB >= PrevVol', 'PrevAdjClose >= MACDSIGNAL', 'BETA >= PrevHigh', 'UpperBB >= MACDSIGNAL', 'PrevIndClose <= STDDEV', 'MACDHIST <= MidBB', 'PrevIndClose <= 9MA', 'PrevOpen >= PrevLow', 'PrevIndVol >= UpperBB', 'PrevIndVol <= SAR', 'PrevLow <= PrevVol', 'STDDEV >= PrevLow', 'PrevIndClose >= PrevVol', 'PrevLow <= UpperBB', 'PrevIndHigh >= PrevHigh', 'PrevIndOpen >= Open', 'TSF <= UpperBB', 'LINEARREG_ANGLE >= 9MA', 'UpperBB <= ATR', 'Open >= UpperBB', '9MA >= PrevIndOpen', 'PrevIndLow >= STDDEV', 'VAR <= LowerBB', 'MidBB >= TSF', 'MOM <= PrevIndHigh', 'MACDSIGNAL >= 9MA', 'BETA <= MACDHIST', 'RSI <= LINEARREG_ANGLE', 'SAR >= TSF', 'ROC >= BETA', 'STDDEV >= PrevIndLow', 'PrevIndHigh >= ROC', 'MACD >= PrevLow', 'RSI >= LINEARREG_SLOPE', 'Open >= MACD', 'ADX >= MACDHIST', 'PrevIndVol <= PrevAdjClose', 'PrevIndLow >= PrevIndHigh', 'PrevOpen <= VAR', 'PrevHigh <= MACDHIST', 'PrevIndHigh >= PrevIndVol', 'MOM >= MACD', 'MOM <= 20MA', 'MACD >= 20MA', 'PrevOpen >= RSI', 'PrevLow <= VAR', 'LINEARREG <= PrevIndLow', 'TSF <= SAR', 'PrevHigh <= TSF', 'PrevHigh >= ADX', 'PrevIndVol <= MACDSIGNAL', 'BETA >= PrevOpen', 'PrevIndClose <= MOM', 'PrevVol <= ADX', 'PrevAdjClose <= PrevVol', 'ROC <= PrevOpen', 'RSI <= LINEARREG', 'ROC >= PrevLow', 'LowerBB <= PrevIndVol', '9MA <= MidBB', 'LowerBB >= MACDSIGNAL', 'ADX >= TSF', 'BETA <= PrevLow', 'PrevAdjClose <= MACD', 'PrevIndVol <= PrevVol', 'STDDEV <= RSI', 'PrevIndHigh >= SAR', 'RSI <= MACDHIST', 'MidBB <= TSF', '20MA >= 9MA', 'ATR >= TSF', 'LowerBB >= 9MA', '20MA <= MACDSIGNAL', 'LowerBB >= PrevIndHigh', 'PrevVol >= PrevIndVol', 'BETA <= UpperBB', 'PrevIndVol <= STDDEV', 'PrevOpen <= RSI', 'LINEARREG_ANGLE <= MACDSIGNAL', 'UpperBB >= MACDHIST', 'Open <= PrevIndHigh', 'MOM >= PrevHigh', 'PrevIndClose <= ADX', 'PrevIndOpen >= TSF', 'PrevHigh >= ATR', 'LowerBB >= PrevIndOpen', 'LINEARREG_SLOPE >= MOM', 'MOM >= PrevLow', 'PrevIndLow <= 9MA', 'ADX <= RSI', 'PrevIndHigh <= PrevIndLow', 'ADX >= LINEARREG', 'PrevAdjClose >= LINEARREG', 'PrevAdjClose <= PrevIndHigh', 'ADX <= MACDHIST', 'LowerBB <= UpperBB', 'Open >= MACDSIGNAL', 'ROC <= VAR', 'MACDHIST >= MidBB', 'ATR >= SAR', 'PrevIndLow <= PrevVol', 'MACD <= RSI', 'UpperBB >= PrevVol', '20MA >= MACDSIGNAL', 'MACD <= Open', 'MACD >= PrevIndClose', 'PrevOpen <= PrevIndOpen', 'TSF <= MACDHIST', 'PrevIndClose <= VAR', 'PrevIndOpen >= PrevIndClose', 'MOM >= LINEARREG_SLOPE', 'PrevIndLow >= LINEARREG', 'ROC >= LINEARREG_SLOPE', 'VAR >= SAR', 'PrevIndLow >= MACD', 'BETA <= ADX', 'PrevIndHigh <= SAR', 'PrevIndLow >= PrevOpen', 'MACDHIST >= ATR', 'MACDHIST >= LINEARREG_SLOPE', 'PrevIndHigh >= STDDEV', 'RSI >= ADX', 'VAR >= PrevVol', 'ADX <= PrevIndHigh', 'MidBB <= MACD', 'MACD >= 9MA', 'PrevAdjClose >= PrevIndHigh', 'MACD <= PrevAdjClose', 'PrevIndClose <= PrevVol', 'BETA <= MOM', 'PrevHigh >= LINEARREG', 'RSI <= ADX', 'MACDSIGNAL <= PrevIndOpen', 'PrevIndClose', 'PrevLow <= LINEARREG_SLOPE', '9MA >= BETA', 'PrevIndLow >= ADX', 'ADX >= MACD', 'PrevIndLow <= VAR', 'MACDSIGNAL <= STDDEV', 'PrevIndLow <= STDDEV', 'PrevHigh <= SAR', 'PrevLow <= PrevIndClose', 'PrevIndVol <= LowerBB', 'ADX >= ATR', 'PrevAdjClose >= ROC', 'ATR <= Open', 'BETA <= 20MA', 'MidBB >= LowerBB', 'VAR <= STDDEV', 'PrevIndOpen >= PrevHigh', 'SAR <= PrevAdjClose', '20MA >= LINEARREG', 'MOM <= PrevIndClose', 'ROC >= 9MA', 'PrevOpen >= BETA', '9MA <= PrevIndHigh', 'MACDHIST <= LINEARREG_ANGLE', '9MA <= UpperBB', 'SAR >= PrevOpen', 'PrevIndClose >= Open', 'ADX <= VAR', 'ATR >= Open', 'LINEARREG >= PrevHigh', 'TSF <= LowerBB', '9MA >= PrevAdjClose', '20MA >= TSF', 'ADX >= MACDSIGNAL', 'ROC <= LowerBB', 'Open >= PrevOpen', 'MACDHIST >= Open', 'MACD >= MidBB', 'STDDEV >= PrevHigh', 'RSI >= SAR', 'MACDHIST >= LINEARREG_ANGLE', 'LINEARREG_SLOPE >= PrevIndOpen', 'MACDSIGNAL <= BETA', 'LINEARREG_SLOPE >= ADX', 'LowerBB >= PrevLow', 'ATR >= 9MA', 'MACDHIST <= RSI', 'PrevOpen <= BETA', 'LowerBB <= TSF', 'PrevAdjClose <= PrevIndVol', 'MACD >= VAR', 'MidBB >= STDDEV', 'LowerBB >= PrevOpen', 'Open <= PrevVol', 'LINEARREG_ANGLE >= 20MA', '20MA >= MidBB', 'MACD >= MACDSIGNAL', 'ROC >= PrevIndOpen', 'VAR >= PrevOpen', 'PrevOpen <= UpperBB', 'MACDHIST <= PrevIndHigh', 'PrevIndVol >= STDDEV', 'RSI >= MOM', 'PrevAdjClose >= MidBB', 'PrevLow >= PrevIndOpen', 'PrevIndOpen <= STDDEV', 'ATR <= PrevIndClose', 'LowerBB >= Open', 'PrevOpen >= PrevIndLow', 'TSF >= UpperBB', 'MACDHIST <= UpperBB', 'MACDHIST >= SAR', 'PrevIndHigh >= LINEARREG_ANGLE', 'MOM >= VAR', 'ATR <= PrevOpen', 'ADX >= STDDEV', 'ADX <= PrevOpen', 'LINEARREG_SLOPE >= 9MA', 'LINEARREG <= PrevVol', 'MACDHIST <= PrevAdjClose', 'TSF >= PrevVol', 'RSI <= PrevHigh', 'SAR >= STDDEV', 'SAR >= MidBB', 'LowerBB >= PrevIndLow', 'PrevIndHigh >= PrevVol', 'PrevIndLow >= PrevAdjClose', 'STDDEV <= MACDHIST', '20MA >= MACDHIST', 'Open >= VAR', 'PrevIndVol >= Open', 'ROC >= MOM', 'PrevIndVol <= UpperBB', 'LINEARREG >= PrevAdjClose', 'MOM <= PrevVol', 'PrevHigh <= PrevLow', 'MACDSIGNAL >= LowerBB', 'BETA >= TSF', 'LowerBB <= PrevOpen', 'MACDSIGNAL >= PrevVol', '20MA >= MOM', 'ADX <= PrevAdjClose', 'STDDEV <= BETA', 'MACD <= PrevVol', 'MidBB <= MOM', 'PrevHigh <= BETA', 'ROC <= MidBB', 'LINEARREG <= ATR', 'PrevVol <= LINEARREG_ANGLE', 'LINEARREG <= 20MA', 'VAR >= ADX', 'LowerBB >= LINEARREG_SLOPE', 'MidBB <= PrevHigh', 'TSF <= LINEARREG_SLOPE', 'LINEARREG_SLOPE <= PrevLow', 'MOM >= SAR', 'LowerBB >= VAR', 'LINEARREG >= PrevOpen', 'LINEARREG_SLOPE <= TSF', 'MACDSIGNAL >= ADX', 'BETA >= UpperBB', 'PrevIndLow <= 20MA', 'PrevIndOpen >= PrevVol', 'RSI <= 9MA', 'UpperBB >= PrevIndHigh', 'BETA <= PrevIndOpen', 'LowerBB >= UpperBB', 'Open <= PrevIndClose', 'PrevHigh <= MidBB', 'LINEARREG >= PrevLow', 'Open >= MOM', 'BETA >= SAR', 'PrevIndClose >= UpperBB', 'LINEARREG_ANGLE <= PrevVol', 'ROC >= RSI', 'VAR <= RSI', 'TSF <= MOM', 'ATR <= UpperBB', 'PrevHigh >= 9MA', 'SAR >= PrevIndVol', 'PrevIndClose >= STDDEV', 'MACD <= LowerBB', '9MA >= STDDEV', 'PrevIndOpen <= UpperBB', '9MA <= LINEARREG_ANGLE', 'LINEARREG_SLOPE <= ADX', 'MACDHIST <= PrevVol', 'PrevLow <= PrevIndLow', 'PrevHigh >= TSF', 'PrevIndHigh >= RSI', 'MACDHIST >= PrevIndVol', 'Open >= ATR', 'Open >= SAR', 'PrevIndClose <= LowerBB', '9MA <= PrevAdjClose', 'PrevIndLow >= VAR', 'MACDHIST <= LINEARREG', '9MA <= PrevIndLow', 'PrevIndOpen >= MACD', 'PrevIndLow >= 9MA', 'RSI <= ATR', 'ROC <= RSI', 'LINEARREG_ANGLE <= PrevIndClose', 'ADX >= LINEARREG_ANGLE', 'BETA >= ROC', '20MA >= Open', 'PrevIndVol <= PrevIndClose', 'Open >= PrevIndHigh', 'TSF >= ATR', 'ADX >= MidBB', 'PrevHigh >= 20MA', 'LINEARREG >= PrevIndHigh', 'MACD >= PrevIndHigh', 'MOM >= PrevIndLow', 'PrevIndOpen <= LINEARREG_SLOPE', 'MidBB >= MACDSIGNAL', 'LowerBB <= ROC', '9MA >= SAR', 'LINEARREG_SLOPE <= STDDEV', 'ADX <= TSF', 'PrevHigh', 'PrevHigh >= LINEARREG_SLOPE', 'Open <= LINEARREG_ANGLE', 'MACD <= UpperBB', 'PrevIndVol <= PrevIndLow', 'MACDSIGNAL >= LINEARREG', 'Open <= LINEARREG', 'ATR <= PrevIndVol', 'PrevIndLow <= PrevOpen', 'TSF <= LINEARREG_ANGLE', 'VAR <= PrevIndOpen', 'PrevOpen >= MACDSIGNAL', '9MA <= PrevIndOpen', 'STDDEV <= Open', 'LINEARREG_SLOPE >= STDDEV', 'LINEARREG_SLOPE >= TSF', 'PrevOpen >= PrevVol', 'RSI <= UpperBB', 'UpperBB <= ROC', 'BETA <= ROC', 'ATR >= PrevIndVol', 'LINEARREG >= 9MA', 'ROC <= LINEARREG_SLOPE', 'MACDSIGNAL <= 9MA', 'ROC >= MACDSIGNAL', 'MACDHIST <= MOM', 'Open >= STDDEV', 'PrevOpen >= PrevIndVol', 'RSI <= PrevVol', 'LINEARREG_ANGLE >= MidBB', 'ATR <= MACDSIGNAL', 'MACDHIST >= RSI', '9MA >= PrevIndVol', 'PrevIndVol <= PrevIndHigh', 'MACDHIST >= ADX', 'PrevVol <= LINEARREG', 'PrevVol <= Open', '20MA >= BETA', 'LINEARREG >= MOM', 'LINEARREG >= MidBB', 'ATR <= TSF', 'SAR >= Open', 'PrevIndHigh >= PrevIndOpen', 'PrevIndHigh <= TSF', 'PrevIndOpen <= MACD', 'VAR <= PrevHigh', 'TSF <= RSI', 'ROC >= TSF', 'VAR >= LowerBB', 'PrevIndLow <= MOM', 'PrevIndOpen >= MACDHIST', 'PrevHigh >= Open', 'LINEARREG_ANGLE >= PrevVol', 'MidBB >= LINEARREG_ANGLE', 'PrevOpen >= MOM', 'LINEARREG <= SAR', 'BETA >= MACDSIGNAL', 'RSI <= PrevAdjClose', 'BETA <= SAR', 'MACD >= MOM', 'ADX >= MOM', 'PrevIndOpen >= MidBB', 'ATR <= PrevAdjClose', 'UpperBB >= 9MA', 'PrevIndLow >= PrevIndClose', '9MA >= PrevLow', 'RSI >= PrevHigh', 'ROC >= MACD', 'STDDEV <= SAR', 'ATR >= LINEARREG_SLOPE', 'LINEARREG_SLOPE <= LINEARREG', '20MA >= SAR', 'PrevIndClose <= MACDHIST', 'MidBB <= SAR', 'MidBB >= ROC', '9MA >= PrevOpen', 'ADX >= UpperBB', 'PrevIndHigh >= LINEARREG', 'MACDSIGNAL >= ROC', 'PrevIndClose >= SAR', 'Open <= LowerBB', 'MidBB >= PrevHigh', 'ATR <= PrevVol', 'MACD <= MOM', 'TSF <= MACD', 'TSF >= MACDHIST', 'PrevOpen >= LINEARREG_ANGLE', 'STDDEV >= PrevIndHigh', 'PrevAdjClose >= VAR', 'PrevOpen >= ATR', 'TSF >= 9MA', 'PrevVol <= PrevOpen', 'MACD <= LINEARREG_ANGLE', 'PrevAdjClose <= PrevIndClose', 'BETA >= PrevIndOpen', 'TSF <= ATR', 'PrevIndVol >= PrevIndOpen', 'PrevIndVol >= PrevIndClose', 'BETA <= STDDEV', 'RSI >= LINEARREG_ANGLE', 'LowerBB >= MOM', 'MOM <= PrevAdjClose', '9MA >= TSF', 'VAR >= MidBB', 'SAR <= VAR', 'Open <= MidBB', 'PrevVol <= LowerBB', 'PrevHigh <= LINEARREG', 'BETA <= PrevIndHigh', 'PrevIndClose >= PrevAdjClose', 'ATR >= PrevVol', 'RSI >= PrevIndClose', 'PrevLow >= Open', 'LowerBB <= LINEARREG', 'RSI >= TSF', 'PrevLow >= STDDEV', 'ADX <= PrevLow', 'LINEARREG_ANGLE <= UpperBB', 'PrevIndClose <= PrevLow', '9MA <= VAR', 'STDDEV <= UpperBB', 'TSF >= MOM', 'PrevIndOpen >= ADX', 'PrevIndLow <= UpperBB', 'PrevIndHigh >= PrevLow', 'ATR <= 20MA', '9MA >= MACDHIST', 'PrevLow <= MACD', 'ROC >= PrevAdjClose', 'LINEARREG_SLOPE >= MACDSIGNAL', 'LINEARREG_ANGLE >= PrevLow', 'PrevOpen >= MACDHIST', 'STDDEV <= PrevIndClose', 'PrevIndVol >= MidBB', 'PrevLow >= PrevHigh', 'UpperBB <= MACDSIGNAL', 'PrevHigh >= RSI', 'TSF >= MidBB', '9MA >= ROC', 'ATR >= MACD', 'VAR >= MACD', 'MACD >= SAR', 'MACDSIGNAL <= Open', 'MOM <= ADX', 'PrevVol <= UpperBB', 'PrevHigh <= PrevIndLow', 'MACD >= LowerBB', 'LINEARREG_SLOPE <= MACDSIGNAL', 'BETA <= MidBB', 'ROC <= MACD', 'MACD >= LINEARREG', 'LINEARREG_ANGLE <= PrevOpen', 'Open', 'ROC <= TSF', 'MACDSIGNAL <= 20MA', 'LowerBB <= PrevIndLow', 'VAR <= PrevIndHigh', 'PrevIndClose >= LowerBB', '9MA >= PrevIndLow', 'MACDSIGNAL <= LINEARREG_ANGLE', 'ROC >= LowerBB', 'PrevAdjClose <= VAR', 'SAR <= PrevOpen', 'PrevHigh <= 9MA', 'PrevHigh <= RSI', 'ROC <= STDDEV', 'SAR >= MACD', 'PrevIndHigh <= MACD', 'BETA >= PrevLow', '20MA <= ROC', 'MACDSIGNAL <= PrevLow', 'ADX <= PrevIndLow', 'TSF <= ROC', 'MACDSIGNAL <= LowerBB', 'STDDEV <= 9MA', 'STDDEV <= PrevIndHigh', 'PrevHigh <= ADX', 'PrevLow <= 20MA', 'LowerBB >= PrevIndClose', 'LowerBB >= LINEARREG_ANGLE', 'MOM >= PrevOpen', 'BETA >= 20MA', '20MA >= LowerBB', 'ADX >= PrevIndVol', 'LINEARREG <= PrevIndClose', '9MA <= PrevIndVol', 'PrevLow <= STDDEV', 'PrevHigh <= PrevIndClose', '20MA >= LINEARREG_SLOPE', 'ADX >= PrevIndOpen', 'LINEARREG_ANGLE <= MACDHIST', '20MA >= STDDEV', 'RSI >= PrevVol', 'ROC >= PrevIndLow', 'PrevIndOpen <= SAR', 'UpperBB <= PrevIndHigh', 'PrevIndOpen <= Open', 'PrevHigh >= MACD', 'Open >= PrevIndVol', 'PrevOpen >= MACD', 'PrevAdjClose <= 20MA', 'STDDEV >= RSI', 'Open <= RSI', 'PrevIndLow <= PrevLow', 'MACDHIST >= PrevHigh', 'MACD <= PrevIndLow', 'LINEARREG_ANGLE <= LINEARREG', 'PrevIndLow <= PrevIndHigh', 'STDDEV >= MOM', 'UpperBB >= Open', 'MACDHIST >= STDDEV', 'LINEARREG_ANGLE >= MACDSIGNAL', 'PrevAdjClose <= MOM', 'MidBB >= BETA', 'LINEARREG_SLOPE <= LINEARREG_ANGLE', 'STDDEV <= PrevIndOpen', 'STDDEV >= MACDHIST', 'PrevOpen >= LINEARREG_SLOPE', 'SAR <= ROC', 'BETA >= PrevVol', 'PrevHigh >= MACDSIGNAL', 'MidBB <= ADX', 'MACDSIGNAL >= UpperBB', '9MA >= UpperBB', 'MOM <= VAR', 'PrevVol >= TSF', 'RSI >= ROC', 'LINEARREG_SLOPE >= PrevIndHigh', 'STDDEV <= LINEARREG_ANGLE', '20MA >= PrevAdjClose', '20MA >= PrevHigh', 'TSF <= BETA', 'PrevOpen <= STDDEV', 'LINEARREG_ANGLE <= PrevIndVol', 'PrevIndLow <= BETA', 'LINEARREG_SLOPE <= RSI', 'SAR >= PrevVol', 'PrevAdjClose >= PrevLow', 'LowerBB <= ADX', 'ATR >= MACDSIGNAL', 'ADX <= PrevIndVol', 'ADX <= MACDSIGNAL', '20MA >= LINEARREG_ANGLE', 'ADX <= UpperBB', 'PrevHigh >= MOM', 'BETA <= Open', 'PrevIndOpen <= PrevAdjClose', 'PrevAdjClose >= LINEARREG_SLOPE', 'ADX <= LINEARREG_ANGLE', 'MACDSIGNAL >= ATR', 'BETA >= RSI', 'PrevIndVol <= 20MA', '9MA <= PrevOpen', 'Open <= MACDSIGNAL', 'MACDSIGNAL <= MidBB', 'MACDHIST >= PrevOpen', 'PrevHigh <= MACDSIGNAL', 'PrevLow >= PrevAdjClose', 'PrevIndVol >= SAR', 'LINEARREG_ANGLE <= Open', 'PrevAdjClose >= UpperBB', 'PrevIndVol <= PrevLow', 'PrevIndVol >= PrevLow', 'MACDHIST <= PrevHigh', 'MACDHIST >= 20MA', 'PrevAdjClose >= RSI', 'TSF >= LINEARREG_ANGLE', 'MidBB <= ATR', 'PrevAdjClose >= TSF', 'SAR <= LINEARREG', 'LINEARREG_SLOPE >= RSI', 'PrevIndLow >= RSI', 'LINEARREG <= VAR', 'PrevIndOpen >= PrevIndLow', 'LowerBB >= RSI', 'LowerBB <= BETA', 'PrevIndOpen >= PrevLow', 'TSF >= BETA', 'RSI <= PrevLow', 'PrevHigh <= PrevVol', '20MA >= PrevIndVol', 'PrevIndClose >= BETA', 'MACDSIGNAL >= MOM', 'PrevLow <= PrevIndOpen', 'LINEARREG_ANGLE <= MidBB', 'MOM <= LINEARREG_SLOPE', 'PrevIndVol <= MACD', 'PrevAdjClose >= LINEARREG_ANGLE', 'ADX <= BETA', 'PrevIndClose >= PrevLow', 'PrevOpen <= MACD', 'ATR >= PrevIndLow', 'PrevIndHigh >= BETA', 'MOM <= LINEARREG', 'PrevAdjClose >= PrevVol', 'MACDHIST >= MACD', 'PrevAdjClose <= ATR', 'PrevIndHigh <= PrevIndVol', 'SAR >= PrevHigh', 'PrevIndLow <= RSI', 'PrevIndHigh <= LINEARREG_SLOPE', 'PrevIndHigh <= MACDSIGNAL', 'PrevIndLow <= TSF', 'PrevVol <= MACDSIGNAL', 'LINEARREG >= LINEARREG_ANGLE', 'RSI >= LowerBB', 'MidBB >= 9MA', 'PrevIndClose >= MOM', 'TSF >= PrevIndOpen', 'LowerBB >= MidBB', 'LINEARREG >= UpperBB', '20MA <= MACD', 'PrevAdjClose <= ROC', 'PrevIndVol <= VAR', 'MOM >= ADX', 'MidBB >= ADX', 'LINEARREG_ANGLE >= PrevIndOpen', 'LINEARREG >= LowerBB', 'LowerBB >= MACDHIST', 'ROC <= PrevIndVol', 'PrevLow <= SAR', 'PrevIndOpen >= ROC', 'PrevAdjClose <= MACDSIGNAL', 'MACD <= PrevLow', 'MidBB <= PrevIndVol', 'PrevIndOpen <= PrevIndHigh', 'SAR >= 20MA', 'LINEARREG_SLOPE >= 20MA', 'PrevIndLow >= PrevHigh', 'MACD >= MACDHIST', 'TSF >= PrevIndVol', 'VAR <= PrevLow', 'PrevAdjClose >= PrevIndVol', 'PrevAdjClose >= ATR', 'TSF <= PrevIndClose', 'LINEARREG <= LINEARREG_SLOPE', 'LINEARREG_ANGLE >= VAR', 'MidBB >= PrevAdjClose', 'LowerBB >= PrevIndVol', 'LINEARREG_ANGLE >= UpperBB', 'PrevHigh >= LINEARREG_ANGLE', 'VAR <= PrevIndClose', 'ATR <= LINEARREG_ANGLE', '9MA >= PrevHigh', 'PrevHigh >= PrevAdjClose', '9MA <= LowerBB', 'LINEARREG_SLOPE >= PrevAdjClose', 'UpperBB <= 9MA', 'PrevIndLow <= PrevHigh', 'MACD <= PrevIndClose', 'PrevAdjClose >= MOM', 'PrevVol >= VAR', 'MACDHIST <= MACDSIGNAL', 'TSF >= LowerBB', 'PrevIndOpen <= PrevLow', 'PrevIndVol <= LINEARREG_SLOPE', 'STDDEV <= TSF', 'MACD >= PrevAdjClose', 'BETA <= PrevAdjClose', 'PrevIndOpen <= MidBB', 'MACDSIGNAL <= ATR', 'SAR <= MACDSIGNAL', 'PrevIndLow >= PrevIndOpen', 'PrevIndClose >= VAR', 'BETA >= Open', 'PrevIndClose >= PrevIndLow', 'MACD >= TSF', 'RSI >= BETA', 'SAR >= ADX', 'MACD <= MidBB', 'PrevOpen <= MOM', 'PrevLow <= RSI', 'MACDSIGNAL >= PrevIndOpen', 'BETA <= LINEARREG_ANGLE', 'Open <= VAR', 'ROC >= Open', 'ATR >= PrevIndClose', 'PrevIndVol >= MACDSIGNAL', 'TSF >= Open', 'VAR >= LINEARREG_ANGLE', 'PrevLow <= TSF', 'ADX <= PrevHigh', 'PrevAdjClose <= BETA', 'PrevIndHigh >= LowerBB', 'PrevIndLow >= BETA', '9MA >= LowerBB', 'SAR >= PrevIndOpen', '20MA <= PrevIndOpen', 'PrevVol <= PrevIndHigh', 'MACDSIGNAL >= PrevIndHigh', 'ATR <= LowerBB', 'ROC >= LINEARREG', 'PrevIndLow <= LINEARREG_SLOPE', 'PrevLow <= ROC', 'MidBB <= Open', 'LINEARREG_ANGLE >= PrevIndClose', 'Open <= ATR', 'VAR >= 9MA', 'PrevVol <= STDDEV', 'PrevVol >= ATR', '9MA <= RSI', 'STDDEV <= ADX', 'LINEARREG_SLOPE <= PrevIndVol', 'ATR <= SAR', 'ADX >= VAR', '20MA >= ATR', 'STDDEV >= UpperBB', 'PrevVol >= PrevIndClose', 'PrevVol <= PrevLow', 'MACDHIST >= PrevVol', 'LINEARREG_ANGLE <= RSI', 'MidBB >= PrevOpen', 'TSF <= STDDEV', 'MACDSIGNAL >= PrevAdjClose', 'MACDSIGNAL >= 20MA', 'ADX <= MACD', 'MACD <= MACDSIGNAL', 'PrevIndLow >= MidBB', 'PrevVol >= PrevLow', 'BETA <= RSI', 'LINEARREG_ANGLE <= PrevHigh', 'Open >= ADX', 'UpperBB >= MACD', 'Open >= PrevAdjClose', 'PrevAdjClose >= BETA', 'MACDHIST >= PrevAdjClose', 'PrevVol <= SAR', 'LINEARREG_ANGLE >= TSF', '9MA <= PrevVol', 'PrevIndVol <= ADX', 'VAR >= PrevIndOpen', 'MOM <= SAR', 'VAR <= Open', 'SAR >= LINEARREG_SLOPE', 'PrevHigh <= PrevIndHigh', '20MA <= PrevLow', 'MidBB <= MACDHIST', 'BETA >= PrevIndLow', 'UpperBB >= PrevIndOpen', 'ROC <= PrevIndLow', 'UpperBB >= LowerBB', 'MidBB >= LINEARREG', 'PrevOpen <= LINEARREG_ANGLE', 'MACDSIGNAL >= LINEARREG_SLOPE', 'PrevAdjClose <= LINEARREG_ANGLE', 'PrevOpen <= ATR', 'LINEARREG <= STDDEV', 'VAR >= UpperBB', 'ADX <= STDDEV', 'PrevIndHigh >= PrevOpen', 'MACDHIST >= PrevIndOpen', 'SAR >= MOM', 'LINEARREG_SLOPE <= LowerBB', 'STDDEV >= PrevVol', 'PrevIndVol >= MOM', 'LINEARREG >= PrevIndClose', 'ATR >= ROC', 'PrevIndLow >= MOM', 'MidBB >= RSI', 'BETA >= PrevIndVol', 'MACD >= ADX', 'LINEARREG <= RSI', 'PrevAdjClose <= PrevIndLow', 'UpperBB <= MACDHIST', 'PrevIndOpen <= LINEARREG', 'LINEARREG_ANGLE <= ADX', 'ATR >= LowerBB', 'MidBB >= MACDHIST', 'PrevOpen <= PrevIndClose', 'UpperBB <= ADX', 'Open >= PrevLow', 'LINEARREG_ANGLE <= SAR', 'PrevLow >= PrevVol', 'PrevLow >= PrevIndVol', 'PrevHigh >= MidBB', 'PrevAdjClose >= PrevIndOpen', 'ATR >= PrevIndHigh', 'ATR <= MACD', 'SAR >= RSI', 'BETA >= MACD', 'MACDSIGNAL >= RSI', 'UpperBB >= RSI', 'LINEARREG <= LINEARREG_ANGLE', 'LINEARREG_SLOPE <= VAR', 'MACDHIST <= PrevIndLow', 'PrevIndClose <= ROC', 'Open <= ROC', 'LINEARREG_SLOPE >= ROC', 'LINEARREG_SLOPE <= 9MA', 'STDDEV <= MACD', 'RSI <= ROC', 'TSF >= 20MA', '9MA <= PrevIndClose', 'ROC <= PrevVol', 'ADX <= ATR', 'SAR >= LowerBB', 'ATR <= LINEARREG', 'PrevIndClose <= LINEARREG_SLOPE', 'RSI <= MACDSIGNAL', 'PrevLow >= PrevIndClose', 'TSF <= MidBB', 'PrevHigh >= ROC', 'PrevOpen >= LowerBB', 'LINEARREG <= ROC', 'PrevIndOpen <= ATR', 'MidBB >= SAR', 'Open >= MidBB', 'VAR <= PrevOpen', 'PrevVol >= STDDEV', 'STDDEV <= PrevIndVol', 'LINEARREG >= STDDEV', 'MACDHIST <= Open', 'PrevLow >= ATR', 'PrevIndVol <= PrevOpen', 'LINEARREG_SLOPE >= PrevHigh', 'PrevHigh >= PrevVol', 'SAR >= MACDHIST', '20MA >= PrevOpen', 'UpperBB >= MOM', 'ATR >= MidBB', '9MA >= ADX', 'MidBB <= ROC', 'LowerBB <= STDDEV', 'MidBB <= RSI', 'MACD <= PrevOpen', 'PrevAdjClose <= PrevHigh', 'PrevLow <= MidBB', 'TSF <= PrevVol', 'RSI >= LINEARREG', '20MA <= BETA', 'BETA >= PrevIndClose', 'TSF <= PrevOpen', 'ATR <= PrevHigh', 'LINEARREG_ANGLE <= ATR', 'VAR >= Open', 'PrevIndLow <= ADX', 'PrevLow >= BETA', 'ATR >= MACDHIST', 'ADX >= ROC', 'STDDEV >= PrevIndClose', 'BETA <= PrevOpen', 'PrevLow <= ADX', 'PrevAdjClose >= PrevOpen', 'PrevVol >= 9MA', 'MOM >= PrevIndOpen', 'PrevVol <= LINEARREG_SLOPE', 'Open <= BETA', 'PrevIndLow <= ATR', 'PrevAdjClose >= PrevHigh', 'PrevIndHigh <= MACDHIST', 'MACD >= PrevVol', 'ATR <= 9MA', 'RSI <= PrevOpen', 'ROC >= PrevIndVol', 'LowerBB >= SAR', 'TSF <= LINEARREG', 'MOM <= LowerBB', 'PrevIndClose <= PrevIndHigh', 'PrevIndLow <= PrevAdjClose', 'ADX <= MOM', 'MACDHIST <= ROC', 'RSI >= PrevAdjClose', 'SAR <= PrevHigh', 'Open <= TSF', 'PrevIndClose >= PrevOpen', 'ROC >= MidBB', 'PrevVol >= PrevHigh', 'PrevIndHigh <= 20MA', 'ROC <= 20MA', 'ATR >= PrevOpen', 'MOM >= ROC', '9MA >= PrevVol', 'PrevIndVol <= MidBB', 'PrevIndHigh >= MACD', 'MACD <= STDDEV', 'PrevIndVol >= ADX', 'PrevOpen >= ADX', 'PrevAdjClose <= UpperBB', 'PrevHigh <= VAR', 'PrevOpen >= PrevAdjClose', 'MidBB <= PrevVol', 'MACDSIGNAL >= PrevIndClose', '9MA >= PrevIndClose', 'BETA >= LINEARREG_ANGLE', 'PrevHigh <= MOM', 'SAR <= PrevIndClose', 'ROC <= MACDHIST', 'MACDSIGNAL <= UpperBB', 'PrevAdjClose <= PrevLow', 'UpperBB <= PrevIndOpen', 'MACDHIST >= PrevIndLow', 'PrevOpen <= PrevIndVol', 'STDDEV >= LINEARREG_SLOPE', 'VAR <= PrevVol', 'TSF <= PrevIndOpen', 'MidBB >= 20MA', 'BETA >= STDDEV', 'STDDEV >= PrevIndOpen', 'ADX >= PrevOpen', 'Open >= LowerBB', 'PrevIndOpen >= UpperBB', 'PrevAdjClose >= PrevIndClose', 'PrevAdjClose <= RSI', 'ROC >= LINEARREG_ANGLE', 'MACDHIST <= PrevLow', 'Open <= ADX', '9MA <= ADX', 'MACDHIST <= PrevIndClose', 'TSF <= PrevIndVol', 'PrevVol >= MOM', 'PrevLow >= MACDSIGNAL', 'PrevIndClose >= ROC', 'UpperBB >= PrevOpen', 'PrevIndLow <= LINEARREG_ANGLE', 'VAR >= LINEARREG_SLOPE', 'LowerBB >= BETA', 'UpperBB <= PrevVol', 'ADX >= RSI', 'MOM >= PrevIndHigh', 'PrevIndVol >= PrevAdjClose', 'PrevHigh >= SAR', 'PrevHigh <= PrevIndVol', 'LINEARREG_ANGLE >= MOM', 'LINEARREG_SLOPE >= Open', 'LINEARREG <= BETA', 'PrevIndHigh <= MidBB', 'PrevOpen <= PrevHigh', 'PrevIndVol >= LINEARREG', 'PrevIndHigh >= ATR', 'MACDSIGNAL <= PrevHigh', 'UpperBB >= BETA', 'BETA >= PrevIndHigh', 'BETA <= PrevIndLow', 'SAR <= PrevLow', 'PrevIndOpen >= SAR', 'LINEARREG <= LowerBB', 'MACD <= 9MA', 'UpperBB <= 20MA', 'BETA <= PrevHigh', 'MACD >= RSI', '20MA >= VAR', 'MACDSIGNAL >= BETA', '20MA <= ATR', 'PrevIndOpen >= LINEARREG_SLOPE', 'PrevAdjClose <= PrevIndOpen', 'MidBB >= PrevIndLow', 'PrevOpen >= PrevIndHigh', '20MA <= SAR', 'MidBB >= Open', 'SAR <= LowerBB', 'TSF >= ADX', 'LINEARREG >= Open', 'SAR <= Open', 'BETA >= ATR', 'PrevIndHigh <= PrevHigh', 'BETA >= 9MA', 'Open >= MACDHIST', 'PrevOpen <= ADX', 'MACD <= LINEARREG', 'MOM <= ATR', 'MACDHIST <= ADX', 'LINEARREG_ANGLE <= PrevAdjClose', 'MidBB >= MACD', 'ROC <= MACDSIGNAL', 'PrevIndVol >= PrevVol', 'PrevVol >= MidBB', 'PrevIndOpen <= PrevHigh', 'UpperBB >= ROC', 'BETA >= LINEARREG', 'LINEARREG >= ATR', 'PrevIndHigh >= MACDHIST', 'MOM >= PrevIndVol', 'PrevIndHigh <= LowerBB', '9MA >= LINEARREG_ANGLE', 'SAR >= LINEARREG_ANGLE', '20MA >= UpperBB', '20MA >= PrevIndHigh', 'PrevIndOpen <= MACDSIGNAL', 'ADX <= PrevVol', 'PrevLow <= 9MA', 'MOM >= BETA', 'MidBB >= PrevIndVol', 'ATR >= 20MA', 'TSF <= MACDSIGNAL', 'VAR <= SAR', 'MidBB <= UpperBB', 'MACD >= Open', 'PrevIndLow >= LowerBB', 'MACDSIGNAL <= PrevAdjClose', '20MA <= PrevOpen', 'VAR >= 20MA', 'TSF <= ADX', 'PrevLow <= MACDSIGNAL', 'MOM >= MidBB', 'STDDEV >= PrevOpen', 'LINEARREG_ANGLE <= PrevLow', 'MACDHIST >= PrevLow', 'MACD <= ROC', 'UpperBB >= PrevIndVol', 'MOM <= MACDSIGNAL', 'SAR >= PrevIndClose', 'PrevIndClose >= PrevIndVol', 'ROC <= MOM', 'VAR >= PrevAdjClose', 'MidBB <= LINEARREG_SLOPE', 'LINEARREG_SLOPE >= PrevLow', 'PrevHigh <= STDDEV', 'MACDHIST >= ROC', 'MACDSIGNAL >= MidBB', 'STDDEV >= BETA', 'ATR <= LINEARREG_SLOPE', 'PrevIndHigh <= ROC', 'MACDHIST >= VAR', 'MACDSIGNAL >= Open', '20MA >= PrevIndClose', 'PrevIndClose <= ATR', 'MACDSIGNAL <= SAR', 'ATR <= PrevIndOpen', 'ROC >= SAR', 'VAR', 'PrevIndOpen >= ATR', 'MACDHIST <= MACD', 'VAR >= STDDEV', 'PrevAdjClose <= LINEARREG_SLOPE', 'STDDEV >= SAR', 'MACDHIST >= LINEARREG', 'PrevIndVol <= BETA', 'SAR >= PrevAdjClose', 'LINEARREG_ANGLE >= PrevAdjClose', 'PrevIndOpen <= ADX', 'UpperBB >= PrevLow', 'ATR >= PrevAdjClose', 'VAR <= PrevAdjClose', 'UpperBB >= PrevIndLow', 'ADX >= LowerBB', 'VAR >= ATR', 'PrevHigh >= PrevIndOpen', 'LINEARREG_SLOPE', '20MA >= PrevIndLow', 'PrevIndLow >= 20MA', 'PrevLow >= RSI', 'UpperBB >= LINEARREG_SLOPE', '20MA <= RSI', 'PrevIndLow >= PrevVol', 'PrevLow >= TSF', 'Open <= MACDHIST', '20MA >= PrevLow', 'LINEARREG_ANGLE >= STDDEV', 'PrevIndHigh >= PrevAdjClose', 'PrevIndClose <= 20MA', 'ROC >= PrevIndHigh', 'RSI >= PrevIndVol', 'PrevIndVol >= MACDHIST', 'PrevHigh <= 20MA', 'UpperBB <= PrevIndLow', 'PrevAdjClose >= 20MA', '20MA <= PrevHigh', 'ADX >= PrevIndLow', 'ATR >= LINEARREG_ANGLE', 'TSF >= SAR', 'PrevHigh <= LINEARREG_ANGLE', 'PrevAdjClose >= STDDEV', 'RSI <= PrevIndClose', 'MidBB <= PrevAdjClose', 'PrevIndHigh >= LINEARREG_SLOPE', 'MACDSIGNAL <= PrevIndLow', 'LINEARREG_ANGLE <= MACD', 'PrevIndClose <= BETA', 'PrevVol >= SAR', 'MOM <= ROC', 'PrevHigh >= BETA', 'PrevAdjClose <= MACDHIST', 'MOM >= TSF', 'MACDSIGNAL >= SAR', '20MA <= UpperBB', 'LINEARREG_ANGLE >= PrevOpen', 'VAR >= PrevHigh', 'ATR >= MOM', 'LINEARREG >= MACD', 'PrevIndHigh >= UpperBB', 'UpperBB <= MidBB', 'PrevIndOpen <= MOM', 'SAR >= VAR', 'LINEARREG_SLOPE >= BETA', 'MOM >= 20MA', 'PrevVol >= PrevOpen', 'PrevIndVol >= PrevIndHigh', 'PrevIndLow <= PrevIndOpen', 'SAR <= PrevIndVol', 'PrevIndOpen <= BETA', 'Open <= MOM', 'ROC <= BETA', 'PrevIndOpen >= PrevAdjClose', 'PrevHigh >= PrevIndVol', 'MACDSIGNAL <= PrevIndClose', 'PrevLow >= LINEARREG_SLOPE', 'VAR <= LINEARREG_SLOPE', 'ATR <= RSI', 'MACD <= BETA', 'BETA >= ADX', 'ATR >= LINEARREG', 'PrevIndVol >= PrevHigh', 'SAR >= 9MA', '9MA >= RSI', 'PrevLow <= LowerBB', 'RSI >= MACDSIGNAL', 'TSF', 'LINEARREG_ANGLE <= 20MA', 'PrevIndClose >= LINEARREG_SLOPE', 'STDDEV >= 9MA', 'PrevVol >= PrevIndOpen', 'RSI <= TSF', 'LINEARREG_SLOPE >= PrevIndVol', 'PrevIndClose <= RSI', 'UpperBB <= TSF', 'MACDHIST <= PrevIndVol', 'Open <= MACD', 'PrevIndOpen >= LowerBB', 'LINEARREG >= BETA', '9MA >= LINEARREG_SLOPE', '9MA <= ATR', 'PrevAdjClose >= ADX', 'RSI >= PrevIndLow', 'PrevHigh <= LowerBB', 'UpperBB <= LINEARREG', 'SAR >= BETA', 'MidBB <= PrevOpen', 'MidBB >= MOM', '20MA >= MACD', 'RSI <= STDDEV', 'Open >= PrevIndClose', 'LowerBB <= 9MA', 'PrevIndLow <= MACDSIGNAL', 'VAR <= TSF', 'Open <= PrevOpen', 'LowerBB >= TSF', 'MOM >= LINEARREG', 'ADX <= 9MA', 'MACDHIST <= VAR', 'ATR <= BETA', 'PrevVol <= BETA', 'PrevLow >= 9MA', 'ROC >= PrevOpen', 'LINEARREG >= PrevVol', 'PrevVol <= PrevIndVol', 'VAR >= PrevIndVol', 'PrevIndOpen <= ROC', 'STDDEV <= MidBB', 'PrevIndVol <= MACDHIST', 'PrevVol >= Open', 'ATR <= MACDHIST', 'PrevAdjClose >= MACD', 'MOM >= PrevAdjClose', 'ADX >= LINEARREG_SLOPE', 'SAR >= ROC', 'BETA <= LINEARREG', '20MA <= Open', '9MA <= SAR', 'ADX >= PrevAdjClose', 'UpperBB <= BETA', 'MOM >= MACDSIGNAL', 'LINEARREG_SLOPE >= PrevVol', 'TSF >= MACDSIGNAL', 'PrevAdjClose <= ADX', 'BETA <= PrevIndVol', 'VAR >= RSI', 'STDDEV <= ATR', 'SAR <= LINEARREG_ANGLE', 'PrevOpen <= ROC', 'PrevIndClose >= LINEARREG', 'ROC <= UpperBB', 'SAR <= STDDEV', 'MOM >= LINEARREG_ANGLE', 'PrevLow >= 20MA', 'PrevOpen >= ROC', 'MACD >= PrevHigh', '20MA >= RSI', 'Open >= ROC', 'LINEARREG_ANGLE <= LINEARREG_SLOPE', 'LINEARREG >= PrevIndOpen', 'PrevVol <= MidBB', 'PrevVol <= PrevIndOpen', 'ATR >= STDDEV', 'ADX <= LowerBB', 'ROC <= PrevIndHigh', 'Open <= SAR', '20MA <= LowerBB', 'MidBB <= LINEARREG_ANGLE', 'VAR <= ATR', 'PrevIndVol <= RSI', 'PrevIndLow <= MACDHIST', 'LINEARREG_SLOPE >= MidBB', 'RSI >= 9MA', 'MACDHIST <= STDDEV', 'MACD <= SAR', 'VAR <= PrevIndLow', 'PrevIndVol >= TSF', 'MACD >= UpperBB', 'PrevOpen >= LINEARREG', 'TSF >= LINEARREG_SLOPE', 'UpperBB >= ADX', 'PrevLow >= PrevIndHigh', 'MidBB <= 20MA', 'PrevAdjClose >= MACDHIST', 'PrevAdjClose <= STDDEV', 'PrevHigh >= PrevIndHigh', 'LINEARREG <= MidBB', '9MA >= PrevIndHigh', 'MACDHIST <= 9MA', 'BETA >= LowerBB', 'PrevIndLow >= MACDSIGNAL', 'SAR <= ATR', 'PrevAdjClose <= LowerBB', 'PrevIndOpen >= RSI', 'PrevIndHigh <= PrevAdjClose', 'MACD <= PrevIndOpen', 'MACDHIST <= TSF', 'LINEARREG_ANGLE <= LowerBB', 'MACDHIST <= LowerBB', 'PrevIndVol >= 20MA', '20MA >= ROC', 'LINEARREG_SLOPE >= LINEARREG', '20MA <= MOM', 'LowerBB <= MidBB', 'PrevOpen >= 20MA', 'UpperBB >= LINEARREG_ANGLE', 'PrevIndHigh >= MACDSIGNAL', 'PrevIndClose <= PrevIndOpen', 'PrevIndOpen <= VAR', 'MACDHIST <= ATR', 'PrevIndLow <= LowerBB', 'PrevIndHigh <= PrevOpen', 'LINEARREG <= 9MA', 'MACD <= ATR', 'PrevOpen >= PrevIndClose', 'LINEARREG_SLOPE >= PrevIndClose', 'ROC >= VAR', 'LINEARREG >= PrevIndLow', 'PrevIndVol >= VAR', 'BETA >= MACDHIST', 'ADX >= SAR', 'ROC <= 9MA', 'UpperBB >= PrevHigh', 'MACD <= PrevIndHigh', 'PrevIndHigh <= RSI', 'PrevIndHigh <= PrevVol', 'LINEARREG <= MACDHIST', 'LINEARREG >= SAR', 'MACD <= ADX', 'TSF >= PrevIndLow', 'PrevIndOpen <= PrevIndLow', 'LowerBB >= PrevHigh', 'PrevVol >= LowerBB', 'LINEARREG_SLOPE >= PrevOpen', 'LINEARREG_ANGLE >= BETA', 'RSI <= VAR', 'RSI >= UpperBB', 'ROC >= ADX', 'ROC <= PrevAdjClose', 'STDDEV >= LINEARREG_ANGLE', 'LowerBB <= PrevIndClose', 'STDDEV >= LowerBB', 'ADX >= Open', 'MACDHIST <= 20MA', 'PrevIndVol >= BETA', 'PrevIndLow >= LINEARREG_ANGLE', 'PrevIndHigh >= MidBB', 'MOM >= Open', 'RSI >= VAR', 'ROC <= ADX', 'LowerBB >= STDDEV', 'PrevOpen >= UpperBB', 'ROC >= PrevIndClose', 'TSF <= PrevIndHigh', 'PrevVol <= VAR', '9MA >= 20MA', 'MOM <= PrevOpen', 'TSF >= PrevLow', '9MA >= MOM', 'PrevIndVol >= LINEARREG_ANGLE', 'PrevIndLow >= SAR', 'UpperBB <= LINEARREG_SLOPE', 'PrevVol <= ROC', 'PrevIndLow >= LINEARREG_SLOPE', 'MOM <= PrevIndVol', 'STDDEV >= PrevAdjClose', 'PrevIndOpen >= 20MA', 'MACD >= PrevOpen', 'VAR <= BETA', 'MACDSIGNAL >= PrevOpen', 'MidBB >= PrevIndHigh', 'PrevIndVol <= ATR', 'LINEARREG_SLOPE <= Open', '20MA <= STDDEV', 'UpperBB <= PrevLow', 'RSI >= PrevIndOpen', 'PrevVol >= MACD', 'BETA <= TSF', 'LINEARREG_SLOPE <= PrevAdjClose', 'PrevIndVol >= RSI', 'ATR <= PrevLow', 'PrevIndHigh <= PrevLow', 'PrevVol <= 9MA', 'VAR <= 9MA', 'ROC <= SAR', 'MACD >= PrevIndLow', 'LINEARREG_ANGLE >= PrevHigh', 'PrevHigh >= PrevIndLow', 'PrevVol <= MACD', 'PrevIndVol >= PrevOpen', 'UpperBB <= SAR', 'VAR >= MACDSIGNAL', 'MidBB >= LINEARREG_SLOPE', 'LINEARREG_SLOPE <= PrevIndClose', 'PrevOpen <= LowerBB', 'RSI <= Open', 'Open >= 9MA', 'LINEARREG_SLOPE <= UpperBB', 'ATR <= PrevIndHigh', 'VAR <= 20MA', 'MACD >= ATR', 'ROC >= PrevHigh', 'PrevIndClose >= MACDSIGNAL', 'MOM >= 9MA', 'PrevIndHigh <= PrevIndClose', 'PrevIndVol >= MACD', 'PrevHigh >= VAR', 'SAR <= MACD', 'LowerBB <= MACDHIST', 'LINEARREG_ANGLE >= PrevIndHigh', 'LowerBB <= 20MA', 'PrevLow <= LINEARREG_ANGLE', 'PrevIndOpen >= 9MA', 'MACDSIGNAL >= MACDHIST', 'LINEARREG_ANGLE >= Open', '20MA', 'PrevOpen >= 9MA', 'SAR <= PrevIndOpen', 'PrevHigh >= PrevLow', 'PrevIndVol <= ROC', 'VAR <= LINEARREG_ANGLE', '9MA <= TSF', 'PrevIndVol >= LowerBB', 'ROC <= PrevLow', '20MA >= ADX', 'SAR >= PrevIndHigh', 'TSF >= PrevOpen', 'STDDEV >= MACD', 'LINEARREG <= PrevIndVol', 'PrevHigh <= LINEARREG_SLOPE', 'PrevLow >= LINEARREG_ANGLE', 'TSF <= PrevAdjClose', 'PrevHigh >= UpperBB', 'TSF >= PrevIndHigh', 'LINEARREG_SLOPE <= PrevIndHigh', 'PrevIndClose <= MACD', 'BETA <= 9MA', 'RSI <= LINEARREG_SLOPE', 'PrevIndClose <= PrevAdjClose', 'PrevIndVol <= MOM', 'MOM >= RSI', 'RSI >= STDDEV', 'PrevIndHigh <= ADX', 'PrevLow <= PrevAdjClose', 'PrevIndLow <= MACD', 'MidBB <= LowerBB', 'PrevIndHigh <= 9MA', 'ADX <= LINEARREG', 'MACDHIST >= MOM', 'PrevIndHigh <= Open', 'LINEARREG_ANGLE <= PrevIndLow', 'ROC <= Open', 'PrevIndHigh <= MOM', 'MACDSIGNAL <= PrevVol', 'MACDSIGNAL >= LINEARREG_ANGLE', 'Open <= PrevIndOpen', 'PrevIndOpen <= PrevIndVol', 'LINEARREG_SLOPE >= SAR', 'PrevIndClose >= PrevHigh', 'TSF >= ROC', 'PrevLow <= PrevIndHigh', '9MA', 'PrevIndClose >= MidBB', 'PrevHigh >= STDDEV', 'PrevIndLow <= ROC', 'LowerBB >= ATR', 'PrevLow <= MACDHIST', 'PrevIndVol <= PrevHigh', 'MACDHIST >= BETA', 'LowerBB >= LINEARREG', 'TSF >= VAR', '9MA >= MACD', 'BETA <= PrevIndClose', 'LINEARREG_SLOPE <= ROC', 'PrevIndHigh >= 20MA', '9MA <= MACDHIST', 'PrevOpen >= TSF', 'LINEARREG_SLOPE >= ATR', 'PrevIndHigh >= VAR', 'STDDEV <= PrevVol', 'LowerBB >= 20MA', 'LINEARREG_ANGLE >= ADX', 'ROC <= LINEARREG', 'MOM <= 9MA', 'PrevLow >= VAR', 'LINEARREG_SLOPE <= BETA', 'PrevIndClose <= LINEARREG_ANGLE', 'PrevIndOpen >= BETA', 'SAR <= UpperBB', 'MACDHIST <= LINEARREG_SLOPE', '9MA <= MACD', 'MACD <= PrevHigh', 'LINEARREG_SLOPE <= PrevHigh', 'LowerBB <= PrevIndOpen', 'Open >= PrevHigh', 'LINEARREG <= PrevAdjClose', 'Open <= PrevIndVol', 'LINEARREG_SLOPE >= MACDHIST', 'PrevIndHigh <= BETA', 'SAR >= MACDSIGNAL', 'ROC <= ATR', 'ROC >= ATR', 'PrevLow >= ADX', 'MACDSIGNAL <= VAR', '9MA >= VAR', 'LowerBB <= VAR', 'STDDEV >= ADX', 'MACDHIST <= SAR', 'PrevVol >= UpperBB', 'LowerBB >= PrevAdjClose', 'ROC <= PrevIndClose', 'MACDHIST >= LowerBB', 'MACDSIGNAL <= MACD', 'PrevVol >= MACDHIST', 'PrevHigh >= PrevOpen', 'MOM <= RSI', 'ADX <= 20MA', 'PrevVol <= PrevAdjClose', 'PrevOpen <= PrevIndHigh', 'LINEARREG_SLOPE <= PrevIndOpen', 'RSI <= MACD', 'PrevIndHigh >= TSF', 'RSI <= MOM', 'PrevIndHigh <= UpperBB', 'ADX >= PrevVol', 'SAR <= PrevIndHigh', 'SAR <= PrevIndLow', 'PrevIndHigh <= ATR', 'MACDSIGNAL >= MACD', '20MA <= VAR', 'PrevOpen <= LINEARREG', 'SAR <= ADX', 'PrevVol <= MACDHIST', 'ROC >= STDDEV', 'ATR >= BETA', 'LowerBB >= ADX', 'LINEARREG_SLOPE >= MACD', 'PrevIndOpen >= PrevIndHigh', 'MidBB <= STDDEV', 'LINEARREG_SLOPE <= PrevIndLow', 'MACDSIGNAL <= RSI', 'MOM >= LowerBB', 'PrevLow <= PrevOpen', 'PrevIndHigh <= VAR', 'PrevIndHigh >= ADX', 'MACDSIGNAL >= TSF', 'VAR >= ROC', 'UpperBB <= MOM', 'PrevAdjClose <= PrevOpen', 'LINEARREG_ANGLE <= PrevIndOpen', 'PrevLow >= MOM', 'Open >= LINEARREG_SLOPE', 'LINEARREG_ANGLE >= LINEARREG', '9MA <= MOM', 'STDDEV <= LINEARREG_SLOPE', 'MACD >= ROC', '9MA <= MACDSIGNAL', 'LINEARREG_ANGLE >= LowerBB', 'PrevIndLow <= Open', 'RSI <= PrevIndVol', 'MACDSIGNAL >= PrevIndLow', 'LINEARREG_ANGLE <= STDDEV', 'MOM <= UpperBB', 'STDDEV <= PrevOpen', 'SAR <= LINEARREG_SLOPE', 'VAR <= ROC', 'LINEARREG_ANGLE >= MACDHIST', 'STDDEV <= VAR', 'PrevVol >= LINEARREG', 'LINEARREG_SLOPE <= MidBB', 'ATR <= MidBB', 'RSI <= SAR', 'MACDSIGNAL >= PrevIndVol', 'LINEARREG <= MACDSIGNAL', 'MOM >= UpperBB', 'TSF >= MACD', 'PrevIndClose <= PrevHigh', 'PrevIndClose >= ATR', 'PrevIndVol <= LINEARREG_ANGLE', 'ADX <= PrevIndClose', 'Open <= LINEARREG_SLOPE', 'BETA >= MOM', 'RSI <= 20MA', 'PrevLow >= SAR', 'LowerBB <= MOM', 'SAR <= 9MA', 'MOM >= PrevIndClose', 'STDDEV >= MidBB', 'LINEARREG_SLOPE >= UpperBB', 'PrevIndVol >= LINEARREG_SLOPE', 'PrevAdjClose >= PrevIndLow', 'UpperBB <= PrevIndClose', 'ROC <= PrevHigh', 'PrevVol <= PrevIndLow', 'LINEARREG_SLOPE <= MOM', 'ATR >= PrevLow', 'MACD >= STDDEV', 'STDDEV <= PrevHigh', 'LowerBB <= LINEARREG_SLOPE', 'PrevLow >= MACD', 'LINEARREG_SLOPE <= ATR', 'PrevIndClose <= PrevOpen', 'SAR >= UpperBB', 'VAR <= UpperBB', 'PrevIndHigh <= STDDEV', 'PrevVol >= RSI', 'MACD <= LINEARREG_SLOPE', 'VAR <= PrevIndVol', 'LINEARREG >= ADX', 'STDDEV >= Open', 'MACDHIST >= PrevIndClose', 'LINEARREG_SLOPE >= PrevIndLow', '9MA >= MACDSIGNAL', 'TSF >= STDDEV', 'PrevVol >= ROC', 'PrevOpen >= PrevHigh', 'LINEARREG_ANGLE <= PrevIndHigh', 'LINEARREG_SLOPE <= 20MA', 'PrevIndOpen <= 9MA', 'VAR <= ADX', 'PrevOpen <= 20MA', 'MidBB >= UpperBB', 'LINEARREG_ANGLE >= ROC', 'UpperBB >= VAR', '20MA <= ADX', 'LINEARREG_ANGLE >= RSI', '9MA <= ROC', '20MA <= PrevIndClose', 'UpperBB <= PrevOpen', 'ADX >= 9MA', 'RSI <= MidBB', 'LowerBB <= MACD', '9MA <= Open', 'MOM >= MACDHIST', 'MACDSIGNAL <= PrevIndHigh', 'LINEARREG_SLOPE <= PrevOpen', 'ATR >= PrevIndOpen', 'PrevLow', 'TSF <= VAR', 'PrevIndLow <= PrevIndVol', 'PrevIndClose >= MACD', '20MA <= PrevIndVol', 'VAR >= MACDHIST', 'PrevIndVol >= 9MA', 'SAR <= MACDHIST', 'PrevAdjClose', 'PrevVol <= 20MA', 'LINEARREG_ANGLE >= ATR', 'PrevAdjClose >= SAR', 'PrevAdjClose <= MidBB', 'ROC >= 20MA', 'LINEARREG_SLOPE <= PrevVol', 'MidBB <= PrevIndHigh', 'RSI >= MACDHIST', 'PrevIndClose >= 20MA', 'LINEARREG', '9MA >= LINEARREG', 'PrevIndClose >= ADX', 'MOM <= LINEARREG_ANGLE', 'ROC <= LINEARREG_ANGLE', 'MACDHIST <= PrevOpen', 'VAR <= MACD', 'PrevLow <= BETA', 'LowerBB <= ATR', 'UpperBB >= 20MA', 'MidBB <= MACDSIGNAL', 'Open >= PrevIndOpen', 'MOM <= Open', 'ATR <= ADX', 'VAR >= BETA', 'PrevIndClose <= UpperBB', 'PrevHigh <= MACD', 'PrevVol >= PrevIndHigh', 'LINEARREG <= ADX', '20MA <= PrevIndHigh', 'PrevIndVol >= ATR', 'RSI >= PrevOpen', '9MA >= Open', 'MACDHIST >= UpperBB', 'LowerBB <= LINEARREG_ANGLE', 'PrevVol >= PrevIndLow', 'MACDHIST >= 9MA', 'PrevIndOpen >= MACDSIGNAL', 'BETA >= VAR', 'ADX >= PrevHigh', 'PrevOpen >= MidBB', 'PrevOpen <= PrevVol', 'BETA >= MidBB', 'MidBB >= PrevLow', 'LowerBB <= MACDSIGNAL', 'LINEARREG_SLOPE <= SAR', 'UpperBB >= LINEARREG', 'LINEARREG_ANGLE >= MACD', 'STDDEV <= PrevAdjClose', 'BETA >= PrevAdjClose', 'PrevLow <= PrevHigh', 'TSF <= 9MA', 'SAR >= LINEARREG', 'PrevVol <= PrevHigh', 'PrevOpen >= SAR', 'UpperBB <= LINEARREG_ANGLE', 'ADX >= 20MA', 'PrevOpen >= STDDEV', 'Open >= PrevIndLow', '9MA >= MidBB', 'PrevHigh <= PrevIndOpen', 'LowerBB <= PrevVol', 'LINEARREG_ANGLE >= PrevIndVol', 'MOM <= PrevLow', 'PrevVol >= LINEARREG_SLOPE', 'PrevIndClose <= MACDSIGNAL', 'ADX >= PrevIndClose', 'BETA <= ATR', 'MOM <= PrevIndLow', 'MOM <= PrevHigh', 'TSF >= PrevHigh', 'ADX <= LINEARREG_SLOPE', 'RSI >= 20MA', 'PrevAdjClose <= SAR', 'PrevIndLow >= ROC', '9MA <= BETA', 'VAR <= MACDHIST', 'LINEARREG_ANGLE >= SAR', 'LINEARREG <= MACD', 'UpperBB >= SAR', 'MACD <= TSF', 'Open >= BETA', 'STDDEV <= ROC', 'STDDEV >= PrevIndVol', 'MidBB', 'PrevIndHigh >= MOM', 'LINEARREG >= VAR', 'PrevIndHigh >= 9MA', '20MA <= LINEARREG_ANGLE', 'LINEARREG <= PrevLow', 'LINEARREG_SLOPE >= LowerBB', 'Open >= LINEARREG_ANGLE', '20MA <= PrevIndLow', 'LINEARREG >= MACDHIST', 'MidBB >= PrevIndOpen', 'ROC <= PrevIndOpen', 'Open >= 20MA', 'ATR <= PrevIndLow', 'UpperBB >= STDDEV', 'UpperBB >= TSF', 'ATR >= PrevHigh', 'RSI <= PrevIndHigh', 'RSI <= PrevIndOpen', 'PrevIndClose <= MidBB', 'PrevIndHigh <= LINEARREG_ANGLE', 'ADX <= MidBB', 'MOM >= ATR', 'SAR >= ATR', 'UpperBB <= Open', 'LINEARREG <= PrevIndHigh', 'UpperBB >= PrevIndClose', '20MA >= PrevVol', 'RSI >= MidBB', 'SAR <= MOM', 'PrevIndClose >= RSI', 'STDDEV >= ATR', 'MOM <= TSF', 'MACDSIGNAL <= PrevIndVol', 'PrevIndLow >= TSF', 'STDDEV >= ROC', 'MACDSIGNAL >= PrevHigh', '20MA >= PrevIndOpen', 'MidBB <= LINEARREG', 'MACDHIST >= MACDSIGNAL', 'SAR >= PrevIndLow', 'BETA >= LINEARREG_SLOPE', 'PrevIndVol <= 9MA', 'LowerBB >= ROC', 'MACD <= PrevIndVol', 'LINEARREG >= RSI', 'PrevLow >= PrevOpen', 'PrevIndLow >= ATR', 'PrevIndClose >= 9MA', 'LINEARREG_ANGLE <= 9MA', 'UpperBB <= VAR', '20MA <= MACDHIST', 'PrevIndLow >= PrevIndVol', 'PrevLow >= LINEARREG', 'PrevVol >= PrevAdjClose', 'STDDEV <= PrevLow', 'MidBB <= VAR', 'MACDSIGNAL <= TSF', 'PrevIndOpen <= PrevVol', 'BETA <= PrevVol', '9MA >= ATR', 'PrevIndClose <= PrevIndLow', 'TSF >= LINEARREG', 'VAR >= PrevIndLow', 'MACD >= LINEARREG_ANGLE', 'SAR <= PrevVol', 'PrevIndVol <= Open', 'MACDSIGNAL <= LINEARREG', 'STDDEV >= VAR', 'PrevIndLow >= MACDHIST', 'LINEARREG >= LINEARREG_SLOPE', 'LowerBB <= PrevHigh', 'UpperBB <= PrevIndVol', 'PrevIndOpen <= PrevOpen', 'Open >= LINEARREG', 'PrevIndHigh >= PrevIndClose', 'PrevIndOpen >= MOM', 'PrevIndClose <= LINEARREG', 'PrevIndOpen >= VAR', 'LINEARREG_ANGLE >= LINEARREG_SLOPE', 'PrevIndLow <= MidBB', 'Open >= PrevVol', '20MA <= PrevAdjClose', 'PrevIndOpen <= MACDHIST', 'VAR >= LINEARREG', 'PrevIndClose <= SAR', 'LINEARREG_ANGLE <= VAR', 'ATR <= STDDEV', 'PrevVol >= ADX', '9MA <= LINEARREG', 'MidBB <= PrevIndOpen', 'PrevVol >= 20MA', 'MACDHIST >= TSF', 'PrevIndHigh <= LINEARREG', 'ATR <= ROC', 'ROC >= UpperBB', 'PrevLow >= MidBB', 'PrevLow <= MOM', '20MA <= PrevVol', 'BETA <= LowerBB', '20MA <= 9MA', 'PrevAdjClose >= 9MA', 'MidBB >= VAR', 'RSI <= BETA', 'VAR <= MidBB', 'PrevHigh <= ROC', 'PrevHigh >= LowerBB', 'PrevOpen <= MACDHIST', 'LINEARREG <= MOM', '20MA <= TSF', 'RSI >= Open', 'PrevVol <= ATR', 'RSI >= MACD', 'PrevIndClose >= LINEARREG_ANGLE', 'MACD >= PrevIndOpen', 'UpperBB <= RSI', 'PrevIndClose <= TSF', 'PrevVol <= MOM', 'PrevIndOpen >= LINEARREG', 'LowerBB >= MACD', 'LINEARREG >= 20MA', 'UpperBB >= PrevAdjClose', 'VAR >= MOM', 'PrevVol >= BETA', 'STDDEV <= LowerBB', 'LINEARREG >= TSF', 'STDDEV >= MACDSIGNAL', 'LINEARREG >= ROC', 'PrevIndLow', 'PrevOpen <= MACDSIGNAL', 'LINEARREG <= UpperBB', 'ATR >= UpperBB', 'PrevHigh >= MACDHIST', 'PrevIndVol >= ROC', 'PrevIndClose >= PrevIndHigh', 'PrevAdjClose >= Open', 'LowerBB <= PrevIndHigh', 'LINEARREG_SLOPE >= VAR', 'ATR >= ADX', 'PrevIndOpen <= TSF', 'MOM <= PrevIndOpen', 'TSF >= PrevIndClose', 'PrevHigh <= UpperBB', 'PrevOpen >= PrevIndOpen', 'VAR <= LINEARREG', 'MACDSIGNAL <= ADX', 'PrevIndVol <= TSF', 'TSF <= 20MA', 'MACDSIGNAL >= STDDEV', 'LINEARREG_ANGLE <= BETA', 'STDDEV >= 20MA', 'PrevLow <= PrevIndVol', 'PrevIndHigh >= Open', 'PrevOpen >= VAR', 'STDDEV >= TSF', 'SAR <= BETA', 'LINEARREG >= MACDSIGNAL', 'LINEARREG <= TSF', 'RSI <= PrevIndLow', 'PrevIndOpen <= 20MA', 'TSF >= RSI', 'PrevVol >= LINEARREG_ANGLE', 'MACD <= 20MA', 'MACD <= MACDHIST', 'PrevOpen <= LINEARREG_SLOPE', 'PrevAdjClose >= LowerBB', '20MA <= LINEARREG_SLOPE', 'MACD >= BETA', 'PrevHigh >= PrevIndClose', 'RSI >= ATR', 'SAR <= TSF', 'MidBB <= PrevIndLow', 'PrevIndVol <= PrevIndOpen', 'PrevIndOpen >= LINEARREG_ANGLE', 'ROC >= PrevVol', 'PrevOpen >= Open', 'TSF >= PrevAdjClose', 'STDDEV <= PrevIndLow', 'PrevIndClose <= Open', 'LINEARREG_ANGLE <= TSF', '9MA <= LINEARREG_SLOPE', 'MidBB >= ATR', 'UpperBB <= MACD', 'ADX >= PrevIndHigh', 'LINEARREG_ANGLE >= PrevIndLow', 'PrevLow >= LowerBB', 'PrevLow >= MACDHIST', 'PrevIndOpen <= LowerBB', 'ADX >= PrevLow', 'PrevIndOpen >= STDDEV', 'MOM <= MACDHIST', 'STDDEV <= MOM', 'LINEARREG_ANGLE <= MOM']\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['missing', 'single_unique', 'collinear', 'zero_importance', 'low_importance'] methods have been run\n",
      "\n",
      "Removed 1925 features.\n",
      "            PrevOpen   PrevVol  UpperBB  LowerBB   SAR   ADX  MACD  \\\n",
      "Date                                                                 \n",
      "2012-01-06     14.30 55,400.00    15.03    11.45 12.97 43.68  0.42   \n",
      "2012-01-09     15.00 97,500.00    15.17    11.53 13.17 44.32  0.45   \n",
      "2012-01-10     14.61 75,200.00    15.20    11.78 13.39 44.90  0.47   \n",
      "2012-01-11     14.51 65,500.00    15.26    12.03 13.58 45.76  0.50   \n",
      "2012-01-12     14.76 22,500.00    15.34    12.09 13.75 46.21  0.50   \n",
      "\n",
      "            MACDSIGNAL  MACDHIST  MOM    ...      BETA <= LINEARREG_SLOPE  \\\n",
      "Date                                     ...                                \n",
      "2012-01-06        0.27      0.15 1.68    ...                         True   \n",
      "2012-01-09        0.31      0.15 0.76    ...                         True   \n",
      "2012-01-10        0.34      0.13 0.42    ...                         True   \n",
      "2012-01-11        0.37      0.13 0.79    ...                         True   \n",
      "2012-01-12        0.40      0.10 0.35    ...                        False   \n",
      "\n",
      "            Open <= 20MA  Open <= PrevHigh  PrevAdjClose <= LINEARREG  \\\n",
      "Date                                                                    \n",
      "2012-01-06         False             False                       True   \n",
      "2012-01-09         False              True                       True   \n",
      "2012-01-10         False              True                       True   \n",
      "2012-01-11         False              True                      False   \n",
      "2012-01-12         False              True                       True   \n",
      "\n",
      "            PrevIndHigh <= PrevIndOpen  PrevHigh <= Open  PrevLow <= Open  \\\n",
      "Date                                                                        \n",
      "2012-01-06                       False              True             True   \n",
      "2012-01-09                       False             False             True   \n",
      "2012-01-10                       False             False             True   \n",
      "2012-01-11                       False             False             True   \n",
      "2012-01-12                       False             False             True   \n",
      "\n",
      "            Open <= PrevLow  Open  Adj Close  \n",
      "Date                                          \n",
      "2012-01-06            False 15.00      14.52  \n",
      "2012-01-09            False 14.61      14.50  \n",
      "2012-01-10            False 14.51      14.81  \n",
      "2012-01-11            False 14.76      14.48  \n",
      "2012-01-12            False 14.47      14.50  \n",
      "\n",
      "[5 rows x 93 columns]\n",
      "Our wonderful, new dataset has been created!\n"
     ]
    }
   ],
   "source": [
    "# Let's go!\n",
    "#df = df.set_index('Date')\n",
    "print(df.head())\n",
    "\n",
    "def feature_removal_func(df):\n",
    "# Name all the feature columns and target label column for FeatureSelector...\n",
    "# Save the open price as FS removes it due to high correlation, but we want to keep it    \n",
    "    openprice = df[\"Open\"]\n",
    "    features = df.drop(\"Adj Close\", axis=1)\n",
    "    label = df[\"Adj Close\"]\n",
    "\n",
    "# Now, drop all columns of low importance, crazy high correlation, etc.\n",
    "    from feature_selector import FeatureSelector\n",
    "    fs = FeatureSelector(data = features, labels = label)\n",
    "    fs.identify_all(selection_params = {'missing_threshold': 0.6,    \n",
    "                                    'correlation_threshold': 0.98, \n",
    "                                    'task': 'regression',    \n",
    "                                    'eval_metric': 'mse', \n",
    "                                    'cumulative_importance': 0.99})\n",
    "    all_to_remove = fs.check_removal()\n",
    "    print(all_to_remove[:])\n",
    "    df = fs.remove(methods = 'all')\n",
    "\n",
    "# Re-Add the Open. And Adj Close to the df because FeatureTools removes it once you assign it as the label for some reason\n",
    "    df['Open'] = openprice\n",
    "    df['Adj Close'] = label\n",
    "    return(df)\n",
    "\n",
    "df = feature_removal_func(df)\n",
    "\n",
    "print(df.head())\n",
    "print('Our wonderful, new dataset has been created!')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Wonderful! Well done!\n",
    "# Now, let's do some quick housekeeping on our new dataset.\n",
    "# Move the Open and Adj Close columns to the end again\n",
    "opencolumn = df['Open']\n",
    "adjclosecolumn = df['Adj Close']\n",
    "\n",
    "df = df.drop(['Open'],axis=1)\n",
    "df = df.drop(['Adj Close'],axis=1)\n",
    "\n",
    "df['Open'] = opencolumn\n",
    "df['Adj Close'] = adjclosecolumn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Index(['PrevOpen', 'PrevVol', 'UpperBB', 'LowerBB', 'SAR', 'ADX', 'MACD',\n",
      "       'MACDSIGNAL', 'MACDHIST', 'MOM', 'ROC', 'RSI', 'ATR', 'BETA',\n",
      "       'LINEARREG_ANGLE', 'STDDEV', 'PrevIndOpen', 'PrevIndVol',\n",
      "       'MACDSIGNAL <= LINEARREG_SLOPE', 'PrevOpen <= 9MA', 'SAR <= 20MA',\n",
      "       'BETA <= MACD', 'MACDHIST <= BETA', 'MOM <= MACD', 'BETA <= MACDSIGNAL',\n",
      "       'LINEARREG <= PrevHigh', '9MA <= PrevLow', 'PrevOpen <= SAR',\n",
      "       'PrevIndLow <= SAR', 'PrevIndLow <= PrevIndClose', 'SAR <= RSI',\n",
      "       'MidBB <= PrevLow', 'PrevAdjClose <= TSF', 'TSF <= Open',\n",
      "       'MidBB <= 9MA', 'MACD <= VAR', 'MOM <= BETA', 'PrevOpen <= Open',\n",
      "       'TSF <= PrevLow', 'STDDEV <= MACDSIGNAL', 'VAR <= MOM',\n",
      "       'PrevHigh <= PrevOpen', 'LowerBB <= SAR', 'PrevIndOpen <= PrevIndClose',\n",
      "       'LINEARREG_SLOPE <= MACD', 'PrevOpen <= TSF', 'PrevOpen <= MidBB',\n",
      "       'PrevOpen <= PrevIndLow', '9MA <= PrevHigh', 'ATR <= MOM',\n",
      "       'LowerBB <= PrevAdjClose', 'LINEARREG_ANGLE <= ROC',\n",
      "       'PrevIndLow <= LINEARREG', 'BETA <= VAR', 'LowerBB <= Open',\n",
      "       'LINEARREG_SLOPE <= MACDHIST', 'Open <= UpperBB', 'MACDSIGNAL <= ROC',\n",
      "       'TSF <= PrevHigh', 'SAR <= MidBB', 'MACDSIGNAL <= MOM',\n",
      "       'PrevAdjClose <= 9MA', 'PrevOpen <= PrevAdjClose', 'MOM <= STDDEV',\n",
      "       'TSF <= PrevIndLow', 'MACDSIGNAL <= MACDHIST', 'UpperBB <= PrevHigh',\n",
      "       'ATR <= VAR', 'ADX <= SAR', '9MA <= 20MA', 'VAR <= MACDSIGNAL',\n",
      "       'PrevOpen <= PrevLow', 'Open <= 9MA', 'UpperBB <= PrevAdjClose',\n",
      "       'PrevLow <= LINEARREG', 'ADX <= ROC', 'LowerBB <= PrevLow',\n",
      "       'LINEARREG <= Open', 'RSI <= LowerBB', 'PrevAdjClose <= Open',\n",
      "       '20MA <= MidBB', 'Open <= PrevAdjClose', 'LINEARREG <= PrevOpen',\n",
      "       'BETA <= LINEARREG_SLOPE', 'Open <= 20MA', 'Open <= PrevHigh',\n",
      "       'PrevAdjClose <= LINEARREG', 'PrevIndHigh <= PrevIndOpen',\n",
      "       'PrevHigh <= Open', 'PrevLow <= Open', 'Open <= PrevLow', 'Open',\n",
      "       'Adj Close'],\n",
      "      dtype='object')\n"
     ]
    }
   ],
   "source": [
    "# Check it to make sure it worked\n",
    "print(df.columns)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[INFO] New Feature types:\n",
      "PrevOpen                         float64\n",
      "PrevVol                          float64\n",
      "UpperBB                          float64\n",
      "LowerBB                          float64\n",
      "SAR                              float64\n",
      "ADX                              float64\n",
      "MACD                             float64\n",
      "MACDSIGNAL                       float64\n",
      "MACDHIST                         float64\n",
      "MOM                              float64\n",
      "ROC                              float64\n",
      "RSI                              float64\n",
      "ATR                              float64\n",
      "BETA                             float64\n",
      "LINEARREG_ANGLE                  float64\n",
      "STDDEV                           float64\n",
      "PrevIndOpen                      float64\n",
      "PrevIndVol                       float64\n",
      "MACDSIGNAL <= LINEARREG_SLOPE      int64\n",
      "PrevOpen <= 9MA                    int64\n",
      "SAR <= 20MA                        int64\n",
      "BETA <= MACD                       int64\n",
      "MACDHIST <= BETA                   int64\n",
      "MOM <= MACD                        int64\n",
      "BETA <= MACDSIGNAL                 int64\n",
      "LINEARREG <= PrevHigh              int64\n",
      "9MA <= PrevLow                     int64\n",
      "PrevOpen <= SAR                    int64\n",
      "PrevIndLow <= SAR                  int64\n",
      "PrevIndLow <= PrevIndClose         int64\n",
      "                                  ...   \n",
      "MOM <= STDDEV                      int64\n",
      "TSF <= PrevIndLow                  int64\n",
      "MACDSIGNAL <= MACDHIST             int64\n",
      "UpperBB <= PrevHigh                int64\n",
      "ATR <= VAR                         int64\n",
      "ADX <= SAR                         int64\n",
      "9MA <= 20MA                        int64\n",
      "VAR <= MACDSIGNAL                  int64\n",
      "PrevOpen <= PrevLow                int64\n",
      "Open <= 9MA                        int64\n",
      "UpperBB <= PrevAdjClose            int64\n",
      "PrevLow <= LINEARREG               int64\n",
      "ADX <= ROC                         int64\n",
      "LowerBB <= PrevLow                 int64\n",
      "LINEARREG <= Open                  int64\n",
      "RSI <= LowerBB                     int64\n",
      "PrevAdjClose <= Open               int64\n",
      "20MA <= MidBB                      int64\n",
      "Open <= PrevAdjClose               int64\n",
      "LINEARREG <= PrevOpen              int64\n",
      "BETA <= LINEARREG_SLOPE            int64\n",
      "Open <= 20MA                       int64\n",
      "Open <= PrevHigh                   int64\n",
      "PrevAdjClose <= LINEARREG          int64\n",
      "PrevIndHigh <= PrevIndOpen         int64\n",
      "PrevHigh <= Open                   int64\n",
      "PrevLow <= Open                    int64\n",
      "Open <= PrevLow                    int64\n",
      "Open                             float64\n",
      "Adj Close                        float64\n",
      "Length: 93, dtype: object\n"
     ]
    }
   ],
   "source": [
    "# Now check the new types of the features, because you'll see now\n",
    "# that we have True and False boolean values, that we'll turn into numerical\n",
    "# values of 1 and 0 below, because machine learners like numbers not words.\n",
    "# Convert boolean True False datapoints to 1 and 0\n",
    "df = df.applymap(lambda x: 1 if x == True else x)\n",
    "df = df.applymap(lambda x: 0 if x == False else x)\n",
    "\n",
    "# Now check to make sure our data is all numeric int's and float's.\n",
    "print('[INFO] New Feature types:')\n",
    "print(df.dtypes)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "            PrevOpen   PrevVol  UpperBB  LowerBB   SAR   ADX  MACD  \\\n",
      "Date                                                                 \n",
      "2012-01-06     14.30 55,400.00    15.03    11.45 12.97 43.68  0.42   \n",
      "2012-01-09     15.00 97,500.00    15.17    11.53 13.17 44.32  0.45   \n",
      "2012-01-10     14.61 75,200.00    15.20    11.78 13.39 44.90  0.47   \n",
      "2012-01-11     14.51 65,500.00    15.26    12.03 13.58 45.76  0.50   \n",
      "2012-01-12     14.76 22,500.00    15.34    12.09 13.75 46.21  0.50   \n",
      "\n",
      "            MACDSIGNAL  MACDHIST  MOM    ...      BETA <= LINEARREG_SLOPE  \\\n",
      "Date                                     ...                                \n",
      "2012-01-06        0.27      0.15 1.68    ...                            1   \n",
      "2012-01-09        0.31      0.15 0.76    ...                            1   \n",
      "2012-01-10        0.34      0.13 0.42    ...                            1   \n",
      "2012-01-11        0.37      0.13 0.79    ...                            1   \n",
      "2012-01-12        0.40      0.10 0.35    ...                            0   \n",
      "\n",
      "            Open <= 20MA  Open <= PrevHigh  PrevAdjClose <= LINEARREG  \\\n",
      "Date                                                                    \n",
      "2012-01-06             0                 0                          1   \n",
      "2012-01-09             0                 1                          1   \n",
      "2012-01-10             0                 1                          1   \n",
      "2012-01-11             0                 1                          0   \n",
      "2012-01-12             0                 1                          1   \n",
      "\n",
      "            PrevIndHigh <= PrevIndOpen  PrevHigh <= Open  PrevLow <= Open  \\\n",
      "Date                                                                        \n",
      "2012-01-06                           0                 1                1   \n",
      "2012-01-09                           0                 0                1   \n",
      "2012-01-10                           0                 0                1   \n",
      "2012-01-11                           0                 0                1   \n",
      "2012-01-12                           0                 0                1   \n",
      "\n",
      "            Open <= PrevLow  Open  Adj Close  \n",
      "Date                                          \n",
      "2012-01-06                0 15.00      14.52  \n",
      "2012-01-09                0 14.61      14.50  \n",
      "2012-01-10                0 14.51      14.81  \n",
      "2012-01-11                0 14.76      14.48  \n",
      "2012-01-12                0 14.47      14.50  \n",
      "\n",
      "[5 rows x 93 columns]\n",
      "Done!\n"
     ]
    }
   ],
   "source": [
    "# Let's see the dataframe head again.\n",
    "print(df.head())\n",
    "print('Done!')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(1914, 93)\n"
     ]
    }
   ],
   "source": [
    "# I inserted this cell here again to deal with more nan values that\n",
    "# show up after our feature engineering step. I'm going to print what my\n",
    "# dataframe shape looks like before and after dropping the nan values.\n",
    "# If the change is insignifcant, we'll carry on, otherwise I may look at\n",
    "# replacing them with the mean of each column.\n",
    "print(df.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(1914, 93)\n"
     ]
    }
   ],
   "source": [
    "# Now drop em\n",
    "df = df.dropna(how='any')\n",
    "print(df.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "PrevOpen                         False\n",
      "PrevVol                          False\n",
      "UpperBB                          False\n",
      "LowerBB                          False\n",
      "SAR                              False\n",
      "ADX                              False\n",
      "MACD                             False\n",
      "MACDSIGNAL                       False\n",
      "MACDHIST                         False\n",
      "MOM                              False\n",
      "ROC                              False\n",
      "RSI                              False\n",
      "ATR                              False\n",
      "BETA                             False\n",
      "LINEARREG_ANGLE                  False\n",
      "STDDEV                           False\n",
      "PrevIndOpen                      False\n",
      "PrevIndVol                       False\n",
      "MACDSIGNAL <= LINEARREG_SLOPE    False\n",
      "PrevOpen <= 9MA                  False\n",
      "SAR <= 20MA                      False\n",
      "BETA <= MACD                     False\n",
      "MACDHIST <= BETA                 False\n",
      "MOM <= MACD                      False\n",
      "BETA <= MACDSIGNAL               False\n",
      "LINEARREG <= PrevHigh            False\n",
      "9MA <= PrevLow                   False\n",
      "PrevOpen <= SAR                  False\n",
      "PrevIndLow <= SAR                False\n",
      "PrevIndLow <= PrevIndClose       False\n",
      "                                 ...  \n",
      "MOM <= STDDEV                    False\n",
      "TSF <= PrevIndLow                False\n",
      "MACDSIGNAL <= MACDHIST           False\n",
      "UpperBB <= PrevHigh              False\n",
      "ATR <= VAR                       False\n",
      "ADX <= SAR                       False\n",
      "9MA <= 20MA                      False\n",
      "VAR <= MACDSIGNAL                False\n",
      "PrevOpen <= PrevLow              False\n",
      "Open <= 9MA                      False\n",
      "UpperBB <= PrevAdjClose          False\n",
      "PrevLow <= LINEARREG             False\n",
      "ADX <= ROC                       False\n",
      "LowerBB <= PrevLow               False\n",
      "LINEARREG <= Open                False\n",
      "RSI <= LowerBB                   False\n",
      "PrevAdjClose <= Open             False\n",
      "20MA <= MidBB                    False\n",
      "Open <= PrevAdjClose             False\n",
      "LINEARREG <= PrevOpen            False\n",
      "BETA <= LINEARREG_SLOPE          False\n",
      "Open <= 20MA                     False\n",
      "Open <= PrevHigh                 False\n",
      "PrevAdjClose <= LINEARREG        False\n",
      "PrevIndHigh <= PrevIndOpen       False\n",
      "PrevHigh <= Open                 False\n",
      "PrevLow <= Open                  False\n",
      "Open <= PrevLow                  False\n",
      "Open                             False\n",
      "Adj Close                        False\n",
      "Length: 93, dtype: bool\n"
     ]
    }
   ],
   "source": [
    "# Now confirm if we have any nan values. All Falses? Great!\n",
    "print(pd.isnull(df).any())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "# This section has been added in to remove certain characters that XGBoost\n",
    "# doesn't recognize in the Feature/Column names or else we are going to get \n",
    "# an error after training occurs:\n",
    "# ValueError: feature_names may not contain [, ] or <\n",
    "import re\n",
    "regex = re.compile(r\"\\[|\\]|<\", re.IGNORECASE)\n",
    "df.columns = [regex.sub(\"_\", col) if any(x in str(col) for x in set(('[', ']'))) else col for col in df.columns.values]\n",
    "df.columns = [regex.sub(\"LESSTHAN\", col) if any(x in str(col) for x in set(('<'))) else col for col in df.columns.values]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 7. SPLIT THE DATASET FOR TRAINING/TESTING AND PREDICTIONS"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "# We're getting close to training our model! Congrats for\n",
    "# hanging on this far.\n",
    "\n",
    "# First, we're going to extract a section of recent data from our dataset\n",
    "# to be used as a \"prediction\" dataset after we've trained our model.\n",
    "# This dataset will be used to see how well our model predicts on\n",
    "# new data that is hasn't yet seen before/wasn't involved in the\n",
    "# training process.\n",
    "prediction_df = df.tail(90).copy() # Make sure you make it a copy, else errors arise later. default 90 days\n",
    "df = df.iloc[:-90,:].copy() # subtracting those 90 rows/days from the soon to be training dataset so that\n",
    "                            # we're not training on the prediction data as well."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Shape of features:  (1824, 92)\n"
     ]
    }
   ],
   "source": [
    "# Split the dataset into features (X) and a target (Y)\n",
    "# and print the shapes of them\n",
    "X = df.drop('Adj Close', axis=1) # Everything but what we're trying to predict\n",
    "Y = df['Adj Close'] # This is what we're trying to predict\n",
    "\n",
    "print('Shape of features: ', X.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Shape of target:  (1824,)\n"
     ]
    }
   ],
   "source": [
    "print('Shape of target: ', Y.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Standardize the data. Commenting this out until I can figure out how to\n",
    "# unscale the prediction dataset for review later. This helps with model\n",
    "# training performance and other reasons, but be careful because if you standardize a training\n",
    "# dataset, then go to use the model on a new prediction dataset, if that \n",
    "# new dataset has values greater or less than your standarized scale,\n",
    "# the model might not be as accurate. Maybe scale to a MinMax of having\n",
    "# a 10-20% buffer above/below???\n",
    "#from sklearn.preprocessing import StandardScaler, MinMaxScaler\n",
    "#scaler = MinMaxScaler().fit(X)\n",
    "#scaled_X = scaler.transform(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Split dataset into training and testing/validation data. Think of this as,\n",
    "# the model learns from the testing data, then tests to see how well it did\n",
    "# (internally) on the testing data. Much like what we're going to do with the\n",
    "# model on our prediction dataset later.\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "seed = 9 # For reproducibility of the data in future steps\n",
    "test_size = 0.25 # 75% training, 25% testing\n",
    "\n",
    "# This also randomly shuffles the data. We're going to disable this because we \n",
    "# are dealing with a time series problem. We don't want our model to just\n",
    "# recognize an overall uptrend in our data though, so we're going to employ\n",
    "# cross validation in a minute to combat this.\n",
    "X_train, X_test, Y_train, Y_test = train_test_split(X, Y, test_size=test_size, random_state=seed, shuffle=False)\n",
    "\n",
    "# Now, becuase we're using many different types of regression algorithms in\n",
    "# TPOT, we need to make sure that every one of them can understand our data\n",
    "# in the same way, so long story short, convert the training features, and \n",
    "# testing features to Numpy arrays, but leave the Y's as a DF. It's a long\n",
    "# story and I had many errors to deal with in figuring this portion out.\n",
    "# See https://github.com/EpistasisLab/tpot/issues/738\n",
    "# or https://stackoverflow.com/questions/57488274/feature-names-mismatch-when-passing-x-test-to-predict-function-again-still\n",
    "# ...if you want.\n",
    "X_train=X_train.values\n",
    "X_test=X_test.values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[INFO] Printing the shapes of the training/testing feature/label sets...\n",
      "(1368, 92)\n",
      "(456, 92)\n",
      "(1368,)\n",
      "(456,)\n"
     ]
    }
   ],
   "source": [
    "print('[INFO] Printing the shapes of the training/testing feature/label sets...')\n",
    "print(X_train.shape)\n",
    "print(X_test.shape)\n",
    "print(Y_train.shape)\n",
    "print(Y_test.shape)\n",
    "\n",
    "\n",
    "# I know it's tedious to keep printing the data and shapes like this, but the\n",
    "# further along you get in your coding, the more important it becomes to know\n",
    "# exactly what type of data is being passed from step to step to help mitigate\n",
    "# and pinpoint where you might run into any errors."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 8. TRAIN A MODEL USING TPOT"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The time is finally here! Time to train a regression model. Think...\n",
    "\n",
    "regression = predicting a number\n",
    "\n",
    "classification = predicting a category\n",
    "\n",
    "Since we're predicting the daily closing price, we need regression.\n",
    "TPOT is a module that automates the process of finding the best\n",
    "machine learning model (or stack of models) for your dataset, with \n",
    "the best hyperparameter(s) for each one, a process that would \n",
    "normally need to be done manually. This automated machine learning\n",
    "is a no-brainer. Let it do the magic for you so you don't have to.\n",
    "Notice how I like the idea of automation, like feature engineering,\n",
    "selection, regression analysis, etc.? It just makes sense, even if\n",
    "it's a little more computationally taxing (although we try to do it \n",
    "as efficiently as possible).\n",
    "\n",
    "The way I describe this part is, you have some specific music (data), and\n",
    "you're trying to pick the best sound system (regression pipeline)\n",
    "to get the best sound (prediction). Each of the amplifiers (algorithms)\n",
    "have a set of dials (hyperparameters) that need adjusting and tweaking\n",
    "to get right. And maybe stacking multiple amplifiers would make the music sound\n",
    "even better! So what TPOT does is it trains a model to get the best possible\n",
    "result (i.e. lowest loss function) by trying many, MANY different combinations\n",
    "of amplifiers and dial settings to end up with the best possible sound system.\n",
    "Make sense? Let's do it!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\windowshopr\\AppData\\Local\\Programs\\Python\\Python36\\lib\\site-packages\\deap\\tools\\_hypervolume\\pyhv.py:33: ImportWarning: Falling back to the python version of hypervolume module. Expect this to be very slow.\n",
      "  \"module. Expect this to be very slow.\", ImportWarning)\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, description='Optimization Progress', max=440, style=ProgressStyle(descript…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Generation 1 - Current best internal CV score: -5.617574847627383\n",
      "Generation 2 - Current best internal CV score: -5.392925650474486\n",
      "Generation 3 - Current best internal CV score: -5.392925650474486\n",
      "Generation 4 - Current best internal CV score: -5.392925650474486\n",
      "Generation 5 - Current best internal CV score: -5.392925650474486\n",
      "Generation 6 - Current best internal CV score: -5.260649779184337\n",
      "Generation 7 - Current best internal CV score: -5.224619294199957\n",
      "Generation 8 - Current best internal CV score: -5.217801746185675\n",
      "Generation 9 - Current best internal CV score: -5.217801746185675\n",
      "Generation 10 - Current best internal CV score: -5.215785947649995\n",
      "\n",
      "Best pipeline: RandomForestRegressor(SelectFwe(SelectPercentile(input_matrix, percentile=40), alpha=0.029), bootstrap=True, max_features=0.9000000000000001, min_samples_leaf=8, min_samples_split=15, n_estimators=100)\n"
     ]
    }
   ],
   "source": [
    "from sklearn.metrics import mean_squared_error, r2_score, explained_variance_score, max_error, mean_absolute_error, mean_squared_log_error, median_absolute_error\n",
    "from tpot import TPOTRegressor\n",
    "\n",
    "# I made a custom config dictionary for TPOT to use.\n",
    "# I've made this list full of Regressors that use the\n",
    "# .feature_importances_ attribute of sklearn to review\n",
    "# what the feature imporances are once the model is run.\n",
    "\n",
    "# This is the list of regressors and pre-processors\n",
    "# that I've used. You can add or subtract to this list,\n",
    "# but you may end up running into errors when it comes to displaying\n",
    "# the feature importances part that you'll have to deal with.\n",
    "# Read here for more: https://epistasislab.github.io/tpot/using/#customizing-tpots-operators-and-parameters\n",
    "\n",
    "# Define the protecting function.\n",
    "def train_function(X_train, Y_train):\n",
    "    warnings.simplefilter(action='once', category=FutureWarning)\n",
    "    warnings.simplefilter(action='once', category=UserWarning)\n",
    "    warnings.simplefilter(action='once', category=DeprecationWarning)\n",
    "    warnings.simplefilter(action='once', category=ImportWarning)\n",
    "    \n",
    "    tpot_config = {\n",
    "\n",
    "    'xgboost.XGBRegressor': {\n",
    "        'n_estimators': [100],\n",
    "        'max_depth': range(1, 11),\n",
    "        'learning_rate': [1e-3, 1e-2, 1e-1, 0.5, 1.],\n",
    "        'subsample': np.arange(0.05, 1.01, 0.05),\n",
    "        'min_child_weight': range(1, 21),\n",
    "        'nthread': [1],\n",
    "        'objective': ['reg:squarederror']\n",
    "    },\n",
    "        \n",
    "\n",
    "    'sklearn.ensemble.ExtraTreesRegressor': {\n",
    "        'n_estimators': [100],\n",
    "        'max_features': np.arange(0.05, 1.01, 0.05),\n",
    "        'min_samples_split': range(2, 21),\n",
    "        'min_samples_leaf': range(1, 21),\n",
    "        'bootstrap': [True, False]\n",
    "    },\n",
    "\n",
    "    'sklearn.ensemble.GradientBoostingRegressor': {\n",
    "        'n_estimators': [100],\n",
    "        'loss': [\"ls\", \"lad\", \"huber\", \"quantile\"],\n",
    "        'learning_rate': [1e-3, 1e-2, 1e-1, 0.5, 1.],\n",
    "        'max_depth': range(1, 11),\n",
    "        'min_samples_split': range(2, 21),\n",
    "        'min_samples_leaf': range(1, 21),\n",
    "        'subsample': np.arange(0.05, 1.01, 0.05),\n",
    "        'max_features': np.arange(0.05, 1.01, 0.05),\n",
    "        'alpha': [0.75, 0.8, 0.85, 0.9, 0.95, 0.99]\n",
    "    },\n",
    "\n",
    "    'sklearn.ensemble.AdaBoostRegressor': {\n",
    "        'n_estimators': [100],\n",
    "        'learning_rate': [1e-3, 1e-2, 1e-1, 0.5, 1.],\n",
    "        'loss': [\"linear\", \"square\", \"exponential\"]\n",
    "    },\n",
    "\n",
    "    'sklearn.tree.DecisionTreeRegressor': {\n",
    "        'max_depth': range(1, 11),\n",
    "        'min_samples_split': range(2, 21),\n",
    "        'min_samples_leaf': range(1, 21)\n",
    "    },\n",
    "\n",
    "    'sklearn.ensemble.RandomForestRegressor': {\n",
    "        'n_estimators': [100],\n",
    "        'max_features': np.arange(0.05, 1.01, 0.05),\n",
    "        'min_samples_split': range(2, 21),\n",
    "        'min_samples_leaf': range(1, 21),\n",
    "        'bootstrap': [True, False]\n",
    "    },\n",
    "\n",
    "\n",
    "    # Preprocesssors\n",
    "    'sklearn.preprocessing.Binarizer': {\n",
    "        'threshold': np.arange(0.0, 1.01, 0.05)\n",
    "    },\n",
    "\n",
    "    'sklearn.decomposition.FastICA': {\n",
    "        'tol': np.arange(0.0, 1.01, 0.05)\n",
    "    },\n",
    "\n",
    "    'sklearn.cluster.FeatureAgglomeration': {\n",
    "        'linkage': ['ward', 'complete', 'average'],\n",
    "        'affinity': ['euclidean', 'l1', 'l2', 'manhattan', 'cosine']\n",
    "    },\n",
    "\n",
    "    'sklearn.preprocessing.MaxAbsScaler': {\n",
    "    },\n",
    "\n",
    "    'sklearn.preprocessing.MinMaxScaler': {\n",
    "    },\n",
    "\n",
    "    'sklearn.preprocessing.Normalizer': {\n",
    "        'norm': ['l1', 'l2', 'max']\n",
    "    },\n",
    "\n",
    "    'sklearn.kernel_approximation.Nystroem': {\n",
    "        'kernel': ['rbf', 'cosine', 'chi2', 'laplacian', 'polynomial', 'poly', 'linear', 'additive_chi2', 'sigmoid'],\n",
    "        'gamma': np.arange(0.0, 1.01, 0.05),\n",
    "        'n_components': range(1, 11)\n",
    "    },\n",
    "\n",
    "    'sklearn.decomposition.PCA': {\n",
    "        'svd_solver': ['randomized'],\n",
    "        'iterated_power': range(1, 11)\n",
    "    },\n",
    "\n",
    "    'sklearn.preprocessing.PolynomialFeatures': {\n",
    "        'degree': [2],\n",
    "        'include_bias': [False],\n",
    "        'interaction_only': [False]\n",
    "    },\n",
    "\n",
    "    'sklearn.kernel_approximation.RBFSampler': {\n",
    "        'gamma': np.arange(0.0, 1.01, 0.05)\n",
    "    },\n",
    "\n",
    "    'sklearn.preprocessing.RobustScaler': {\n",
    "    },\n",
    "\n",
    "    'sklearn.preprocessing.StandardScaler': {\n",
    "    },\n",
    "\n",
    "    'tpot.builtins.ZeroCount': {\n",
    "    },\n",
    "\n",
    "    'tpot.builtins.OneHotEncoder': {\n",
    "        'minimum_fraction': [0.05, 0.1, 0.15, 0.2, 0.25],\n",
    "        'sparse': [False],\n",
    "        'threshold': [10]\n",
    "    },\n",
    "\n",
    "\n",
    "    # Selectors\n",
    "    'sklearn.feature_selection.SelectFwe': {\n",
    "        'alpha': np.arange(0, 0.05, 0.001),\n",
    "        'score_func': {\n",
    "            'sklearn.feature_selection.f_regression': None\n",
    "        }\n",
    "    },\n",
    "\n",
    "    'sklearn.feature_selection.SelectPercentile': {\n",
    "        'percentile': range(1, 100),\n",
    "        'score_func': {\n",
    "            'sklearn.feature_selection.f_regression': None\n",
    "        }\n",
    "    },\n",
    "\n",
    "    'sklearn.feature_selection.VarianceThreshold': {\n",
    "        'threshold': [0.0001, 0.0005, 0.001, 0.005, 0.01, 0.05, 0.1, 0.2]\n",
    "    },\n",
    "\n",
    "    'sklearn.feature_selection.SelectFromModel': {\n",
    "        'threshold': np.arange(0, 1.01, 0.05),\n",
    "        'estimator': {\n",
    "            'sklearn.ensemble.ExtraTreesRegressor': {\n",
    "                'n_estimators': [100],\n",
    "                'max_features': np.arange(0.05, 1.01, 0.05)\n",
    "            }\n",
    "        }\n",
    "    }\n",
    "\n",
    "}\n",
    "\n",
    "# Now comes another important aspect of the training/testing phase;\n",
    "# Cross Validation. CV is action of replacing the testing portion\n",
    "# of your training/testing dataset with a new section within that\n",
    "# same dataset. So in our example, we are using a 75/25 split.\n",
    "# During the first \"fold\" of our CV run, we train on that 75 \n",
    "# data, and test on the 25. Then in the next fold, we take a random\n",
    "# 25% data from the 75% data and swap it out with the current 25%.\n",
    "# This is important because it's a way to prevent \"over-fitting\" our\n",
    "# model to our data. If we were training a model on Amazon's stock prices,\n",
    "# without randomly shuffling the testing/training data, and without CV,\n",
    "# eventually our model would learn to just buy and hold that stock to \n",
    "# get the best result, but that doesn't mean the model has good\n",
    "# predictive capabilities because it wouldn't know what to do when it \n",
    "# starts going down. CV helps to truncate the full historical data to \n",
    "# train on various sections within the data. More here:\n",
    "\n",
    "# https://towardsdatascience.com/overfitting-vs-underfitting-a-complete-example-d05dd7e19765\n",
    "\n",
    "# TPOT does this cross validation for us already. Yay!\n",
    "\n",
    "\n",
    "# Cross Validation folds to run\n",
    "    folds   = 30 # Default is 10, I'm doing more \n",
    "\n",
    "    #earlystop = 30 # could have this feature added in if you want, I don't for now)\n",
    "\n",
    "\n",
    "# Define the TPOT Regressor.\n",
    "# It's nice to have Dask installed/running during this part. Dask utilizes\n",
    "# parallel computing, meaning dividing the same task across a number of \n",
    "# \"workers\" (i.e. computer processors) to get a job done faster!\n",
    "# If you don't want to use it, make it false, and get rid of the n_jobs parameter.\n",
    "# n_jobs = -1 means use all available workers. Can set to 1, 2, 300, depending on\n",
    "# how many processors you have. See the TPOT API for more info :D\n",
    "\n",
    "# Also, review what the generations and population sizes mean when it comes to how\n",
    "# many regressors are going to be used during the optimization process. Again, look here:\n",
    "# https://epistasislab.github.io/tpot/api/#regression\n",
    "    best_model = TPOTRegressor(use_dask=True, n_jobs=1, config_dict=tpot_config, cv=folds,\n",
    "                          generations=10, population_size=40, verbosity=2, random_state=seed)\n",
    "                          #early_stop=earlystop) #memory='./PipelineCache',       memory='auto',\n",
    "\n",
    "# Fit the TPOT regression to our training data to find the best model. Woohoo!\n",
    "    best_model.fit(X_train, Y_train)\n",
    "\n",
    "# Export the TPOT pipeline if you want to use it for anything later\n",
    "    if os.path.exists('./exported pipelines'):\n",
    "        pass\n",
    "    else:\n",
    "        os.mkdir('./exported pipelines')\n",
    "    best_model.export('./exported pipelines/' + ticker_input + '-prediction-pipeline.py')\n",
    "    \n",
    "    best_model = best_model\n",
    "    return(best_model)\n",
    "\n",
    "# Now run the function!\n",
    "best_model = train_function(X_train, Y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[INFO] TPOTs best pipeline is:\n",
      "RandomForestRegressor(bootstrap=True, criterion='mse', max_depth=None,\n",
      "                      max_features=0.9000000000000001, max_leaf_nodes=None,\n",
      "                      min_impurity_decrease=0.0, min_impurity_split=None,\n",
      "                      min_samples_leaf=8, min_samples_split=15,\n",
      "                      min_weight_fraction_leaf=0.0, n_estimators=100,\n",
      "                      n_jobs=None, oob_score=False, random_state=None,\n",
      "                      verbose=0, warm_start=False)\n",
      "We are done training the model! You can view the feature importances plot in your directory now.\n",
      "It is now time to predict the values in the testing/validation set and see how well it did internally...\n"
     ]
    }
   ],
   "source": [
    "print('[INFO] TPOTs best pipeline is:')\n",
    "print(best_model.fitted_pipeline_.steps[-1][1])\n",
    "print('We are done training the model! You can view the feature importances plot in your directory now.')\n",
    "print('It is now time to predict the values in the testing/validation set and see how well it did internally...')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[INFO] Printing the shape of X_test for debugging the predict function...\n",
      "(456, 92)\n"
     ]
    }
   ],
   "source": [
    "print('[INFO] Printing the shape of X_test for debugging the predict function...')\n",
    "#print(X_test[:,:])\n",
    "print(X_test.shape)\n",
    "\n",
    "# Now let's make sure X_test columns are in the same layout as X_train...\n",
    "#X_test = X_test[X_train.columns]\n",
    "#print(X_test.shape) # for debugging"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 9. TEST THE MODEL ON THE TESTING/VALIDATION DATASET"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "=============================\n",
      "TPOT's final score on testing dataset is :  -3.5369446818583024\n",
      "=============================\n",
      "[INFO] MSE on test set : 3.537\n",
      "[INFO] Max Error on test set : 8.634\n",
      "[INFO] MAE on test set : 1.434\n",
      "[INFO] MSLE on test set : 0.002\n",
      "[INFO] Median AE on test set : 1.132\n",
      "---------------------------------\n",
      "[INFO] R2 Score on test set : 0.993\n",
      "[INFO] Adjusted R2 Score on test set : 0.993\n",
      "[INFO] Explained Variance Score on test set : 0.993\n",
      "--\n",
      "Done predicting the testing set. Now time to move on to our lovely prediction data\n",
      "to see how well it really performs!\n"
     ]
    }
   ],
   "source": [
    "# This is to see how well it did internally using its testing data (25%).\n",
    "# Make predictions using the tuned model and display error metrics,\n",
    "\n",
    "# Now make the predictions using our model and display the metrics\n",
    "predictions = best_model.predict(X_test)\n",
    "\n",
    "# ...but first, let's assign some variables to use for calculating the \n",
    "# Adjust R Squared value\n",
    "data_points = int(X_test.shape[0])\n",
    "feature_points = int(X_test.shape[1])\n",
    "r_squared = round(r2_score(Y_test, predictions), 3)\n",
    "\n",
    "\n",
    "print('=============================')\n",
    "print(\"TPOT's final score on testing dataset is : \", best_model.score(X_test, Y_test))\n",
    "print('=============================')\n",
    "print(\"[INFO] MSE on test set : {}\".format(round(mean_squared_error(Y_test, predictions), 3)))\n",
    "print(\"[INFO] Max Error on test set : {}\".format(round(max_error(Y_test, predictions), 3)))\n",
    "print(\"[INFO] MAE on test set : {}\".format(round(mean_absolute_error(Y_test, predictions), 3)))\n",
    "print(\"[INFO] MSLE on test set : {}\".format(round(mean_squared_log_error(Y_test, predictions), 3)))\n",
    "print(\"[INFO] Median AE on test set : {}\".format(round(median_absolute_error(Y_test, predictions), 3)))\n",
    "print('---------------------------------')\n",
    "print('[INFO] R2 Score on test set : {}'.format(r_squared))\n",
    "print('[INFO] Adjusted R2 Score on test set : {}'.format(round(1 - (1 - r_squared) * (data_points*feature_points - 1) / (data_points*feature_points - feature_points - 1), 3))) # Adjusted R Squared formula :D\n",
    "print('[INFO] Explained Variance Score on test set : {}'.format(round(explained_variance_score(Y_test, predictions), 3)))\n",
    "\n",
    "# Homework assignment; google what these error metrics mean :D\n",
    "# Hint. https://www.bmc.com/blogs/mean-squared-error-r2-and-variance-in-regression-analysis/   \n",
    "\n",
    "# Now create another lovely plot for us to save.\n",
    "# Plot between predictions and Y_test\n",
    "x_axis = np.array(range(0, predictions.shape[0]))\n",
    "plt.figure(figsize=(40,20)) # again adjust this size as needed\n",
    "plt.plot(x_axis, predictions, linestyle=\"--\", marker=\"o\", alpha=0.7, color='r', label=\"predictions\")\n",
    "plt.plot(x_axis, Y_test, linestyle=\"--\", marker=\"o\", alpha=0.7, color='g', label=\"Y_test\")\n",
    "plt.xlabel('Row number')\n",
    "plt.ylabel('PRICE')\n",
    "plt.title('Predictions vs Y_test')\n",
    "plt.legend(loc='lower right')\n",
    "plt.savefig(\"./plots/predictions_vs_ytest.png\")\n",
    "plt.clf()\n",
    "plt.close()\n",
    "print('--')\n",
    "print('Done predicting the testing set. Now time to move on to our lovely prediction data')\n",
    "print('to see how well it really performs!')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Let's stop here for a second, take a breath and evaluate what we've done so far."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, we downloaded daily historical stock data for an inputted stock, then scraped Yahoo Finance's website for Industry titles and subsequently downloaded its historical data as well. We then used Pandas to maniuplate the dataframe and add a couple of moving averages to the dataset. From there, we utilized automatic feature engineering and selection to find new, hidden relationships within our data, giving us insight into the data which we may or may not know anything about. We then passed that new dataset to an automatic machine learning module that optimized for the best regression pipeline with the highest predictive power we could find. Pat yourself on the back! This is some new-aged stuff you're doing here.\n",
    "\n",
    "So looking at the results of the training/testing of our model, we see that our final model can predict the closing price of a stock to within an average of $3.54 (MSE), with an R2 value of 0.99, which suggests that our model almost perfectly explains the variance in our data. Now two things to keep in mind here:\n",
    "\n",
    "    1. With such a high R2, would this suggest that we overfitted our model to the training data? Well, we would have to try it out on some new, unseen data to get a final verdict on its power (which we're about to do).\n",
    "\n",
    "    2. Is predicting the closing price to within $3.54 average good enough for the data? Well that depends. If we were trying to predict the closing price of say, AMZN, which as of this writing is trading around the $1,775 - $2,000 mark, that would be an awesome prediction. If we are using a lower priced stock, like CLVS which is around $5 - $10, it's probably not the greatest. The point is, it all depends on the data you're using, and that's a decision you'll have to make for yourself. I will touch on possible ways to (potentially) improve this model later."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [],
   "source": [
    "#print(prediction_df.columns) # for debugging purposes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 9. PREDICT USING UNSEEN DATA"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(90, 92)\n",
      "90\n",
      "92\n",
      "0.78\n",
      "============================\n",
      "[INFO] MSE on prediction set : 4.265\n",
      "[INFO] Max Error on prediction set : 7.191\n",
      "[INFO] MAE on prediction set : 1.212\n",
      "[INFO] MSLE on prediction set : 0.041\n",
      "[INFO] Median AE on prediction set : 0.692\n",
      "---------------------------------\n",
      "[INFO] R2 Score on prediction set : 0.78\n",
      "[INFO] Adjusted R2 Score on prediction set : 0.778\n",
      "[INFO] Explained Variance Score on prediction set : 0.828\n"
     ]
    }
   ],
   "source": [
    "# Armed with our optimized regression pipeline thanks to TPOT, let's\n",
    "# run it against our previously held-back prediciton dataset and see\n",
    "# how it does!\n",
    "\n",
    "# Move the Open and Adj Close columns to the end again.\n",
    "opencolumn = prediction_df['Open']\n",
    "adjclosecolumn = prediction_df['Adj Close']\n",
    "prediction_df = prediction_df.drop(['Open'],axis=1)\n",
    "prediction_df = prediction_df.drop(['Adj Close'],axis=1)\n",
    "prediction_df['Open'] = opencolumn\n",
    "prediction_df['Adj Close'] = adjclosecolumn\n",
    "\n",
    "# Set our X features and Y label.\n",
    "features = prediction_df.drop(['Adj Close'], axis=1)\n",
    "labels = prediction_df['Adj Close']\n",
    "\n",
    "print(features.shape) # for debugging\n",
    "\n",
    "# And standardize them\n",
    "#scaler = MinMaxScaler().fit(features)\n",
    "#features = scaler.transform(features)\n",
    "\n",
    "# Remember how we converted the features to a Numpy array during the\n",
    "# training/testing portion? We need to do the same here for our model\n",
    "# to read the data properly. Again, only do this for the features, not\n",
    "# the label, or you'll likely get errors.\n",
    "features=features.values\n",
    "\n",
    "# Predict the labels! The good part.\n",
    "results = best_model.predict(features)\n",
    "\n",
    "# Now, let's add the predictions to our dataset, and then export the\n",
    "# results to a .csv file for later analysis.\n",
    "predictions_list = []\n",
    "for preds in results:\n",
    "    predictions_list.append(preds)\n",
    "prediction_df['Predictions'] = predictions_list\n",
    "export_filename = 'historical data/' + ticker_input + '_Final_Prediction_Performance.csv'\n",
    "prediction_df.to_csv(export_filename, index=True)\n",
    "\n",
    "# ...but first, let's assign some variables to use for calculating the \n",
    "# Adjust R Squared value\n",
    "data_points = int(features.shape[0])\n",
    "feature_points = int(features.shape[1])\n",
    "r_squared = round(r2_score(labels, results), 3)\n",
    "\n",
    "print(data_points)\n",
    "print(feature_points)\n",
    "print(r_squared) # for debugging\n",
    "\n",
    "# Print our results and compare them to the previous testing error metrics.\n",
    "print('============================')\n",
    "print(\"[INFO] MSE on prediction set : {}\".format(round(mean_squared_error(labels, results), 3)))\n",
    "print(\"[INFO] Max Error on prediction set : {}\".format(round(max_error(labels, results), 3)))\n",
    "print(\"[INFO] MAE on prediction set : {}\".format(round(mean_absolute_error(labels, results), 3)))\n",
    "print(\"[INFO] MSLE on prediction set : {}\".format(round(mean_squared_log_error(labels, results), 3)))\n",
    "print(\"[INFO] Median AE on prediction set : {}\".format(round(median_absolute_error(labels, results), 3)))\n",
    "print('---------------------------------')\n",
    "print('[INFO] R2 Score on prediction set : {}'.format(round(r_squared, 3)))\n",
    "print('[INFO] Adjusted R2 Score on prediction set : {}'.format(round(1 - (1 - r_squared) * (data_points*feature_points - 1) / (data_points*feature_points - feature_points - 1), 3)))\n",
    "print('[INFO] Explained Variance Score on prediction set : {}'.format(round(explained_variance_score(labels, results), 3)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see, we actually had worse results in terms of the MSE, and a lesser R2 in terms of how well the model explains the variance in the data. Some potential reasons for this, looking at the price data in the next section, are that the stock in the last 90 days took a huge dive and consistently performed very poorly, something the regression was not used to seeing during training. The predicted values were well above the actual closing price for many days. So we could run the model again, holding out a different section of stock data to use as our unseen prediction dataset, include those recent 90 days in the training and see if it performs any better.\n",
    "\n",
    "So from here, what we do is manually compare the predictions column to the Adj Close column and see what else we can derive from the predictions. Let's load up the Open, Adj Close and Predictions columns from our exported .csv file from the previous step and see what we can see."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Now fit the best pipeline to the training data to find feature importances...\n",
      "Done!\n"
     ]
    }
   ],
   "source": [
    "# Extract what the best pipeline was and fit it to the training set\n",
    "# to get an idea of the most important features used by the model were.\n",
    "# Is there a way to do this during the first .fit() function so that\n",
    "# we don't have to run it a second time here? Althought it's not a big deal.\n",
    "print('Now fit the best pipeline to the training data to find feature importances...')\n",
    "exctracted_best_model = best_model.fitted_pipeline_.steps[-1][1]\n",
    "\n",
    "# Train only the `exctracted_best_model` using the training/vildation set\n",
    "exctracted_best_model.fit(X_train, Y_train)\n",
    "\n",
    "# plot model's feature importance and save the plot for later\n",
    "feature_importance = exctracted_best_model.feature_importances_\n",
    "feature_importance = 100.0 * (feature_importance / feature_importance.max())\n",
    "sorted_idx = np.argsort(feature_importance)\n",
    "pos        = np.arange(sorted_idx.shape[0]) + .5\n",
    "plt.figure(figsize=(40,20)) # play with this to adjust the size of the plot to your specs\n",
    "plt.barh(pos, feature_importance[sorted_idx], align='center')\n",
    "plt.yticks(pos, df.columns[sorted_idx])\n",
    "plt.xlabel('Relative Importance')\n",
    "plt.title('Variable Importance')\n",
    "plt.savefig(\"./plots/feature_importance.png\")\n",
    "plt.clf()\n",
    "plt.close()\n",
    "print('Done!')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 10. POST-PREDICTION ANALYSIS"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "         Date  Open  Adj Close  Predictions\n",
      "0  2019-04-09 25.02      24.34        25.26\n",
      "1  2019-04-10 24.38      24.74        24.29\n",
      "2  2019-04-11 24.76      23.76        24.63\n",
      "3  2019-04-12 23.88      23.48        23.32\n",
      "4  2019-04-15 21.75      20.55        21.74\n"
     ]
    }
   ],
   "source": [
    "# Load in the 3 columns of interest from the .csv file.\n",
    "pred_df = pd.read_csv(export_filename,usecols = ['Date','Open','Adj Close','Predictions'])\n",
    "print(pred_df.head())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "          Date  Open  Adj Close  Predictions  AdjClosePercent  PredPercent  \\\n",
      "0   2019-04-09 25.02      24.34        25.26            -0.03         0.01   \n",
      "1   2019-04-10 24.38      24.74        24.29             0.01        -0.00   \n",
      "2   2019-04-11 24.76      23.76        24.63            -0.04        -0.01   \n",
      "3   2019-04-12 23.88      23.48        23.32            -0.02        -0.02   \n",
      "4   2019-04-15 21.75      20.55        21.74            -0.06        -0.00   \n",
      "5   2019-04-16 20.53      21.21        20.35             0.03        -0.01   \n",
      "6   2019-04-17 21.39      20.80        21.48            -0.03         0.00   \n",
      "7   2019-04-18 20.73      20.88        20.39             0.01        -0.02   \n",
      "8   2019-04-22 20.67      20.32        20.33            -0.02        -0.02   \n",
      "9   2019-04-23 20.25      20.83        19.58             0.03        -0.03   \n",
      "10  2019-04-24 20.83      19.87        20.37            -0.05        -0.02   \n",
      "11  2019-04-25 19.83      19.10        19.43            -0.04        -0.02   \n",
      "12  2019-04-26 19.13      19.07        18.94            -0.00        -0.01   \n",
      "13  2019-04-29 19.07      18.81        18.82            -0.01        -0.01   \n",
      "14  2019-04-30 18.80      18.27        18.80            -0.03         0.00   \n",
      "15  2019-05-01 18.28      17.71        18.06            -0.03        -0.01   \n",
      "16  2019-05-02 18.32      18.44        18.33             0.01         0.00   \n",
      "17  2019-05-03 18.41      18.95        18.36             0.03        -0.00   \n",
      "18  2019-05-06 18.55      19.10        18.78             0.03         0.01   \n",
      "19  2019-05-07 19.05      19.46        18.75             0.02        -0.02   \n",
      "20  2019-05-08 19.31      19.98        19.12             0.03        -0.01   \n",
      "21  2019-05-09 19.78      19.46        19.32            -0.02        -0.02   \n",
      "22  2019-05-10 19.31      19.28        19.17            -0.00        -0.01   \n",
      "23  2019-05-13 18.80      18.20        19.00            -0.03         0.01   \n",
      "24  2019-05-14 18.45      18.74        18.71             0.02         0.01   \n",
      "25  2019-05-15 18.63      18.77        18.85             0.01         0.01   \n",
      "26  2019-05-16 18.90      18.09        18.79            -0.04        -0.01   \n",
      "27  2019-05-17 17.93      17.07        18.05            -0.05         0.01   \n",
      "28  2019-05-20 17.00      16.36        17.79            -0.04         0.05   \n",
      "29  2019-05-21 16.71      17.41        17.69             0.04         0.06   \n",
      "..         ...   ...        ...          ...              ...          ...   \n",
      "60  2019-07-05 14.61      14.20        14.71            -0.03         0.01   \n",
      "61  2019-07-08 14.06      14.24        14.45             0.01         0.03   \n",
      "62  2019-07-09 14.09      14.07        14.39            -0.00         0.02   \n",
      "63  2019-07-10 14.12      13.83        14.49            -0.02         0.03   \n",
      "64  2019-07-11 13.86      13.24        13.91            -0.04         0.00   \n",
      "65  2019-07-12 13.13      13.25        13.42             0.01         0.02   \n",
      "66  2019-07-15 13.32      12.96        13.76            -0.03         0.03   \n",
      "67  2019-07-16 12.98      12.80        13.02            -0.01         0.00   \n",
      "68  2019-07-17 12.80      12.33        12.62            -0.04        -0.01   \n",
      "69  2019-07-18 12.30      11.57        12.49            -0.06         0.02   \n",
      "70  2019-07-19 11.66      10.95        12.40            -0.06         0.06   \n",
      "71  2019-07-22 10.99      10.84        12.40            -0.01         0.13   \n",
      "72  2019-07-23 10.92      11.20        12.37             0.03         0.13   \n",
      "73  2019-07-24 11.22      11.07        12.37            -0.01         0.10   \n",
      "74  2019-07-25 11.04      10.63        12.37            -0.04         0.12   \n",
      "75  2019-07-26 10.72      10.79        12.42             0.01         0.16   \n",
      "76  2019-07-29 10.78      10.70        12.37            -0.01         0.15   \n",
      "77  2019-07-30 10.62      11.13        12.37             0.05         0.16   \n",
      "78  2019-07-31 11.20      10.55        12.37            -0.06         0.10   \n",
      "79  2019-08-01  9.33       8.87        12.37            -0.05         0.33   \n",
      "80  2019-08-02  8.46       9.26        12.37             0.09         0.46   \n",
      "81  2019-08-05  9.00       8.86        12.36            -0.02         0.37   \n",
      "82  2019-08-06  8.97       9.42        12.36             0.05         0.38   \n",
      "83  2019-08-07  9.25       8.97        12.38            -0.03         0.34   \n",
      "84  2019-08-08  7.76       5.83        12.37            -0.25         0.59   \n",
      "85  2019-08-09  5.68       5.68        12.34             0.00         1.17   \n",
      "86  2019-08-12  5.69       5.67        12.34            -0.00         1.17   \n",
      "87  2019-08-13  5.40       5.69        12.34             0.05         1.28   \n",
      "88  2019-08-14  5.51       5.34        12.34            -0.03         1.24   \n",
      "89  2019-08-15  5.31       5.20        12.39            -0.02         1.33   \n",
      "\n",
      "   UpDown PredsUpDown  WinLose  WinLosePerc  \n",
      "0    Down          Up     0.00         0.48  \n",
      "1      Up        Down     0.00         0.48  \n",
      "2    Down        Down     1.00         0.48  \n",
      "3    Down        Down     1.00         0.48  \n",
      "4    Down        Down     1.00         0.48  \n",
      "5      Up        Down     0.00         0.48  \n",
      "6    Down          Up     0.00         0.48  \n",
      "7      Up        Down     0.00         0.48  \n",
      "8    Down        Down     1.00         0.48  \n",
      "9      Up        Down     0.00         0.48  \n",
      "10   Down        Down     1.00         0.48  \n",
      "11   Down        Down     1.00         0.48  \n",
      "12   Down        Down     1.00         0.48  \n",
      "13   Down        Down     1.00         0.48  \n",
      "14   Down          Up     0.00         0.48  \n",
      "15   Down        Down     1.00         0.48  \n",
      "16     Up          Up     1.00         0.48  \n",
      "17     Up        Down     0.00         0.48  \n",
      "18     Up          Up     1.00         0.48  \n",
      "19     Up        Down     0.00         0.48  \n",
      "20     Up        Down     0.00         0.48  \n",
      "21   Down        Down     1.00         0.48  \n",
      "22   Down        Down     1.00         0.48  \n",
      "23   Down          Up     0.00         0.48  \n",
      "24     Up          Up     1.00         0.48  \n",
      "25     Up          Up     1.00         0.48  \n",
      "26   Down        Down     1.00         0.48  \n",
      "27   Down          Up     0.00         0.48  \n",
      "28   Down          Up     0.00         0.48  \n",
      "29     Up          Up     1.00         0.48  \n",
      "..    ...         ...      ...          ...  \n",
      "60   Down          Up     0.00         0.48  \n",
      "61     Up          Up     1.00         0.48  \n",
      "62   Down          Up     0.00         0.48  \n",
      "63   Down          Up     0.00         0.48  \n",
      "64   Down          Up     0.00         0.48  \n",
      "65     Up          Up     1.00         0.48  \n",
      "66   Down          Up     0.00         0.48  \n",
      "67   Down          Up     0.00         0.48  \n",
      "68   Down        Down     1.00         0.48  \n",
      "69   Down          Up     0.00         0.48  \n",
      "70   Down          Up     0.00         0.48  \n",
      "71   Down          Up     0.00         0.48  \n",
      "72     Up          Up     1.00         0.48  \n",
      "73   Down          Up     0.00         0.48  \n",
      "74   Down          Up     0.00         0.48  \n",
      "75     Up          Up     1.00         0.48  \n",
      "76   Down          Up     0.00         0.48  \n",
      "77     Up          Up     1.00         0.48  \n",
      "78   Down          Up     0.00         0.48  \n",
      "79   Down          Up     0.00         0.48  \n",
      "80     Up          Up     1.00         0.48  \n",
      "81   Down          Up     0.00         0.48  \n",
      "82     Up          Up     1.00         0.48  \n",
      "83   Down          Up     0.00         0.48  \n",
      "84   Down          Up     0.00         0.48  \n",
      "85   Down          Up     0.00         0.48  \n",
      "86   Down          Up     0.00         0.48  \n",
      "87     Up          Up     1.00         0.48  \n",
      "88   Down          Up     0.00         0.48  \n",
      "89   Down          Up     0.00         0.48  \n",
      "\n",
      "[90 rows x 10 columns]\n",
      "              Date  Open  Adj Close  Predictions  AdjClosePercent  \\\n",
      "count           90 90.00      90.00        90.00            90.00   \n",
      "unique          90   nan        nan          nan              nan   \n",
      "top     2019-06-19   nan        nan          nan              nan   \n",
      "freq             1   nan        nan          nan              nan   \n",
      "mean           NaN 15.06      14.91        15.87            -0.01   \n",
      "std            NaN  4.45       4.43         3.32             0.04   \n",
      "min            NaN  5.31       5.20        12.34            -0.25   \n",
      "25%            NaN 12.90      12.80        12.72            -0.04   \n",
      "50%            NaN 14.85      14.73        14.78            -0.01   \n",
      "75%            NaN 18.52      18.40        18.74             0.01   \n",
      "max            NaN 25.02      24.74        25.26             0.10   \n",
      "\n",
      "        PredPercent UpDown PredsUpDown  WinLose  WinLosePerc  \n",
      "count         90.00     90          90    90.00        90.00  \n",
      "unique          nan      2           2      nan          nan  \n",
      "top             nan   Down          Up      nan          nan  \n",
      "freq            nan     57          58      nan          nan  \n",
      "mean           0.11    NaN         NaN     0.48         0.48  \n",
      "std            0.30    NaN         NaN     0.50         0.00  \n",
      "min           -0.03    NaN         NaN     0.00         0.48  \n",
      "25%           -0.01    NaN         NaN     0.00         0.48  \n",
      "50%            0.01    NaN         NaN     0.00         0.48  \n",
      "75%            0.05    NaN         NaN     1.00         0.48  \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "max            1.33    NaN         NaN     1.00         0.48  \n",
      "         Date  Open  Adj Close  Predictions  AdjClosePercent  PredPercent  \\\n",
      "0  2019-04-09 25.02      24.34        25.26            -0.03         0.01   \n",
      "1  2019-04-10 24.38      24.74        24.29             0.01        -0.00   \n",
      "2  2019-04-11 24.76      23.76        24.63            -0.04        -0.01   \n",
      "3  2019-04-12 23.88      23.48        23.32            -0.02        -0.02   \n",
      "4  2019-04-15 21.75      20.55        21.74            -0.06        -0.00   \n",
      "\n",
      "  UpDown PredsUpDown  WinLose  WinLosePerc  \n",
      "0   Down          Up     0.00         0.48  \n",
      "1     Up        Down     0.00         0.48  \n",
      "2   Down        Down     1.00         0.48  \n",
      "3   Down        Down     1.00         0.48  \n",
      "4   Down        Down     1.00         0.48  \n"
     ]
    }
   ],
   "source": [
    "# Now let's do some manual analysis of this dataset and see what we can find.\n",
    "# First, let's calculate some percentages of the predictions and Adj Closes.\n",
    "pred_df['AdjClosePercent'] = round((pred_df['Adj Close'] - pred_df['Open']) / pred_df['Open'], 2)\n",
    "pred_df['PredPercent'] = round((pred_df['Predictions'] - pred_df['Open']) / pred_df['Open'], 2)\n",
    "\n",
    "# Then, let's see if the price closed higher or lower/even that day...\n",
    "pred_df.loc[pred_df['Adj Close'] > pred_df['Open'], 'UpDown'] = 'Up' \n",
    "pred_df.loc[pred_df['Adj Close'] <= pred_df['Open'], 'UpDown'] = 'Down'\n",
    "\n",
    "# ...and compare that to the direction our model predicted.\n",
    "pred_df.loc[pred_df['Predictions'] > pred_df['Open'], 'PredsUpDown'] = 'Up' \n",
    "pred_df.loc[pred_df['Predictions'] <= pred_df['Open'], 'PredsUpDown'] = 'Down'\n",
    "\n",
    "# Add a new column\n",
    "df['WinLose'] = 0\n",
    "\n",
    "# Now, if we predicted the price to close higher than the open,\n",
    "# put a 1, otherwise put a 0.\n",
    "pred_df.loc[pred_df['UpDown'] == pred_df['PredsUpDown'], 'WinLose'] = 1\n",
    "pred_df.loc[pred_df['UpDown'] != pred_df['PredsUpDown'], 'WinLose'] = 0\n",
    "\n",
    "# Now take an average.\n",
    "pred_df['WinLosePerc'] = pred_df['WinLose'].mean()\n",
    "\n",
    "print(pred_df)\n",
    "\n",
    "# Then let's get another summary and just for fun, export it.\n",
    "print(pred_df.describe(include='all'))\n",
    "file_report = 'statistical summaries/' + ticker_input + '_final_preds_summ_' + \".txt\"\n",
    "with open(file_report, \"w\") as f:\n",
    "\n",
    "    f.write(\"\\nColumn names\")\n",
    "    f.write(\"\\n\")\n",
    "    f.write(str(pred_df.columns))\n",
    "    f.write(\"\\n\")\n",
    "\n",
    "    f.write(\"\\nStatistical summary\")\n",
    "    f.write(\"\\n\")\n",
    "    f.write(str(pred_df.describe(include='all')))\n",
    "    f.write(\"\\n\")\n",
    "\n",
    "    f.write(\"\\nAverage Direction Win %\")\n",
    "    f.write(\"\\n\")\n",
    "    f.write(str(pred_df['WinLosePerc'].values))\n",
    "    f.write(\"\\n\")\n",
    "\n",
    "print(pred_df.head())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "From here you could do any other type of analysis you wanted, or generate reports using any of the preceding ways that we did that.\n",
    "\n",
    "As we can see above, our model isn't very good. The mean prediction percentage was +0.11, yet\n",
    "the mean actual price percentage was -0.01. The standard deviation is off, and the min/max are,\n",
    "I would say way off. But let's look at the direction prediction for a second.\n",
    "\n",
    "So what can we do from here? Well, if we improve our model to become better at predicting the closing\n",
    "price (ideas on how to do this in a minute), remember how we exported the TPOT model after it was\n",
    "optimized? That export could be run on a new dataset to predict the closing\n",
    "prices of that dataset. So, that new dataset could consist of today's opening price, and all of the\n",
    "other indicators that were used along the way, and the model would tell you what it thinks the closing\n",
    "price would be! Take a look in the Exported Pipeline folder in the project's directory and your last\n",
    "homework assignment will be to get that working on your own. :D"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Congratulations!\n",
    "\n",
    "You made it to the end of my first repository on how to optimize a Multiple Linear Regression model on stock market data. Give yourself a big pat on the back. As previously discussed, the performance of this model was not the point of this repository, but rather my chance to showcase my programming knowledge thus far, and to clearly explain some of the concepts within as a personal challenge to myself. I hope you learned how to implement some of these operations yourself if you're a newbie. I really appreciated projects like this when I was getting started. Some of my future projects may not be this well documented, unless I'm introducing a new concept.\n",
    "\n",
    "In my next project, I'm going to tackle turning this model into a Classification type problem, whereby we get a Buy or Sell signal, rather than trying to predict the actual closing price (because surely all of those fancy technical indicators and engineered features among them could determine a direction at least, right?). I will do this using TPOT/SkLearn as well.\n",
    "\n",
    "In the future I'm going to recreate this project using Tensorflow to familiarize myself with that module using both Regression and Classification there as well. Then my next big project after that will be to learn how to setup a custom OpenAI Gym stock trading environment where I can get to know how to utilize Reinforcement Learning algorithms in a simulated live trading environment. So stay tuned for all of that."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### How to make the model better?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Some ways to potentially improve this model might be:\n",
    "\n",
    "1. Add in support and resistance levels into the original dataset\n",
    "2. Run a longer TPOT session with more generations and population sizes, and more regressors in the config\n",
    "3. Experiment with more or less cross validation folds during training\n",
    "4. Add historical news sentiment or earnings results to the dataset\n",
    "5. ... so much more\n",
    "\n",
    "Things to do in the future for this project (whether I or someone else gets to it or not):\n",
    "\n",
    "1. Add more industry's to the list to allow more stock symbols to be searched\n",
    "2. Deal with how to standardize the data before training/testing and prediction, then how to unscale it for review\n",
    "3. Matplotlib a candlestick chart after getting the original historical dataset just for fun\n",
    "4. ... anything else you want"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Thanks for reading!"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}