diff --git a/.gitignore b/.gitignore index 3108f41..3fb596e 100644 --- a/.gitignore +++ b/.gitignore @@ -1,4 +1,7 @@ .ipynb_checkpoints/* **/__pycache__/* **/*.mp4 -.vscode/ \ No newline at end of file +.vscode/ +output/ +10. Surrogate Optimization.ipynb +text_generation.ipynb diff --git a/9. DDS.ipynb b/9. DDS.ipynb index 3f858b0..3a8c158 100644 --- a/9. DDS.ipynb +++ b/9. DDS.ipynb @@ -4,7 +4,235 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# Dynamically Dimensioned Search (DDS)\n" + "# Dynamically Dimensioned Search (DDS)\n", + "DDS is a optimization algorithm developed by [Tolson & Shoemaker (2007)](https://agupubs.onlinelibrary.wiley.com/doi/epdf/10.1029/2005WR004723) which focuses on finding the global optimum in a pre-specified number of function evaluations. This makes it especially useful for problems where the function evauation itself can be expensive. It can be adapted to both continuous and integer variables. DDS is especially good for high dimensional problems $(>\\sim 6)$.\n", + "\n", + "## General Description\n", + "The DDS algorithm has the following steps:\n", + "\n", + "1. Pick $x_{best} = $ initial guess of the solution.\n", + "2. Randomly determine which of the decision variables to perturb.\n", + "3. Compute $x_{new}$ by adding to $x_{best}$ a pertubation taken from $\\mathcal{N} (0, \\sigma)$ for the components that were selected in step 2.\n", + "4. If $Cost(x_{new}) < Cost(x_{best})$, then $x_{best} = x_{new}$. Else do nothing.\n", + "5. If maximum number of function evaluations achieved, then stop. Otherwise goto step 2.\n", + "\n", + "Since the problem can have $D$ number of dimensions, the possible combinations of pertubation is $2^D$. The method is called \"dynamically dimensioned\" as the number of dimensions perturbed is changed from iteration to iteration. The length of the pertubation is a (reflected) random variable $\\mathcal{N} (0, \\sigma)$, and hence can be either positive or negative.\n", + "\n", + "The maximum number of function evaluations allowed, $m$, also changes the algorithm. The main idea is that we want to gradually decrease the numebr of dimensions perturbed as the number of iterations increases. This rate of decrease in the number of dimensions depends on $m$. DDS gradually decreases the probability that any one element is selected for pertubation using the formula:\n", + "\n", + "$$P(i) = 1-\\frac{ln(i)}{ln(m)}$$\n", + "\n", + "where $P(i)$ is the probability of pertubation in the $i$th iteration. Other forms of $P(i)$ can be used, but htey must be monotonically decreasing. If no dimension is selected for pertubation in an iteration, then 1 dimension is selected at random to generate $x_{new}$.\n", + "\n", + "## Implementation Details\n", + "0. **DDS Inputs:**\n", + " * neighborhood pertubation size parameter, $r$ (0.2 is deafult)\n", + " * maximum number of function evaluations, $m$\n", + " * lower, $x^{min}$, and upper, $x^{max}$, bounds for all $d$ decision variables\n", + " * initial solution, $x^0 = [x_1, \\ldots, x_d]$\n", + "1. Set counter $i=1$, and evaluate function $F$ at initial solution $\\to F^{best} = F(x^0), x^{best} = x^0$.\n", + "2. Using probability $P(i) = 1-\\frac{ln(i)}{ln(m)}$ select $J$ of the $D$ decision variables for inclusion in set of perturbed varaibles, $\\{R\\}$ in iteration $i$. Add the pertubations to $\\{R\\}$. If $\\{R\\} = \\phi$ then seect one random dimension.\n", + "3. For $j\\in (1,\\ldots, J)$ decision variables in $\\{R\\}$, perturb $x^{best}_j$ using a standard normal random variable, \\mathcal{N} (0, 1), reflecting the decision variable bounds if necessary.\n", + "$$x_j^{new} = x_j^{best} + \\sigma_j \\mathcal{N}(0,1),\\quad\\text{where }\\sigma_j = r(x_j^{max} - x_j^{min})$$\n", + " * **Reflection** has the property of reflecting the part of the probability distribution outside the maimum and minimum bounds back into bounds (similar to what is shown in the figure below). This refers to the folllowing operation:\n", + " * If $x_j^{new} < x_j^{min}\\to x_j^{new} = x_j^{min} + (x_j^{min} - x_j^{new})$. If this leads to $x_j^{new} > x_j^{max}\\to x_j^{new} = x_j^{min}$\n", + " * If $x_j^{new} > x_j^{max}\\to x_j^{new} = x_j^{max} - (x_j^{new} - x_j^{max})$. If this leads to $x_j^{new} < x_j^{min}\\to x_j^{new} = x_j^{max}$\n", + " \n", + " ![Reflected Gaussian](Images/reflected_gaussian.png)\n", + "4. Evaluate $F(x^{new})$ and update the current best solution if necessary: If $F(x^{new})\\leq F^{best}\\to F^{best} = F(x^{new}), x^{best}=x^{new}$\n", + "5. Udate iteration count $i = i+1$ and check stopping criteron: If $i=m$, then return $F^{best}, x^{best}$. Else, goto 2.\n", + "\n", + "Since $r$ can be set to $0.2$, and $m$ is a non-calibrated parameter, DDS essentially has no parameters to be calibrated.\n", + "\n", + "## DDS vs. Other Algorithms\n", + "* **DDS vs. Simulated Annealing:**\n", + " * Similarities: Big changes in current solution are less likely to occur as the number of iterations increase. SA stops uphill move in later iteartions, whereas DDS stops changes in many dimensions. You can use the maximum number of allowed evaluations to change this for both SA and DDS.\n", + " * Difference: Maximum number of evaluations is incorporated directly into DDS. However, DDS does not accept uphill moves, whereas SA does. DDS is also usually more effective in high dimensional problems.\n", + "* **DDS vs. Greedy Search:**\n", + " * Similarities: Both accept a new move only if they are downhill.\n", + " * Difference: Probability of moving from one state to another is constant in greedy search, but changes for DDS.\n", + "* **DDS vs. Genetic Algorithms:**\n", + " * Similarities: GA mutation is similar to choice of dimensions to perturn in DDS\n", + "\n", + "DDS is not particularly good compared ot other algorithms when the dimensionality of the problem is low. This is because of the curse of dimensionality. As the dimensions of the problem grow, the solution space grows exponentially. This makes it difficult for other algorithms to search the space. With DDS, by reducing the number of perturbed dimensions you reduce the size of the neighborhood and reduce the chance of destroying the current best solution by changing it too much.\n", + "\n", + "## Optimizing the \"bump\" function using DDS\n", + "The bump function to be optimized (maximized) is defined below:\n", + "\n", + "$f(\\overrightarrow{x})=\n", + "\\begin{cases}\n", + "|\\frac{\\sum_{i=1}^n cos^4(x_i) - 2\\prod_{i=1}^n cos^2(x_i)}{\\sqrt{\\sum_{i=1}^n ix_i^2}}|,\\quad\\text{if } (\\forall i, 0\\leq x_i \\leq 10)\\text{ and }(\\prod_{i=1}^n x_i \\geq 0.75)\\\\\n", + "0,\\quad\\text{otherwise}\n", + "\\end{cases}$" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [], + "source": [ + "import numpy as np\n", + "from math import log\n", + "from matplotlib import pyplot as plt" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [], + "source": [ + "def bump(x):\n", + " '''\n", + " Evaluate the bump function for value x\n", + " :param x: (d,) numpy array of the value\n", + " '''\n", + " d = x.shape[0]\n", + " if any(x<0) or any(x>10) or np.product(x) < 0.75:\n", + " return 0\n", + " sum1 = np.sum(np.cos(x)**4)\n", + " tim1 = 2 * np.product(np.cos(x)**2)\n", + " sum2 = np.sqrt(np.sum(np.arange(1,d+1)*x**2))\n", + " return abs((sum1 - tim1)/sum2)" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [], + "source": [ + "class DDS():\n", + " def __init__(self, function, x_min, x_max, max_evals, r=0.2, x_initial=None):\n", + " '''\n", + " Run DDS optimization (maximization) for given function\n", + " :param function: function to optimize which accepts (d,) numpy input\n", + " :param x_min: (d,) numpy array of minimum values for each dimension\n", + " :param x_max: (d,) numpy array of maximum values for each dimension\n", + " :param max_evals: maximum number of allowed function evaluations\n", + " :param r: parameter to scale pertubations. Default: 0.2\n", + " :param x_initial: Initial vairable. Default: None (choose randomly between [x_min, x_max))\n", + " '''\n", + " self.d = len(x_min)\n", + " self.x_min = x_min\n", + " self.x_max = x_max\n", + " self.f = function\n", + " self.r = r\n", + " self.m = max_evals\n", + " self.best = x_initial if x_initial is not None else np.random.rand(self.d)*(x_max-x_min)+x_min\n", + " self.best_cost = self.f(self.best)\n", + "\n", + " def run(self):\n", + " x_range = self.x_max - self.x_min\n", + " best_costs = []\n", + " for i in range(1, self.m+1):\n", + " dim_perturb = (np.random.rand(self.d) < 1-(log(i)/log(self.m))).astype('float')\n", + " k = np.count_nonzero(dim_perturb)\n", + " if k == 0:\n", + " dim_perturb[np.random.randint(0, high=self.d, size=1)[0]] = 1.0\n", + " k = 1\n", + " dim_perturb[dim_perturb!=0] = np.random.normal(size=k)\n", + " curr = self.best + dim_perturb*self.r*x_range\n", + "\n", + " reflect_min = curr < self.x_min\n", + " curr[reflect_min] = 2*self.x_min[reflect_min] - curr[reflect_min]\n", + " correct_twice = np.logical_and(reflect_min, curr>self.x_max)\n", + " curr[correct_twice] = self.x_min[correct_twice]\n", + "\n", + " reflect_max = curr > self.x_max\n", + " curr[reflect_max] = 2*self.x_max[reflect_max] - curr[reflect_max]\n", + " correct_twice = np.logical_and(reflect_max, curr self.best_cost:\n", + " self.best = curr\n", + " self.best_cost = curr_cost\n", + " \n", + " best_costs.append(self.best_cost)\n", + "\n", + " return best_costs\n", + "\n", + " def plot(self):\n", + " best_costs = self.run()\n", + " plt.plot(np.arange(len(best_costs)), best_costs)\n", + " plt.xlabel('Iteration number')\n", + " plt.ylabel('Optimal function value')\n", + " plt.title('Optimal function value vs. iterations')" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [ + { + "output_type": "display_data", + "data": { + "text/plain": "
", + "image/svg+xml": "\r\n\r\n\r\n\r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n\r\n", + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEWCAYAAAB8LwAVAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3deZxcVZ338c83+74HyEoCCbIv0iwKIiJLAIXRUUR0FEEzKDziiDOCCyouw+A2OjIzRMXhERFEBSMPTmQxKAiYBAKYABJCICtJyN6d3n/PH/dWqFSqu293urq6ur/v16teXffec2/9bnV3/eqcc+85igjMzMwK9Sl3AGZm1j05QZiZWVFOEGZmVpQThJmZFeUEYWZmRTlBmJlZUU4QBoCkqZJ2SOpbgmN/WdKtrWz/uKRX09cf29mv38rrfk7Sj7rq9bKQNF/SR8sdRxaSPiDp92WO4b8lfbGcMfRkThAVStLFkp6RVCNpnaT/kjSqHfuvkHR6bjkiXomIYRHRVJqIW4yjP/Ad4Mz09V8r0eucKmlV/rqI+EZEVMSHcXcUET+LiDNzy5JC0oxSvV76N/9wQQyXRcRXS/WavZ0TRAWSdBXwb8A/AyOBE4H9gfskDShnbB2wLzAIWFLuQKx8JPUrdwxWRET4UUEPYASwA7igYP0wYD1wSbr8ZeCXwB3AduAJ4Kh020+BZmBneqx/AaYBAfRLy8wHvgb8OS3zW2As8DNgG7AAmJb3+t8DVqbbFgFvydv2ZeDWIudyEFCdvu4O4MHCOPJi+Wj6/GLgYeBbwGbgJeDsvLJjgJ8Aa9LtdwND03NtTl9nBzCxMC7gPJJEtSV9zUPytq0APgM8DWxN39dBRc5pYLr/4Xnrxqevvw8wGrgH2JDGdw8wuYVzLYyv8Hc0EvgxsBZYnf6++haJaWL6+mPy1h0DbAT6AzOAh9Lz2gjckfFv8WLg4fT5H9PYqtP3933p+ncAi9P35M/AkQXv6WfT97QO6AdcDbxI8je7FHhXWvYQoBZoSo+/JV3/P8DX8o75MWAZsAmYC0zM2xbAZcAL6Xt/I6B0W4feg57+cA2i8ryZ5Bv3r/NXRsQO4HfAGXmrzwfuJPnQvA24W1L/iPgH4BXgnZE069zQwmtdCPwDMAk4EHiU5MN3DPAs8KW8sguAo/Ne605Jg1o7kYj4G3BYujgqIk5rrXyeE4DngXHADcCPJSnd9lNgSHrcfYDvRkQ1cDawJj3fYRGxJv+Akg4Cfg58iuQD/V7gtwU1sguAWcB04EiSD8jCc6oj+d28v2C/hyJiPUmt/SckNb6pJB/cP8h43oVuARpJPtyOAc4E9mgyS8/1UeDv81ZfBPwyIhqArwK/J0lek4H/aG8gEXFK+vSo9P29Q9IbgZuBfyT5cnETMFfSwLxd3w+cS/L7byRJDm8hSX5fAW6VNCEiniX5cH80Pf4ezamSTgP+leT9ngC8DNxeUOwdwHHAUWm5s9L1e/0e9EROEJVnHLAx/WcqtDbdnrMoInIfAt8hSSwntuO1fhIRL0bEVpLk82JE3J++9p0kH0oARMStEfFaRDRGxLdJvkm/oX2nltnLEfHDSPpLbiH5MNhX0gSSRHBZRGyOiIaIeCjjMd8H/L+IuC99v74FDCZJyDnfj4g1EbGJpEZ1dAvHuo3dE8RF6TrS9+hXEVETEduBrwNvzRjjLpL2JTnXT0VEdZp8vkuS1FuNKU2mF+ZiAhpIEtbEiKiNiIeLH6LdPgbcFBGPR0RTRNxCUlPI/xv8fkSsjIidABFxZ/oeN0fEHSTf9o/P+HofAG6OiCfSRH0N8CZJ0/LKXB8RWyLiFeAPvP47LNV7UNGcICrPRmBcC222E9LtOStzTyKiGVhF0tyQ1at5z3cWWR6WW5B0laRnJW2VtIXkG2B+supM63JPIqImfToMmAJsiojNHTjmRJJvnLnjNpO8f5OKvS5QQ975F3gQGCzpBEn7k3wI3QUgaYikmyS9LGkbSdPMqA5cPbY/SfPQWklb0vf8JpJaUzG/JPmwnAicQtLc8qd0278AAv4iaYmkS9oZS2sxXpWLL41xCrv/Da7M30HShyQtzit/ONn/jgp/hzuA18j2OyzVe1DR3DFUeR4l+Rb2buAXuZWShpJ8o/xcXtkpedv7kFSdc00rnTaMr6S3kLQlvx1YEhHNkjaT/MO1V3X6cwhJfwbAfhn3XQmMkTQqIrYUbGvrfNcAR+QW0m/ZU0ja9tslPf9fkHxjfxW4J60tAFxFUrM6ISLWSToaeJLi71U1yfuQk/8+rCT5OxjXQm2yMKYt6SWpF5C05/88Im2Yj1hH8m0fSScD90v6Y0Qsy3zSxa0Evh4RX28ttNyTNJn+kOTv6NGIaJK0mNffmyy/w/3zjjeUpGmrzd9hCd+DiuYaRIVJm3u+AvyHpFmS+qdV6DtJagg/zSt+rKR3p7WNT5F8oDyWbnsVOKCTwhpO0ha+Aegn6VqSzvR2i4gNJP/QH5TUN/0md2DGfdeSNIX9p6TR6XuTaxt/FRgraWQLu/8COFfS29NLb68ieb/+3JHzIGm+eR9Js8dteeuHk9S+tkgaw+79OIUWA6ek96iMJGkyAXad6++Bb0saIamPpAMltdZcdRvwIZK+iF0xSXqvpMnp4maSD+KOXO5c+Df1Q+CytCYlSUMlnStpeAv7D01fe0Ma10dIahD5x5/cypV6twEfkXR02s/xDeDxiFjRVuCd+B70KE4QFSjtVP4cSTv5NuBxkm9rb0/bXnN+Q/IhtZmks/ndafs6JJ15X0ir8p/Zy5DmkXww/42kil9LQdNBO32M5BLe10g6m9vzIf0PJO3Jz5Fc1fUpgIh4jqQTenl6zrs1tUXE88AHSTonNwLvJOnEr+/ICUTE4yQ1gIkk703Ov5P0bWwkSdb/28ox7iO5WuppkivD7iko8iFgAMnVPptJmpEmtBLWXGAm8GpEPJW3/jjgcUk70jJXRsRLAGlzywdaPdnXfRm4JX1/L4iIhSS/yx+k8S2jSMd+TkQsBb5NUkt+laRG90hekQdJrjJbJ2ljkf0fAL4I/IqkP+5AWu6TKdTie9Cb5S7xsh5G0peBGRHxwXLHYmaVyTUIMzMrygnCzMyKchOTmZkV5RqEmZkV1WPugxg3blxMmzat3GGYmVWURYsWbYyI8cW29ZgEMW3aNBYuXFjuMMzMKoqkl1vaVtImpvRGruclLZN0dZHtF0vakN5av1h5E6VI+rCkF9LHh0sZp5mZ7alkNYh0bJkbSUYXXQUskDQ3vRkm3x0RcUXBvrk7TKtI7mhclO7bkTF2zMysA0pZgzgeWBYRy9O7UW8nGX46i7OA+yIiN/DafSTDLJuZWRcpZYKYxO7DLaxi91EVc/5e0tOSfikpN7hc1n3NzKxESpkgio1OWXjTxW9JZiU7ErifZGz/rPsiabakhZIWbtiwYa+CNTOz3ZUyQawib7hpdh9qGtg1eUpucLkfAsdm3Tfdf05EVEVE1fjxRa/SMjOzDiplglgAzJQ0PR2e90KSURJ3SWcAyzmPZBpLSEYHPTMdsnk0yVSK80oYq5mZFSjZVUwR0SjpCpIP9r4kUwEukXQdsDAi5gKflHQeyVwCm0iHAo6ITZK+SpJkAK5Lp3k0M+sRXttRx88ef4XGpua9PtZ+Iwdz0QlTOyGq3fWYsZiqqqrCN8qZWaX48cMv8dV7kqv+1ZG5F/McPWUUd33ipA7tK2lRRFQV29Zj7qQ2M6ska7fsZFD/Pjx73Sy0txmiRDxYn5lZGazbVst+IwZ12+QAThBmZmWxbmst+40cVO4wWuUEYWZWBrkaRHfmPggz63XWba3lh39aTm1DU5e/dlNzsGTNNlZv2cm5R05oe4cycoIws4oXEXziZ0/w3Lrtmcpvrqmnuq6RkYP7lziy4qaPG8o5h0/gvKMmluX1s3KCMLOyWrmphvnPr9+rY6zfXsfv/rqOk2aMZezQgW2W7yP4wIn7c9y0MXv1uj2dE4SZldW3f/88dy/eYySddttvxCB+/OHjGNS/bydEZeAEYWZl9tLGak6YPoYbP/DGvTrOsIH9nBw6mROEmZXVy5tqOOeICYwb1nbTkHUtX+ZqZmWzdWcDW2oamDZ2SLlDsSJcgzCzkvnSb/7KXU+ubnF7czoU3NQxQ7soImsPJwgzK5kHnlvPviMGcdKMcS2WGTKgL6cc1PJ2Kx8nCDMriYamZtZs2cnlb5vBVWe+odzhWAe4D8LMSmLtllqaA6aMdv9CpXINwsw6xcYddSx+Zcuu5edfTe5qnjLGCaJSOUGYWbvU1DfS2LznRGMfv3URC1Zs3m1d3z7iwH3cAV2pnCDMLJPahiZ++Mfl/PsDL9BUJEEAfOr0mbz94H13LY8a0p99hnfvEUutZSVNEJJmAd8jmZP6RxFxfQvl3gPcCRwXEQslTQOeBZ5PizwWEZeVMlYza91XfruUn//lFY7dfzRnH77fHtsHD+jLBVVT6N/XXZs9RckShKS+wI3AGcAqYIGkuRGxtKDccOCTwOMFh3gxIo4uVXxm1rKVm2r4ySMraGpu3rVu7uLVHDV5JLdeegKDB3hIi96glDWI44FlEbEcQNLtwPnA0oJyXwVuAD5TwljMrB3uXLiSmx95iVFDXh8Oe+jAfnz9XUc4OfQipUwQk4CVecurgBPyC0g6BpgSEfdIKkwQ0yU9CWwDvhARfyp8AUmzgdkAU6dO7czYzXq1VZt3MmnUYB65+rRyh2JlVMrGwmIzce/q2ZLUB/gucFWRcmuBqRFxDPBp4DZJI/Y4WMSciKiKiKrx48d3UthmtmrzTiaNHlzuMKzMSpkgVgFT8pYnA/mDvg8HDgfmS1oBnAjMlVQVEXUR8RpARCwCXgQOKmGsZpZn1eYaJjtB9HqlbGJaAMyUNB1YDVwIXJTbGBFbgV0DsEiaD3wmvYppPLApIpokHQDMBJaXMFazHmnxyi187/6/0VT8qtQWrd1Wy2TfAd3rlSxBRESjpCuAeSSXud4cEUskXQcsjIi5rex+CnCdpEagCbgsIjaVKlaznuq3T63hTy9s5PBJI9u137FTR3P6IfuUKCqrFCW9DyIi7gXuLVh3bQtlT817/ivgV6WMzaw3WLmphgPGD+Xuy08qdyhWgXxHi1kPtmrzTjcVWYc5QZj1YCs31zDFnc3WQR6Lyaybq6lv5MX11dww7zmWrtmWeb8Attc2ugZhHeYEYdbN/dMdi5m35FX6CN577BT69yt2i1Fx/fr04Z1HTSxhdNaTOUGYdXOvbqvjsIkj+NZ7j+KQCXvcL2pWMu6DMOvmdtY3MWX0ECcH63JOEGbdXHV9I0M8QJ6VgROEWTdXU9/EkIFOENb1nCDMurma+kaGDnB3oXU9JwizbqypOahtaPYcDFYWThBm3djOhiYA1yCsLJwgzLqxmrpGANcgrCycIMy6sZr6tAbhTmorAycIs26suj6tQfR3E5N1PScIs25sp2sQVkZOEGbdWHWaIHyjnJVDpnqrpJOBmRHxk3Q60GER8VJpQzPrfR5b/hp3LFi5a3nNlp0ADPFVTFYGbdYgJH0J+CxwTbqqP3BrloNLmiXpeUnLJF3dSrn3SApJVXnrrkn3e17SWVlez6zS/ef8F7n3mbUsenkzi17ezNqttRw1eSRTxnjIbut6Wb6WvAs4BngCICLWSBre1k6S+gI3AmcAq4AFkuZGxNKCcsOBTwKP5607FLgQOAyYCNwv6aCIaMp0VmYVqLk5WPzKZt79xkn867uPLHc4ZpkSRH1EhKQAkDQ047GPB5ZFxPJ0v9uB84GlBeW+CtwAfCZv3fnA7RFRB7wkaVl6vEczvrZZt7Vs/Q6u+sVi6hqbd1vf1Bxsq23kmCmjyxSZ2e6yJIhfSLoJGCXpY8AlwA8z7DcJWJm3vAo4Ib+ApGOAKRFxj6TPFOz7WMG+kwpfQNJsYDbA1KlTM4RkVn7zlqzjqVVbOfPQfVHB3D+HTRzB6YfuW57AzAq0mSAi4luSzgC2AW8Aro2I+zIcu9i0V7Fro9QH+C5wcXv3zYttDjAHoKqqao/tZt3Rk69s4YDxQ5nzoaq2C5uVUaZLI9KEkCUp5FsFTMlbngysyVseDhwOzFfyNWo/YK6k8zLsa1aRmpqDJ17ZzNvesE+5QzFrU5sJQtJ2Xv/2PoDkKqbqiGhreqsFwExJ04HVJJ3OF+U2RsRWYFze68wHPhMRCyXtBG6T9B2STuqZwF+ynpRZd9PQ1MyjL77G8+u2s6m6nrcf4gRh3V+WJqbdrliS9HckHcZt7dco6QpgHtAXuDkilki6DlgYEXNb2XeJpF+QdGg3Apf7Cibr7h5ZtpHf/XVt0W1/W7eDv6zYBMDIwf057WAnCOv+FNH+pntJj0XEiSWIp8Oqqqpi4cKF5Q7DerHzf/Awz67dzvBBe37v6tNHXHLSdN4ycxz7DB/IPiMGlSFCsz1JWhQRRTvEsjQxvTtvsQ9QRZEOY7PebGd9E0vWbOMf33oA/3zWweUOx6xTZOmkfmfe80ZgBcl9CmY9zkdvWcAzq7e2e7+m5qCxOTh2f9/DYD1Hlj6Ij3RFIGbltqOukfufXc8bp47ioH3bHCxgD8MH9ePNB45ru6BZhWgxQUj6D1ppSoqIT5YkIrMyeWlDNQCzTzmQWYfvV+ZozMqvtRqEe3ytV1m+cQcAB4zPOpqMWc/WYoKIiFu6MhCzcrrxD8u49bGXkWCqR041A7JdxTSeZLjvQ4Fd1+ZFxGkljMusy0QEP374JYYO7MsVb5vBoP6enMcMss0o9zPgWWA68BWSq5gWlDAmsy5T29DEc+ndzZe99UCuOvMN5Q7JrNvIcpnr2Ij4saQrI+Ih4CFJD5U6MLNSW7mphrd/5yHq02G3fYmq2e6yJIiG9OdaSeeSDJo3uXQhmXWNFzfsoL6xmY+cNI0jJ4/kDR24tNWsJ8uSIL4maSRwFfAfwAjgn0oalVkX2FKTfPf5hxP354Dxw8ocjVn3kyVBPJ6OvLoVeFuJ4zHrMpuq6wEYPWRAmSMx656ydFL/WdLvJV0qyY201mNsqamnj2DE4P7lDsWsW2ozQUTETOALwGHAIkn3SPpgySMzK7FNNfWMHNyfvn2KTWBoZllqEETEXyLi0yTzQGwCfBOdVbzN1Q2MHurmJbOWZLlRbgTwLpIZ4Q4E7iLDhEFm5VJT30hdQ3Ob5TZsr3P/g1krsnRSPwXcDVwXEY+WOB6zvbJmy05O/eZ86pvaThAAZx22b4kjMqtcWRLEAdGRaecASbOA75FMOfqjiLi+YPtlwOVAE7ADmB0RSyVNI7l7+/m06GMRcVlHYrDeZd22WuqbmvnACVOZuU/bl66ectD4LojKrDJlmQ+io8mhL3AjcAawClggaW5ELM0rdltE/Hda/jzgO8CsdNuLEXF0R17beq/cXdHnHjGBN8/w3AxmeyNTJ3UHHQ8si4jlEVEP3E7BTHQRsS1vcSieytT2Ui5BDOhXyj9ts96hlP9Fk4CVecur0nW7kXS5pBeBG4D8SYimS3pS0kOS3lLsBSTNlrRQ0sINGzZ0ZuxWoZwgzDpP1uG+PwZMyy8fEZe0tWuRdXvUECLiRuBGSReR3G/xYWAtMDUiXpN0LHC3pMMKahxExBxgDkBVVZVrH0adE4RZp8nSSf0b4E/A/SSdyVmtAqbkLU8mGeivJbcD/wUQEXVAXfp8UVrDOAjPcmdtqG9K/kQH9HWCMNtbWRLEkIj4bAeOvQCYKWk6sJrkPoqL8gtImhkRL6SL5wIvpOvHA5sioknSAcBMYHkHYrBeJtfENNCT/pjttSwJ4h5J50TEve05cEQ0SroCmEdymevNEbFE0nXAwoiYC1wh6XSSIcU3kzQvAZwCXCepkaTWcllEbGrP61vvtKsPwjUIs72WJUFcCXxOUj2vzw0RETGirR3TpHJvwbpr855f2cJ+vwJ+lSE2s924D8Ks82S5D8KzqFjFyN1BPdAJwmyvZalB5G5iOyVdnB8R95QuJLOOy43B5CYms73X5n+RpOtJmpmWpo8r03Vm3U59UzP9+og+HsLbbK9lqUGcAxwdEc0Akm4BngSuLmVgZh1R39js5iWzTpL1P2lU3vORpQjErDPUNza7g9qsk2SpQfwr8KSkP5DcHX0KcE1JozLrICcIs86T5Sqmn0uaDxxHkiA+GxHrSh2YWUfUNzlBmHWWFv+TJB2c/nwjMIFk6IyVwMR0nVm3U9fY5CuYzDpJazWITwOzgW8X2RbAaSWJyGwvJE1MHmbDrDO0mCAiYnb69OyIqM3fJmlQSaMy66A690GYdZos/0l/zrjOrOx8matZ52mxBiFpP5IJfgZLOobX53cYAQzpgtjMMttUXc9z67axqbqe/Ua6gmvWGVrrgzgLuJhkHodv83qC2AZ8rrRhme1uZ30Ti1duIQrmnFq0YjNz/ricHfWN5GZPP3hCm+NImlkGrfVB3ALcIunv09FVzUrmuXXbmPPQcpqi+MSAz6zeyvIN1UW3vemAsVRNG81x08YwoF8fDnGCMOsUWW6UO1bSAxGxBUDSaOCqiPhCaUOz3uQ3i9dw1+LV7D+meOvloP59+dZ7j2Ly6MG7re/ftw9HTxlFX4+9ZNbpsiSIsyNiV5NSRGyWdA7J/NFmnWLTjnr2GT6Q+f/8tnKHYmapLJd79JU0MLcgaTAwsJXyZu32WnU9o4cMKHcYZpYnS4K4FXhA0qWSLgHuA27JcnBJsyQ9L2mZpD1Gf5V0maRnJC2W9LCkQ/O2XZPu97yks7KekFWmTdV1jB3mBGHWnWQZi+kGSc8Abye5kumrETGvrf0k9QVuBM4gGaZjgaS5EbE0r9htEfHfafnzgO8As9JEcSFwGDARuF/SQRHR1L7Ts0qxqbqeI0aParugmXWZTDPKRcTvgN+189jHA8siYjmApNuB80kmHcodd1te+aGw6xrG84HbI6IOeEnSsvR4j7YzBqsQr1XXM3aoaxBm3UmbCULSu4F/A/YhqUEIiIho61rCSSSD++WsAk4ocvzLScZ9GsDr4ztNAh4r2HdSW7Fa91Tf2MxLG4tfogrQ1Bxsr210H4RZN5OlBnED8M6IeLadxy523eEeF7lHxI3AjZIuIrky6sNZ95U0m2RAQaZOndrO8KyrfO3/LeX/Pvpym+Um+A5os24lS4J4tQPJAZJv/VPylicDa1opfzvwX+3ZNyLmAHMAqqqqit9hZWW3dmstk0cP5nPnHNJimf59+/CWmeO6MCoza0uWBLFQ0h3A3UBdbmVE/LqN/RYAMyVNB1aTdDpflF9A0syIeCFdPBfIPZ8L3CbpOySd1DOBv2SI1bqh6rpG9hsxiHOOmFDuUMysHbIkiBFADXBm3roAWk0QEdEo6QpgHtAXuDkilki6DlgYEXOBKySdDjQAm0mal0jL/YKkQ7sRuNxXMFWu6rpGRrl/waziZLnM9SMdPXhE3AvcW7Du2rznV7ay79eBr3f0ta372FHXyOTRHgDYrNJkuYrpJxTvXL6kJBFZj1NT38TQgZ7lzazSZGliuifv+SDgXbTe2Wy2mx11jQwZkOmWGzPrRrI0Me021LeknwP3lywi61Eiguq6RoYNdIIwqzQdmZtxJuCbDiyT2oZmmgOGOkGYVZwsfRDb2b0PYh3w2ZJFZD3KjrpGAIa5D8Ks4rQ2J/VJEfEIMD4iarswJutBauqTBOEahFnlaa2J6fvpzz93RSDWM+VqEO6kNqs8rf3XNqSXuE6W9P3CjRHxydKFZZVsyZqtXPDfj1Lb2Eykc0yPGOQEYVZpWvuvfQdwOskIq4u6JhzrCR5fvonq+ib+8ZQD6N+3D0MH9uPYaaPLHZaZtVOLCSIiNgK3S3o2Ip7qwpiswr2wfgejh/Tn6rMPRio2MK+ZVYIs90E4ORgAW2rqeWrV1jbLLV65hZn7DHdyMKtwbhi2zK777VJ+/eTqTGUvfvO00gZjZiXnBGGZbaqp58DxQ7nhPUe1Wk6CQye0NeGgmXV3rd0H8enWdoyI73R+ONad7axvYuywgRy7vzuczXqD1moQw7ssCqsItY3NjBrcv9xhmFkXae0qpq90ZSDW/dXWNzFoxMByh2FmXSTLWEyDgEuBw0iG+wY8H0RvtLOhicH9PaaSWW+RZTTXnwL7AWcBDwGTge2lDMq6p50NTQwe4ARh1ltkSRAzIuKLQHVE3AKcCxyR5eCSZkl6XtIySVcX2f5pSUslPS3pAUn7521rkrQ4fczNekJWOrUNTQzs5wRh1ltkucy1If25RdLhJMN9T2trJ0l9gRuBM4BVwAJJcyNiaV6xJ4GqiKiR9HHgBuB96badEXF0ttOwrlDrGoRZr5KlBjFH0mjgi8BcYCnJB3lbjgeWRcTyiKgHbgfOzy8QEX+IiJp08TGS5ivrhhqbmmloCvdBmPUiWYba+FH69CHggHYcexKwMm95FXBCK+UvBX6XtzxI0kKgEbg+Iu4u3EHSbGA2wNSpnuSulGobmwGcIMx6kSxXMY0CPkTSrLSrfIbhvosNxBNF1iHpg0AV8Na81VMjYo2kA4AHJT0TES/udrCIOcAcgKqqqqLHts6xs74JgEH9OzJLrZlVoix9EPeSNP88AzS349irgCl5y5OBNYWFJJ0OfB54a0TU5dZHxJr053JJ84FjgBcL97euUduQSxCuQZj1FlkSxKCIaHXYjRYsAGZKmg6sBi4ELsovIOkY4CZgVkSsz1s/GqiJiDpJ44CTyNbvYSWyM00Q7qQ26z2yJIifSvoYcA+Q/w1/U2s7RUSjpCuAeUBf4OaIWCLpOmBhRMwFvgkMA+5Mh4Z+JSLOAw4BbpLUTNKRfn3B1U/WxXI1CPdBmPUeWRJEPckH+ed5vQ8hyNBhHRH3kjRR5a+7Nu/56S3s92cy3mthneuV12pYt612j/XPrt0GuInJrDfJkiA+TXKz3MZSB2Pl1dDUzNnf+yPVaYd0MWOGDujCiMysnLIkiCVATZulrOKt2ryT6vomPn7qgZw8Y9we24cN7MfB+3mQX7PeIkuCaAIWS/oDu/dBtHWZq1WYFRurAXj7wftQNW1MmaMxs3LLkiDuTh/WwzQ3B/VNr1+5vGz9DgCmjRtarpDMrBvJcsZPnKwAABCRSURBVCf1LV0RiHW9i370GI8t3/1itOGD+jHW/QxmRutTjv4iIi6Q9AxF7oCOiCNLGpmVVGNTMwtXbObkGeN484yxu9YfOmEE6SXHZtbLtVaDuDL9+Y6uCMS61uotO2lsDs47aiIXHDel7R3MrNdpcWCdiFibPv1ERLyc/wA+0TXhWam8lHZIu7/BzFqSpZP6DOCzBevOLrLOyqixqZnZP13Emi07M5XfujOZ5mPauCGlDMvMKlhrfRAfJ6kpHCDp6bxNw4FHSh2Ytc+GHXU8+Nx6Dp80gkmjBmfa551HTWT8sIEljszMKlVrNYjbSOZn+Fcgf7rQ7W2Nw2Rdb3N1UiO4/NQZnH3EhDJHY2Y9QYsJIiK2AluB90t6I3AyydVMjwBOEN3Mlp31AIwa4ktUzaxztDn7i6QvArcAY4FxwE8kfaHUgVn7bK1JahCjhvQvcyRm1lNk6aS+CDgmImoBJF0PPAF8rZSBWftsThPEaNcgzKyTZJk/cgUwKG95IJ7Zrdt5vYnJNQgz6xxZahB1wBJJ95H0QZwBPCzp++BB+7qLLTUNDOzXx/M1mFmnyZIg7kofOfNLE4q1pb6xmUeXv0ZD455Tgz+7dpubl8ysU2VJEHcAM0hqDy/m+iKykDQL+B7JlKM/iojrC7Z/Gvgo0AhsAC5J79RG0oeBXGf41zxoIPz6iVVc/etnWtz+xqmjujAaM+vpWrtRrh/wDeAS4GWS/orJkn4CfD4iGlo7sKS+wI0kTVKrgAWS5hbMLf0kUBURNemNeTcA75M0BvgSUEWSmBal+27u6In2BE+v3sqIQf342UdPLLp96hjfFW1mnae1GsQ3Se6anh4R2wEkjQC+lT6ubGVfgOOBZRGxPN33duB8YFeCiIg/5JV/DPhg+vws4L7cDXlp/8cs4OfZTqtnenbtNg6ZMIIjJo8sdyhm1gu0dhXTO4CP5ZIDQERsAz4OnJPh2JOAlXnLq9J1LbmU5M7tzPtKmi1poaSFGzZsyBBSZVq3tZaqr93Pk69s4ZAJI8odjpn1Eq0liIiIYvNANFFkfogiik0qUHQ/SR8kaU76Znv2jYg5EVEVEVXjx4/PEFJlevm1ajbuqOO8oybykZOmlTscM+slWksQSyV9qHBl+mH+XIZjrwLyJxqYDKwpcrzTgc8D50VEXXv27S3q0quWPvSm/dl/rIfnNrOu0VofxOXAryVdAiwi+QZ/HDAYeFeGYy8AZkqaDqwGLiS5K3sXSccANwGzImJ93qZ5wDckjU6XzwSuyfCaPVJtQxOA73Ewsy7V2mB9q4ETJJ0GHEbS7PO7iHggy4EjolHSFSQf9n2BmyNiiaTrgIURMZekSWkYcGc6zeUrEXFeRGyS9FWSJANwXW8eQTZXgxjYL8uN72ZmnaPN+yAi4kHgwY4cPCLuBe4tWHdt3vPTW9n3ZuDmjrxuT+MahJmVg7+SVgDXIMysHPyJUwFeTxCuQZhZ13GCqAC5JqaB/f3rMrOu40+cCuAmJjMrB3/iVIC6hiYG9utDeqWXmVmXcIKoAHWNza49mFmX86dOBahtaGKgL3E1sy7mBFEB6hqbGeQOajPrYv7UqQB1jU2+xNXMupwTRAWobXANwsy6nj91KoBrEGZWDk4QFaC2wVcxmVnX86dOBahrbPJAfWbW5doczdW6TnNz8PTqrdSlQ2sANDYHL2+s4eD9PNWomXUtJ4huZP7f1nPJ/yzcY33fPuLSk6eXISIz682cILqR59ZtB+AnHzmOgX1fb/2bOGow08Z5qlEz61pOEN3Iio3VjBs2kLe9YZ9yh2JmVtpOakmzJD0vaZmkq4tsP0XSE5IaJb2nYFuTpMXpY24p4yyHiNjjsWJjDdPHDSl3aGZmQAlrEJL6AjcCZwCrgAWS5kbE0rxirwAXA58pcoidEXF0qeIrt8tuXcS8Ja/usf6CqslliMbMbE+lbGI6HlgWEcsBJN0OnA/sShARsSLd1lzCOLqd7bUN3P/set4ycxzH7j9613ohzjt6YhkjMzN7XSkTxCRgZd7yKuCEduw/SNJCoBG4PiLuLiwgaTYwG2Dq1Kl7EWrniAjmLVnH1p0NRbbBM6u38sqmGrbXNtLUHHzi1Bm86cCxZYjUzKxtpUwQxWa3iXbsPzUi1kg6AHhQ0jMR8eJuB4uYA8wBqKqqas+xS2Lp2m1cdusTLW7v31ccOmEEffqIsw7bd7fag5lZd1PKBLEKmJK3PBlYk3XniFiT/lwuaT5wDPBiqzuV2bL1OwC47aMnFL0sddigfowY1L+rwzIz65BSJogFwExJ04HVwIXARVl2lDQaqImIOknjgJOAG0oWaSd5+bUaAN64/2gPjWFmFa9kCSIiGiVdAcwD+gI3R8QSSdcBCyNirqTjgLuA0cA7JX0lIg4DDgFuSjuv+5D0QSxt4aXKZnN1PTvqGnctP7t2GxNGDnJyMLMeoaQ3ykXEvcC9BeuuzXu+gKTpqXC/PwNHlDK2Qk3NbXdhfO+BF5i7eDUAzQGrNtdQuNvJM8aVIjwzsy7nO6mBe55ewyd//uQeH/bFnDB9DPuNHATAO46cwAHjh+22/bhp7ng2s57BCQJ4bu12Avj0GQe1Wm7M0AG877gp9O/rUdLNrOdzggB21DUybEA/Pvn2meUOxcys2/BXYaC6rpGhA50rzczyOUEANfVNDB3oK4/MzPI5QZA2MbkGYWa2GycIkiamIQOcIMzM8jlBkNQg3AdhZrY7JwiSPohh7oMwM9uNEwS+isnMrBgnCNxJbWZWTK9PEI1NzdQ1NrsGYWZWoNcniOr6JgAnCDOzAr0+QRBw7pETmLHPsLbLmpn1Ir3+a/PIIf258aI3ljsMM7NuxzUIMzMrygnCzMyKcoIwM7OiSpogJM2S9LykZZKuLrL9FElPSGqU9J6CbR+W9EL6+HAp4zQzsz2VLEFI6gvcCJwNHAq8X9KhBcVeAS4GbivYdwzwJeAE4HjgS5I8l6eZWRcqZQ3ieGBZRCyPiHrgduD8/AIRsSIingaaC/Y9C7gvIjZFxGbgPmBWCWM1M7MCpUwQk4CVecur0nWdtq+k2ZIWSlq4YcOGDgdqZmZ7KmWCUJF10Zn7RsSciKiKiKrx48e3KzgzM2tdKW+UWwVMyVueDKxpx76nFuw7v7UdFi1atFHSy+2Ir9A4YONe7F+JfM69g8+5d+joOe/f0oZSJogFwExJ04HVwIXARRn3nQd8I69j+kzgmtZ2iIi9qkJIWhgRVXtzjErjc+4dfM69QynOuWRNTBHRCFxB8mH/LPCLiFgi6TpJ5wFIOk7SKuC9wE2SlqT7bgK+SpJkFgDXpevMzKyLlHQspoi4F7i3YN21ec8XkDQfFdv3ZuDmUsZnZmYt853Ur5tT7gDKwOfcO/ice4dOP2dFZL2wyMzMehPXIMzMrCgnCDMzK6rXJ4i2BhSsVJJulrRe0l/z1o2RdF86AOJ9ucuIlfh++h48LakiZ1CSNEXSHyQ9K2mJpCvT9T32vCUNkvQXSU+l5/yVdP10SY+n53yHpAHp+oHp8rJ0+7Ryxr83JPWV9KSke9LlHn3OklZIekbSYkkL03Ul/dvu1Qki44CClep/2HP8qquBByJiJvBAugzJ+c9MH7OB/+qiGDtbI3BVRBwCnAhcnv4+e/J51wGnRcRRwNHALEknAv8GfDc9583ApWn5S4HNETED+G5arlJdSXIJfU5vOOe3RcTRefc7lPZvOyJ67QN4EzAvb/ka4Jpyx9WJ5zcN+Gve8vPAhPT5BOD59PlNwPuLlavkB/Ab4Izect7AEOAJklGQNwL90vW7/s5J7kt6U/q8X1pO5Y69A+c6Of1APA24h2R4np5+ziuAcQXrSvq33atrEOzdgIKVaN+IWAuQ/twnXd/j3oe0GeEY4HF6+HmnTS2LgfUkIx+/CGyJ5GZV2P28dp1zun0rMLZrI+4U/w78C6+PBD2Wnn/OAfxe0iJJs9N1Jf3bLumNchVgbwYU7El61PsgaRjwK+BTEbFNKnZ6SdEi6yruvCOiCTha0ijgLuCQYsXSnxV/zpLeAayPiEWSTs2tLlK0x5xz6qSIWCNpH+A+Sc+1UrZTzrm31yD2ZkDBSvSqpAkA6c/16foe8z5I6k+SHH4WEb9OV/f48waIiC0kg1qeCIySlPsCmH9eu8453T4SqLRhbE4CzpO0gmSemdNIahQ9+ZyJiDXpz/UkXwSOp8R/2709QewaUDC94uFCYG6ZYyqluUBu+tYPk7TR59Z/KL3y4URga67aWkmUVBV+DDwbEd/J29Rjz1vS+LTmgKTBwOkkHbd/AHLT+Baec+69eA/wYKSN1JUiIq6JiMkRMY3kf/bBiPgAPficJQ2VNDz3nGQA079S6r/tcne8lPsBnAP8jaTd9vPljqcTz+vnwFqggeTbxKUk7a4PAC+kP8ekZUVyNdeLwDNAVbnj7+A5n0xSjX4aWJw+zunJ5w0cCTyZnvNfgWvT9QcAfwGWAXcCA9P1g9LlZen2A8p9Dnt5/qcC9/T0c07P7an0sST3WVXqv20PtWFmZkX19iYmMzNrgROEmZkV5QRhZmZFOUGYmVlRThBmZlaUE4RVPEk70p/TJF3Uycf+XMHynzvz+J1N0sWSflDuOKxncIKwnmQa0K4EkY7o25rdEkREvLmdMVWUDO+H9SJOENaTXA+8JR0v/5/SQey+KWlBOib+PwJIOlXJvBG3kdxEhKS700HQluQGQpN0PTA4Pd7P0nW52orSY/81HaP/fXnHni/pl5Kek/QzFRkMKi3zb0rmcvibpLek63erAUi6JzfekKQd6T6LJN0v6fj0OMslnZd3+CmS/lfJPCdfyjvWB9PXWyzpplwySI97naTHSUZBNUuU+w5BP/zY2wewI/15KuldtenybOAL6fOBwEJgelquGpieVzZ3B+pgkjuSx+Yfu8hr/T3JyKl9gX2BV0iGWz6VZLTQySRfwB4FTi4S83zg2+nzc4D70+cXAz/IK3cPcGr6PICz0+d3Ab8H+gNHAYvz9l9Lcodt7lyqSAbw+y3QPy33n8CH8o57Qbl/j350v0dvH83VerYzgSMl5cbnGUkygUo98JeIeCmv7CclvSt9PiUt91orxz4Z+HkkI6m+Kukh4DhgW3rsVQDpMNzTgIeLHCM3mOCitExb6oH/TZ8/A9RFRIOkZwr2vy8iXktf/9dprI3AscCCtEIzmNcHdmsiGeDQbDdOENaTCfg/ETFvt5VJk011wfLpJJPK1EiaTzJ+T1vHbkld3vMmWv4/qytSppHdm37z42iIiNzYOM25/SOiOW8UU9hzWOdI470lIq4pEkdtmujMduM+COtJtgPD85bnAR9PhwBH0kHpSJiFRpJMSVkj6WCS4bJzGnL7F/gj8L60n2M8cArJQHB7awXJ3A59JE0hGdK5vc5QMlfxYODvgEdIBnJ7j5K5BHJzGe/fCfFaD+YahPUkTwONkp4imZP7eyRNL0+kHcUbSD4wC/0vcJmkp0mmZnwsb9sc4GlJT0QypHTOXSQduk+RfEP/l4hYlyaYvfEI8BJJE9JfSaYQba+HgZ8CM4DbIiI3wf0XSGYk60Myyu/lwMt7Ga/1YB7N1czMinITk5mZFeUEYWZmRTlBmJlZUU4QZmZWlBOEmZkV5QRhZmZFOUGYmVlR/x+zTbm/9Yk5XwAAAABJRU5ErkJggg==\n" + }, + "metadata": { + "needs_background": "light" + } + } + ], + "source": [ + "d = 20\n", + "dds = DDS(bump, x_min=np.zeros(d), x_max=10*np.ones(d), max_evals=500)\n", + "dds.plot()" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [], + "source": [ + "n_trials = 20\n", + "max_iter = 500\n", + "costs = np.empty((n_trials, max_iter))\n", + "for i in range(n_trials):\n", + " dds = DDS(bump, x_min=np.zeros(d), x_max=10*np.ones(d), max_evals=max_iter)\n", + " costs[i] = dds.run()" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": "" + }, + "metadata": {}, + "execution_count": 6 + }, + { + "output_type": "display_data", + "data": { + "text/plain": "
", + "image/svg+xml": "\r\n\r\n\r\n\r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n\r\n", + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdd3wcd5n48c+zTaveXWTJ3YntmDQcJ6QTEkhCCnA5SkIJgQQO+F04OCDAwR3t6L0dnUASUiAEYxJSSUKaEzvNcYm7ZVmyrN5W25/fHzNy1rIsr2ytVtp93q+XXtqdnZ19ZrSaZ+ZbRVUxxhiTvzzZDsAYY0x2WSIwxpg8Z4nAGGPynCUCY4zJc5YIjDEmz1kiMMaYPGeJIM+IyGwR6RcRbwa2/T8ictMor/+biLS6n1893p8/yud+RkR+OVGflw4ReVhE3p/tONIhIleJyH1ZjuH/RORz2Ywhl1kimORE5GoRWSciIRHZKyI/FZGKMbx/p4icP/RcVRtVtURVE5mJ+JBx+IHvAK93P78jQ59zrog0pS5T1f9V1Slx0p2MVPVmVX390HMRURFZmKnPc7/zjw2L4YOq+qVMfWa+s0QwiYnIx4GvA58AyoHTgDnA/SISyGZsR2A6EATWZzsQkz0i4st2DGYEqmo/k/AHKAP6gbcOW14C7AOucZ//D/BH4DagD3gWOMF97fdAEhh0t/VJYC6ggM9d52Hgy8AT7jp/BaqBm4Fe4Blgbsrnfx/Y7b62Fjgr5bX/AW4aYV+OAQbcz+0HHhoeR0os73cfXw08BnwL6AJ2ABelrFsF/AZodl+/Cyh29zXpfk4/UDc8LuAynITU7X7mkpTXdgL/CbwI9LjHNTjCPhW471+WsqzW/fxpQCWwCmhz41sF1B9iX4fHN/xvVA78CmgB9rh/L+8IMdW5n1+VsuwkoB3wAwuBR9z9agduS/O7eDXwmPv4UTe2Aff4vs1dfgnwvHtMngCOH3ZMP+Ue0wjgA24AtuF8ZzcAb3bXXQKEgYS7/W53+W+BL6ds81pgK9AJrATqUl5T4IPAFvfY/xgQ97UjOga5/mN3BJPX6ThX0HemLlTVfuAe4IKUxZcDd+CcHG8B7hIRv6q+C2gELlWnOOYbh/istwPvAmYBC4AncU6yVcBG4L9T1n0GODHls+4QkeBoO6Kqm4Hj3KcVqnreaOunOBV4GagBvgH8SkTEfe33QJG73WnAd1V1ALgIaHb3t0RVm1M3KCLHAH8APopz4r4b+OuwO6y3AhcC84DjcU6Ew/cpgvO3ecew9z2iqvtw7rZ/g3MHNxvnBP2jNPd7uBuBOM5J7CTg9cBBRV3uvj4J/EvK4iuBP6pqDPgScB9OkqoHfjjWQFT1bPfhCe7xvU1ETgZ+DXwA5yLiZ8BKESlIees7gDfi/P3jOEngLJwk9wXgJhGZqaobcU7iT7rbP6gYVETOA76Kc7xnAruAW4etdglwCnCCu94b3OVHfQxykSWCyasGaHf/aYZrcV8fslZVh/7Zv4OTQE4bw2f9RlW3qWoPTpLZpqoPuJ99B87JBwBVvUlVO1Q1rqrfxrkyPnZsu5a2Xar6C3XqM27E+aefLiIzcU74H1TVLlWNqeojaW7zbcDfVPV+93h9CyjESbxDfqCqzaraiXOHdOIhtnULByaCK91luMfoT6oaUtU+4CvAOWnGuJ+ITMfZ14+q6oCbZL6Lk7xHjclNmm8figmI4SSmOlUNq+pjI29izK4Ffqaqq1U1oao34lz5p34Hf6Cqu1V1EEBV73CPcVJVb8O5el+R5uddBfxaVZ91E/KngdeIyNyUdb6mqt2q2gj8g1f+hpk6BlOaJYLJqx2oOUSZ6kz39SG7hx6oahJowikmSFdryuPBEZ6XDD0RkY+LyEYR6RGRbpwrutSkNJ72Dj1Q1ZD7sARoADpVtesItlmHcwU5tN0kzvGbNdLnAiFS9n+Yh4BCETlVRObgnGz+DCAiRSLyMxHZJSK9OEUqFUfQWmsOTrFOi4h0u8f8Zzh3QSP5I85JsQ44G6eY5J/ua58EBHhaRNaLyDVjjGW0GD8+FJ8bYwMHfgd3p75BRN4tIs+nrL+M9L9Hw/+G/UAH6f0NM3UMpjSruJm8nsS5qnoLcPvQQhEpxrlC/EzKug0pr3twbnmHikTGbXhZETkLp6z3dcB6VU2KSBfOP9ZYDbi/i3DqGwBmpPne3UCViFSoavew1w63v83Aq4aeuFfNDThl72Pi7v/tOFfgrcAq9+of4OM4d0qnqupeETkReI6Rj9UAznEYknocduN8D2oOcXc4PKZut6nnW3HK2/+g6hacq+7FuXpHRM4EHhCRR1V1a9o7PbLdwFdU9SujhTb0wE2av8D5Hj2pqgkReZ5Xjk06f8M5KdsrximSOuzfMIPHYEqzO4JJyi2m+QLwQxG5UET87q3vHThX/L9PWf3VIvIW9+7hozgnjqfc11qB+eMUVilOWXUb4BORz+NUao+Zqrbh/OO+U0S87pXZgjTf24JThPUTEal0j81Q2XUrUC0i5Yd4++3AG0XkdW6T1o/jHK8njmQ/cIpd3oZTXHFLyvJSnLupbhGp4sB6luGeB852+3iU4xR1APv39T7g2yJSJiIeEVkgIqMVM90CvBunrmB/TCLyryJS7z7twjnhHkkz4uHfqV8AH3TvjEREikXkjSJSeoj3F7uf3ebG9V6cO4LU7deP0jLuFuC9InKiWw/xv8BqVd15uMDH8RjkFEsEk5hbufsZnHLsXmA1ztXX69yy0SF/wTkZdeFU+r7FLf8Gp1Ltv9xb8P88ypDuxTkBb8a5NQ8z7JZ/jK7FaRrbgVPpO5aT8btwyns34bSi+iiAqm7CqQze7u7zAUVkqvoy8E6cSsJ24FKcyvTokeyAqq7GuaKvwzk2Q76HU/fQjpOU/z7KNu7HaZ30Ik5LrFXDVnk3EMBpXdOFU/wzc5SwVgKLgFZVfSFl+SnAahHpd9e5XlV3ALjFJFeNurOv+B/gRvf4vlVV1+D8LX/kxreVESrYh6jqBuDbOHe9rTh3aI+nrPIQTquuvSLSPsL7HwQ+B/wJp75sAYeuMxnukMcgnw01qTJTlIj8D7BQVd+Z7ViMMVOT3REYY0yes0RgjDF5zoqGjDEmz9kdgTHG5Lkp14+gpqZG586dm+0wjDFmSlm7dm27qtaO9NqUSwRz585lzZo12Q7DGGOmFBHZdajXrGjIGGPynCUCY4zJc5YIjDEmz025OgJjjElHLBajqamJcDic7VAmVDAYpL6+Hr/fn/Z7LBEYY3JSU1MTpaWlzJ07l1fmM8ptqkpHRwdNTU3Mmzcv7fdZ0ZAxJieFw2Gqq6vzJgkAiAjV1dVjvguyRGCMyVn5lASGHMk+WyIwxpg8Z4nAGGMyRER417vetf95PB6ntraWSy65ZP+yZFKJJ5JE40kisQThWGL/78FonIFInFA0TiyezFicVllsjDEZUlxczEsvvcTg4CDBYJB777uPWbNmoapEYgliiSTJtMf9TOLP0LW73REYY0wGvf4NF/Knu1bSF45z08238OYr3koiCZF4kr7+AT78gWs598zXcOZpp/C3v64EYNeunVx4/ms56zUrOOs1K1j91JMAPPzww5x77rlcccUVLF68mKuuuorxGEHa7giMMTnvC39dz4bm3jG/TxWUV060qdWwi2eW8dk3LkEVRGBoNQWSqiTcS/3L33IFX//qV3j9hRezft063vnuq3nycWdmzm99/aucfe65/Phnv6C7u5vzzj6dc897HbW107hr1T0Eg0G2bd3CNe95F48/tRqA5557jvXr11NXV8cZZ5zB448/zplnnnkkh2U/SwTGmJwWjSeJJ5Ikh105H3QhLc6JXgEOcZGdujiRVCKxw5fbL3vV8TTu2sUfb7+NC95w4QGvPfTgA9xz9yp++L3vAhAJR2ja3ciMmXV84mPXs+7FF/B6vGzdumX/e1asWEF9fT0AJ554Ijt37rREYIzJT7FEku5QjMoiPx4RPB4hmVS6QlFC0QTxRJLBaJxYQvnEGxZnNdaL33gJ//WZT/G3v99PZ2fn/uWqyu9vuY1Fxxx7wPpf/fIXqZ02ncdXryWZTDKtsnT/awUFBfsfe71e4vH4UcdnicAYM6nFE85Vt9cjiAj7+sLEE0p7f4R9vZH9rwX9XuLJ5P6r9OqkEktMjhkY3/nuqykrL+e4Za/in48+sn/5686/gJ/99Cd88zvfQ0R44fnnOOHEk+jt7aVu1iw8Hg+33PR7EolERuOzRGCMmXQi8QR7e8L0R+K0dDu9ZIf6SY1UN5pIKgORo78yzpRZ9fX824f/30HLP/npz3LDJz7O6StORlWZPXsut995F++/7gO868q3cdedf+Ksc86huLg4o/FNuTmLly9frjYxjTFTS8y9qk8klaQqHf1R2vsjVBcXEEsmmVNVhM/rod9tM//y3r60yt9HUx1vY8GiYw+/4hTh8wpFgfSu3Tdu3MiSJUsOWCYia1V1+YjbPvrwjDHmQNF4kr5wjI4B54QfioxctNHRHwVgT9cgChntNGUOzRKBMWZchWMJ1uzsIhxLv1w7agkgqywRGGOO2L6+MJ0DUfrDcRJJJZ5UBqOZrdg0488SgTHmiPSEYmxo7iU+SVrmmCNnicAYM2brm3v2t+YxU5+NNWSMGbN2t5LX5Aa7IzDGjEmmh0TOlIdf3jeu2zv32GmHXeebX/8qf7z9VrxeLx7x8L0f/oSnn36Kq695P0VFRQetf/Pvf8dzz67lW9/9/rjGejgZvSMQkQtF5GUR2SoiN4zw+tUi0iYiz7s/789kPMaYo9MXjrGuqSfbYUwJT69+invvuZtHn3iaJ55+lr/87e/Mqq/npz/6IYOhULbDO0DGEoGIeIEfAxcBS4F3iMjSEVa9TVVPdH9+mal4jDFHLp5I0t4f4bnGbvrCk7cH72Syt6WF6urq/WMDVdfU8Je77qSlpZlLLrqASy68AICbfncjJx+/lItf/zqeevKJrMSayTuCFcBWVd2uqlHgVuDyDH6eMSZDNrb08Xxjt7X3H4Pzzr+APU1NnHz8Uj52/f/jsX8+ygc/9BFmzqxj1T33s+rv97O3pYWvfvmL3PfgI9y16h5e3rQxK7FmMhHMAnanPG9ylw33LyLyooj8UUQaRtqQiFwnImtEZE1bW1smYjXGjKC5e5DHtrTT2msthMaqpKSER55Yzfd/9FNqamp477uu4ubf/+6AddY88zRnnn02NbW1BAIB3nLFv2Yl1kxWFssIy4Y3OP4r8AdVjYjIB4EbgfMOepPqz4GfgzPW0HgHaox5RW84Rkd/lK5QlE5rHXRUvF4vZ519DmedfQ5Lly3jDzf9/qB1REY6VU6sTN4RNAGpV/j1QHPqCqraoaoR9+kvgFdnMB5jzCGEonFaegZ5fnc3T2/vZNu+fksCR2nL5pfZljKhzLoXXqBh9hxKSkvo6+8DYPkpK3js0Ufp7OggFotx151/ykqsmbwjeAZYJCLzgD3A24ErU1cQkZmq2uI+vQzITgGZMXlkIBKnpSeMRyCpzsigLT2DJHO8+D+d5p7jqb9/gE9+/KP09HTj8/mYN38BP/jRT/njHbdxxZsuZcaMmaz6+/3c8NnPcf5rz2LGjJmccOJJGZ97YCQZHYZaRC4Gvgd4gV+r6ldE5IvAGlVdKSJfxUkAcaAT+DdV3TTaNm0YamPGrrU3TFcoSjSepGcwdtRDPE8FNgz1JBmGWlXvBu4etuzzKY8/DXw6kzEYk2/iiSSdA1F8Xg+JpLK3J2yVvWZU1rPYmByzta2fps7BbIdhphAba8iYHNLYEbIk4FKcyeHzzZHssyUCY3JAbzjGpr297OgYyHYok0YcHz3dnXmVDFSVjo4OgsHgmN5nRUPGTCGqSnt/lC2tfSCQTIKizqQwNi/AAfq8ZdDeTXt7+4idmqaSoVxW4PfgOUy/g2AwSH19/Zi2b4nAmCkiHEvwYlMPvYOxbIcyJah46fVVZjuMw9re1s+uzhCD0QR7e8Mkkq8k9r5IjJ5B5yeWUL78pmW887Q54x6DJQJjJpFEUoklksQSSbpDzgnf7/XQ2humrS9ymHebyapnMMaerkHiySShaIKWnjDt/RGauwfZ3fVKnU5FoZ+Az4PHI/g8QmmBjwW1JZQGfbxqVgVnLKzJSHyWCIzJsmRS8XgEVWVH+wA7262cf6pRVSLxJIOxBKFogqbOEHt7w/RH4ry8t4+WnvAB4+sIUFkUYGZFkMtmV3LGgmqKC3wE/d5Dfsb0siDzaoozEr8lAmOyQFXpi8TZ1xthX28YBKLxpJXzT2KqSnNPmIFInEg8STiWoHMgSsdAlCe3dTAYO7hHcMDnYWFtCafMrWLR9BIKfF78XmFGeRCfZ/K01bFEYMwESiaVXZ0hWnoGCUUmfigBk75tbf2sfL6Z3nCM/kicgWhixGG4vSK8ek4lDVWFFAV8FAW8VBYFmF9TjMczNaqpLREYM4GaugbZtq8/22EYl6rSG44zEInTH4mzbk8PuzpC9EfitPVF8HuFeTXFNFQVURzwUVcRpKakgIDPQ8DrobokgN/rwe+dPFf3R8ISgTETpLEjxObWvmyHkbfiiSTxpLKne5A1u7pYvb2DgUiCREo/A49AXUUhVcUBaksLuPT4mdRXHjy3cK6xRGDMBFBVmron1zy1+aCxI8QfnmmkZzBGW19kf4WtV4TjG8qZWRakrNBPWdBPUcBLfWUhFUWBrMacDZYIjMmgWMKpVGztjVidQAZF40m6QlGe393NjvaB/c0zI/EkpUEfx04vZfncSooDPkqCPk5qqEh7JM98YEfCmAza0tpPc7eN/ZMJm1v7eGBjK3u6BukajO2vyK0tLWBWeSFLZpZSXujnrEW1lBTYqW40dnSMGSfhWIJYIkkknqRrIEo8qTb88zhRVfb1RegdjLG+uZe+SJx/bmmjuMC52l9c4GNedTHza4upqyjMdrhTjiUCY45SPJHkxT09dIeiOT/L10QYjCZ4emcnLzZ17x9eoS8cJ558pVI34PNwQn0F7ztz3qidsEx6LBEYc5S6B2M2v+9R6A/HeWBjKzs7B9jRNkA4liShyvTSAqeYp6KQkgIfNSUFlBf5WVBbQnmhP9th5xRLBMYcpe6QJYHDCUXjdIdihOMJwtEkW/b1sWlvH3u6B4nEkijK9LIgJ8+upDTo48TZFcyrLkYOM9KmGR+WCIwZI1Vld+cgOzoGiCeS5NFw92Oiqjy9s5N717fS2Hlg01nBaa+/Ym4VQb+X5XMqmZuhcXTM4aWVCETkTGCRqv5GRGqBElXdkdnQjJl8+sJOZWV/OJ7tUCa19c09bGjp5d71rfg8wptPmkVNSYBCv5eg30t1cYDqkoJsh2lch00EIvLfwHLgWOA3gB+4CTgjs6EZM/m8vLfPksAhRONJdneFePjlNp7c3gHAgtpiPvH6Y/FN8SEYcl06dwRvBk4CngVQ1WYRKc1oVMZMMuFYgqauwf1zBBjHnq5B1rf0sLM9xLo9PftH4Dzv2GlcdkIdRQXew86oZbIvnUQQVVUVEQUQESvIM3lne9uAdQwbZk/3IF/62wbiSaU06GPR9BJOnVvFoumlVBXn3zANU1k6ieB2EfkZUCEi1wLXAL/IbFjGTA5DA5Xt67OOYake29rOH9c2EfR7ueGixUwrLbAr/ynssIlAVb8lIhcAvTj1BJ9X1fszHpkxWRaKxlm9vZNE0poFpXp6Rye/fWInC2tLeM/pc5hRFsx2SOYopdVqyD3x28nf5IWewRi9gzG2tfVbEhhmT/cgdz7XRENlIZ94w7F4p8jEK2Z06bQa6oP9o7cGcFoNDahqWSYDMyYbdneGeHmvzRkw3O6uEC829bDy+WYK/B6uOWOeJYEckk7R0AEthETkTcCKjEVkzAQbjCYYiMbpHIiyp8sqhFO19oa5afUuNrY4yXHpzDKuPWsepUEb4iGXjLlnsareJSI3ZCIYY7Jh095eOmysoIOs3dXFrx/fQTSe5IKl0znv2GnUlARs2IcclE7R0FtSnnpwOpdZwanJCapK96D1DRgSjiV4qbmHNTu7WLOri9qSAt5/1jwW1JZkOzSTQencEVya8jgO7AQuz0g0xkyw1t4IiUR+X9ckk8oLTd209kZ4dEsb+/oilBT4OGthDZefWJeXUzfmm3TqCN47EYEYM9G2tfXT2JF/8whva+tnR/sA3aEY3YNRdnWEaOlx+klMLy3gI69dyLK6MhsWIo8cMhGIyA8ZpQhIVf89IxEZkyGxRJINzb2EYwlEhL5wLG9GDk2q8vLePh7f1s5T2zsB8HmEiiI/VcUBLj1+PstmlVHo91odQB4a7Y5gzYRFYUwGJdyewTvaBghFc38CeVUlmkiSSCoDkQR3r2th9c5OovEkXhEuWDqdi46bQWnQZyf9KUIESoKZmzXgkFtW1Rsz9qnGTIBwLMEzOzuJxPJn/siWnkF+9dgOdqYUeXlFeM2CahZOK2HF3CoCPivymWy8HnFO9gU+PB4h4PUg4iwPeD2UFfqpyeCw3em0GqoFPgUsBfb3JVfV89J474XA9wEv8EtV/doh1rsCuAM4RVXtTsQclf5InPa+CM3u7Ff5YENzLzs7Blj5QjMeES49fiaFAS8Br4fFM8qYUW7DQEwGHg+ICEV+Lz6vh1giyayKQhqqirIaVzr3GjcDtwFvBD4IvAdoO9ybRMQL/Bi4AGgCnhGRlaq6Ydh6pcC/A6vHFroxBwtF4zy9oyOnJ5Hvj8Rp6grRH46zubWftv4I6/b0ADCzPMhHX7fIJn2ZYD6v4Pd6KPB58HiEooCXoM9LSdCH4FzZlwX9eCZpb+x0EkG1qv5KRK5X1UeAR0TkkTTetwLYqqrbAUTkVpxmpxuGrfcl4BvAf44hbmMO0h2Ksm5PT84mgdbeMH9b18Jzjd37x/0PeD0UF3i5/IQ6zj22lqKAz4Z+mGB+n4eTZ1dM6d7W6SSCod42LSLyRqAZqE/jfbOA3SnPm4BTU1cQkZOABlVdJSKHTAQich1wHcDs2bPT+GiTDxJJJZ5MEk8ozd2D7MrBpqBtfREe3NRKU9cgW1r78XmFhbUlvG7JNMqCfmZWBCnwebMdZt6YVlZAdUkBfo8g4rS68opM2iv9dKWTCL4sIuXAx4EfAmXAf6TxvpGOzP7GeiLiAb4LXH24Danqz4GfAyxfvjxPGvyZQxmaPL6xM0Q4llutgAYicba3O5PgtPSEebaxi2g8SV1FIafOr+LiV820YZ8n0NyaIsoKnZO9iFBZ5M/JllbpJILVqtoD9ACvHcO2m4CGlOf1OHcTQ0qBZcDD7oGdAawUkcuswtiMpKVnkM6BKD2DMUKR3EoA4Mz5+4VVG+gccMY9Kg36mFdTzDtOmW2VvROowO+h0O8l6PeycFp+zMqbTiJ4QkR24FQY36mqXWlu+xlgkYjMA/YAbweuHHrRTS41Q89F5GHgPy0JmJHEEkk2t/YTi+deBcDm1j6e393NP7e0MxhL8L4z5vGqWeUZbTduDlYU8LJ4ZlleTrOZzhATi0RkBc6J/LMisgG4VVVvOsz74iLyEeBenOajv1bV9SLyRWCNqq4ch/hNDovEE6xr6iEaTxJNOHUBuUJVeWZnFytfbGZvTxivCA1Vhbx6TiWnza/KyeKHycTjgZnlhRT6nZY9JQU+gv78rWsRHUMfexGpAb4DXKWqWTlqy5cv1zVr7KYhH+TiJDGqytZ9/ax8sZmNLX3Mqihk+dxKXr90ulX6TpCigJfjGyooKcivOy4RWauqy0d6LZ0OZWXAm3HuCBYAf8YmpjEZpKp0hWLs7c29CeNvW7ObBzbuo9Dv5coVszn3mNop3+JkMvN5hVkVhRT4vJQX+SnweQh4PXbMh0knJb4A3AV8UVWfzHA8Js8NROJsbu3LyYliWnvDPLhxH6fOq+KqU2dTFMivK9JME4EZ5UEqiwJUFgXwegSfZ+o37ZwI6XwT5+tYyo+MOQqNnaGcTAL94ThfuXsjInDpCXWWBMZRUcBLVUmAeTXFVrx2hNKpLLYkYCZENJ7MyeIggIc37yMUTfChcxdYP4BxNLu6iGOm50cTz0yyyxIzKcQTSV5s6s6p2cKSqjyw0ekVvHZXFyfUl3Py7MpshzXlFQW8BHweFk0rpbxo6g7rMJlYIjCTwt7eMN2h3Jg7WFXpC8dZs6uL29c0UVnkZ051Ee9YYcOjjEYECv1eZxhmn4figA+PQCyhKMqMsiDFed7MM1PSHYb6WmBu6vqqek3mwjL55PGt7UTiU7+ncCyR5Jmdnax8oZl2t56jvrKQz79xqVVYHsbcmmLqKwvtJJ8l6dwR/AX4J/AAMPX/W82k0heOMTjFZw2LJ5Os3dnF315qobk7TE1JgLef0kBJgY/j68stCRyGxwNzqovw2xzJWZNOIihS1U9lPBKTl6ZyC6FILEF/JM4v/rmDrW39zCgLcu1Z8zipodJmAUtDRZGfZbPK8XkEnyWBrEonEawSkYtV9e6MR2Pyyq6OAbbu6892GGPW0R/hV4/vYHPrK7G/89TZnH1MLR4bGuIgQb+XOdVFlAZ9+L0eAj4PHhE8gg2lMUmkkwiuBz4jIlFemZtAVbUsc2GZXDcYTbCtbeolgYc27eOPa5uc/gDHz6SqOEB1cQFL6+zfYSS1pQUcO6PUyv4nuXT6EVgjXTOutu7rY2f71JpEJplU7t/Yyh1rmzh+VjlXnTrbpoMchQj4vB7mVhdbEpgC0mo+KiKXAWe7Tx9W1VWZC8nkuqnWTHQgEufr926iuTvM/JpiPvLahVYBfAhej1BW6GfxjFKK82xQt6ksneajXwNOwZnEHuB6ETlTVW/IaGQmJ6kqfZF4tsMYk7+tc1oDvf2UBk5fUG1JYAQiTsuf+TUldnymoHRS9sXAiaqaBBCRG4HnAEsEZsx6BmNTqvfw41vbuW9DK2curOH8JdOzHc6kUxTwsrSujNKgH68lgCkr3Xu3CqDTfVyeoVhMjksm9YCWNpPdX57fw19fbGFhbQnvPM16BQ8X8Hk4blY55YU2zMNUl04i+CrwnIj8A2dC+rOBT2c0KpNzEknlucYuegenRv3Ak9s7WCkaxxYAACAASURBVLWuhRPqy7nmjHn4PNbOPVVJ0MfJs62/RK5Ip9XQH9z5hE/BSQSfUtW9mQ7M5Jbm7sEpUUn8YlM3/9zazgu7uzl2einXnjXfWr24vB5hWlkBC6eV4BGxnsA55JCJQEQWq+omETnZXdTk/q4TkTpVfTbz4ZmpTlV5samHtr5ItkM5rHV7evjhQ1upLAqwYl4VV62Yk/dJQARWzKsi6PfaiT+HjXZH8DHgOuDbI7ymwHkZicjkhFgiSTiWYGd7aFIngcFogvs27GVXZ4gXm3qYXlrA5y9dahOc4NwBnDy7ktKg1QHkukMmAlW9zn14kaoeMFuIiNjMGuaQkknlsS3tJJKTv3XQH59t4tHNbZQX+rlgyXTOXzLNkgBQU1pAfWWhjfefJ9KpLH4CODmNZcYA0D0YmxJJ4LnGLh7Z3MZ5i6dxpc0VgNcrVBT6WTKzLO+LxPLNaHUEM4BZQKGInIRTUQxQBhRNQGxmimruHsx2CCPqD8d5cFMrnQNRegZjvNTci98rXLRsRrZDywqvVwh4PVQVB2ioKqLEegLnrdH+8m8ArgbqceoJhhJBL/CZzIZlpprBaIKOgQg9gzH29ky+eYfX7uriptW76I/EqSj0UxTw8dpja7lg6XQqiwLZDm/CeDxw7IwyKgr9NgSE2W+0OoIbgRtF5F9U9U8TGJOZIhJJpbU3TCiaoKkrRHyS9RiOxBO82NTDnq5B/rauhTnVRXzsgmNoqMy/G1q/z0Np0MdSK/YxI0jnkuDVIvKgqnYDiEgl8HFV/a/MhmYmu017e2npnnxX/wAtPYP89JFtNLvxLawt4T/OX0RBHpwEAz4PxQVegn4vHhFmlAWpLM6fux4zdukkgotUdX9RkKp2icjFgCWCPNTeH2EgEqdnMMa+3snXLDSpysMvt/GHpxvxeYUPnbsg56+CPR5YOrOcoN9DYcBrrZ7MmKWTCLwiUqCqEQARKQRsIPY80dEfobEzRFIhFI0TiSWzHdIhqSq/f3IX/9zaztzqIq45Yx51FYXZDmvciYDf6yHo97KgtpjCgJeigJX3myOXzrfnJuBBEfkNTkeya4AbMxqVmRQSSWVH+8CUGBoikVRWvtDMP7e2c/6SafzLyfU50xO2JOjD73XaahwzvdR6+Zpxl85YQ98QkXXA63BaDn1JVe/NeGQmq7a19dPUNUgsPnnvAIbEk0l+8vA2Xmzq4bT5Vbx1eUPOzB1cGPBy2vzqbIdhclxa95Oqeg9wT4ZjMZNELJFkV8cAyUmcA6LxJLev2c1LzT10DERRhStXzOa8xdOyHdq4EIHK4gD1lblXtGUmn3RmKHsL8HVgGs4dgWCT1+e0za19kzIJ9A7GaO4ZZOu+fu5d38pgLMHJsys4bV41c6qLOGl2ZbZDPGolQR+vmlVOwOex4h8zYdK5I/gGcKmqbsx0MCb7IvHEpGwSunZXFz97dBtDI1csnVnGG46bznF1uTNPUlGBlwW1JdbRy0y4dL5xrZYE8kfPJKoYfraxi7++0EwomqBjIMr8mmIuP7GOWRWFVORQb2CvRzhuVhk1xQU236/JinQSwRoRuQ24C9jfcFxV7zzcG0XkQuD7gBf4pap+bdjrHwQ+DCSAfuA6Vd2QfvhmvHUMRLMdAuv29PD41nbWNnYxsyzIMdNLqSzyc9GymRQGcqeNvNcrzK0uprokQJkN9WyyKJ1EUAaEgNenLFNg1EQgIl7gx8AFOJPaPCMiK4ed6G9R1f9z178M+A5wYfrhm/EUSyTZ25vdYqEntrXzmyd2UlLg48SGCt53xryc7Awm4hRvTS+zEd1N9qXTfPS9R7jtFcBWVd0OICK3ApcD+xOBqvamrF+Mk2DMBAtF4/SF42xvGyCRpfGCEknl7pda+MvzzZQGfdxw4eKcPUlOKytgXk2xTfhiJo10Wg0NdSQ7gKpec5i3zgJ2pzxvAk4dYfsfxpkNLcAhZj0TketwZktj9mwbN368JJPKhpberI4W2heOsWZnFw9vbmNP9yCnzK3k2jPn52xZeXGBj+PrK7IdhjEHSKdoaFXK4yDwZqA5jfeN9J88UkL5MfBjEbkSZ/yi94ywzs+BnwMsX77c7hrGycutfVlNAt2hKN99YAt7ugcpCnh57+lzOW1+dc4mAXDuBoyZbNIpGjpgCGoR+QPwQBrbbgIaUp7XM3oCuRX4aRrbNeNgS2sfe7qyN4HMS3t6+MnD24gmkrzvzHmcOrcqpxMAwPSyIHOq8m8IbDP5HUmD5UVAOuUzzwCLRGQesAd4O3Bl6goiskhVt7hP3whswWSUqtIbjtPYGcpaDC/t6eHXj++guiTAB89ZwKwcHBhuOK9XWDS9BJ91EjOTUDp1BH0cWKSzF/jU4d6nqnER+QhwL07z0V+r6noR+SKwRlVXAh8RkfOBGNDFCMVCZnyoKqt3dNIfjmc1jhebuvnBQ1upKg5w3Vnzcz4JVBT5qS4poDToy8nWTyY3jDZn8Rmq+jhQq6pHVJCsqncDdw9b9vmUx9cfyXZNemKJJOFYgkRSaekJZz0JJJLK39fvpTTo40uXH5ez4+aLQNDvJej3sGxWec7up8kdo90R/AB4NfAEcPLEhGPG04bmXtr6Js/kMXc+18Tm1n6uOnV2zp4cReCEhgpqSqxS2EwdoyWCmNt0tF5EfjD8RVX998yFZY5WLJGcVEngnpdauHd9K2curOG1x+bGCKHDeTxw2vxqmyTGTDmjfWMvAc7Hadu/dmLCMeNlskwmE44leHRLG396dg/H1ZXxjlMaDv+mKaCqJMCsikL8Xg/doShdoRhzq4ssCZgp6ZDfWlVtB24VkY2q+sIExmSOQjSepCsUzWr/gCHb2/u5ZXUjOztC1JYU8M5T50z5yeM9HphfU8Kc6iLEnfymyiaGN1NcOv0ILAlMIc3dznj92bRmVye3r2micyCKzyNcc8ZcTl9Qk9WYjpbHA5VFAeZWF1NpJ36TY+w+NseEoomsfn5fOMbvntxFaYGPN51Yx3mLp0354hKvV3j1nEobIdTkrKn9H2oOMhjLXiJo64vwp2ebCMcS3HDhYupypI/AwtoSSwImp43Wj+Bjo71RVb8z/uGYozU4wXcEqspLzb3cvHoX7f3OXAaXHj8zZ5KACMwoz81RUI0ZMtodQemERWGOWjiW4OW9fUTiE5cIntnZyV9eaGZvT5iSAh9vXV7P/JoSFk4rmbAYMsnjgSUzy2zuYJPzRms19IWJDMQcnY6B6IT1G1BVHt/WwW+f2ElDZSFvP6WBk2dX5lzrmXk1Jcwsz407G2NGk85YQ0HgfcBxOMNQA2nNR2AmUO9gZvsNqCrb2wd4fGs7m1v72dsbpqGykM++cQk+T+5dMVcU+ZlXU5ztMIyZEOlUFv8e2AS8AfgicBVgk9lPMplIBElV7t/QSmNniN1dIZq7w3hFWDithDMX1nDq/KqcSwLFBT7m1hRRXWxDRJj8kU4iWKiq/yoil6vqjSJyC86IomaS6I84U02Op/b+CHc+u4end3ZSXRyguMDHW06axWsWVFNZlFtFQEN8XuHEhgoKA1O705sxY5VOIhi61OwWkWU4w1DPzVhEZkwGowle2tMzrtt8fnc3P31kGwJccvxMLj+hbn8v2lxTXuTfn9jmVBdZxbDJS+kkgp+LSCXwOWAlUAJ8fvS3mExTVToHomza2zeuTUYHownuWLOb6WUF/Mf5x+Ts1b/XIwT9XuZUFTGtzJqHmvyWzhATv3QfPgLMz2w4Jh0d/RFebu0jFBn/pqK/e2on7f1Rrn/dopxLApXFfhqqiigO+CgKeHP2LseYsUqn1VAF8G6c4qD969sw1NmRSDoduGLx5Lhvu7l7kDW7unjD0hksrSsb9+1nU2VxgJNnV9jJ35gRpFM0dDfwFLAOGP+zj0lbfyTOro6BcU8CjR0hnt3dxcMvt1FS4OP8JbkxX4AIlBT4KC7wsaC2xJKAMYeQTiIIquqow02YzOoJxdi4t3dcp5pUVZ7Z2cX9G1vZ0T4AwLJZZVxxcj0VOVAkVBTwcuyMUqptpjBjDiutfgQici2wCtjfdVVVOzMWlTlAS+/guCaBSCzBzU838sS2DmaUBXnHKQ2smFdFaY4MrDY0XWRxgY2paEw60vlPiQLfBD4LqLtMsYrjjFNVRGTcZhsLReM819jNU9s72LS3j0teNZPLTqzDk0NFJgV+D0tnllkSMGYM0vlv+RhOp7L2TAeT71SV3sE40USScCzBln19FPi8hMdhaOmNLb386B9bicSTeEV46/IGLlg6fRyinjy8HmFZXblNHGPMGKWTCNYDoUwHYmDdnh729R44cNzR9hGIJ5Ls6gzxuyd3UVHo531nzWNWRSEFvtzqPev1CKfMq6LE7gSMGbN0/msSwPMi8g8OrCOw5qNHKZ5I0t4fJZZI0h2KHZQEjkZXKMq379vM3l5n7uLSoI9/O2cB82tyY4joIcUFPgr8HmZXFVkSMOYIpfOfc5f7Y8ZRz2CMjS3j2xIo1a1P76YzFOXyE+qoLA6wfE4lwSk+cfxw9VWFLJ6RW/0djMmGdHoW3zgRgeSbrfv6xz0JqCpPbe+kqcvpF3DhcTO49IS6cf2MbAr4PBQFvFQUBZhWVmDTRxozTkabqvJ2VX2riKzjldZC+6nq8RmNLIft7gzRNRAd9+3u7Ajxq8d34PMIc6qKeN3iqdkxzOsR5lQX0eW2lir0e6ko8ufM9JfGTDaj3RFc7/6+ZCICySeNneNf997RH+HWZxoRgW9ecfyk7xNQ4PewfE4VitLUNYgqdA5EiSeTVBcXML82t+oyjJnMRpuqssV9+CFV/VTqayLydeBTB7/LHE7nQHTcJ5gPReN8YdUGQtEEi2eUTuokUFzgwyOwpK5s/7j/x0x/ZXrsob4TxpiJk05l8QUcfNK/aIRl5jB6Qk4F8Xh7ZHMboWiCD549n+PrK8Z9++OlwO/h1HlVeDyHPtFbEjBm4o1WR/BvwIeA+SLyYspLpcDjmQ4sF23eN75zBwA829jFyheaOX5WOcvnVo3rtseTzyucNLty1CRgjMmO0e4IbgHuAb4K3JCyvM/GGRq7zoEoPeM0VMRAJM6Gll427e3jn1vamFNdzNWnzx2XbY83r9epuC4v9Fs7f2MmqdHqCHqAHuAdInIycCZO66HHAUsEaUgklY6BCNF4cv8In0drzc5OfvvkTsKxJD6PcFJDJe89Y+6k6iMQ8HnweYQZ5UFmVxXhs+kfjZnU0pmY5nPAW4E73UW/EZE7VPXLGY1sigtF4+xoH6ClO3xU20mqcv+GVja09LKrI0R/JM6C2mLedOIsjp1eOimLWk6aXTGpK6yNMQdK5179SuAkVQ0DiMjXgGcBSwSHEIrGWb29k0TyoO4XY/bU9g7uWNvE9NICXjWrnNlVRZy9qIaCSXQHAE4dQHmhn2llQUsCxkwx6SSCnUAQGLq0LQC2pbNxEbkQ+D7gBX6pql8b9vrHgPcDcaANuEZVd6UV+SS2vW3gqJNA72CMBzft4+8v7WVeTTGfvmjxpB0uuiTo4/j6cooCVgdgzFSUzn9uBFgvIvfj1BFcADwmIj+AQw8+JyJe4Mfu+k3AMyKyUlU3pKz2HLBcVUNuK6VvAG874r3JslA07gwe1zf24qB4Msl961vZ3j5AJJ5gV0eIUDTBq2dX8p7T50zKJFDg97BoWikzyoPZDsUYcxTSSQR/dn+GPJzmtlcAW1V1O4CI3ApcDuxPBKr6j5T1nwLemea2J41kUukNx+gdjLO5te+It/HnZ/dw74ZWZpQHKQ54Oa6ujIuOm8ns6qJxjnh8iMCr51TaXYAxOSCd/+LbgIU4dwPbhuoK0jAL2J3yvAk4dZT134fTXPUgInIdcB3A7Nmz0/z4zNnXG6apexC/x0NvOHZEfQM27e3l0c3trNvTw6A78cw5x9TyrtPmjHe446q2tICZFUFqigsmZUW1MWbsRutQ5gP+F7gG2AV4gHoR+Q3wWVU9XKP4kc4SIxaci8g7geXAOSO9rqo/B34OsHz58qOvgT1KjZ2ho5o+8vnd3fzoH1spDnh59ZxKqosDVJcEOG1+9ThGOX4K/B6Cfq9NAWlMjhrtv/qbOL2I56lqH4CIlAHfcn+uH+W94NwBNKQ8rweah68kIufjzId8jqqO38ws42wgEt9/5d4zeORJ4NnGLn768DZmlAf53BuXTNqZwnxeYUFtCXUVhXjtyt+YnDZaIrgEOEZV91+Bq2qvW6m7icMngmeARSIyD9gDvB2nKep+InIS8DPgQlXddwTxT4iugShrd3Ud9XZe2tPDbx7fSUNVER+/4JhJmwQCPg8nz6m0nsDG5InR/tM1NQmkLEyIyGGLZ1Q1LiIfAe7FaT76a1VdLyJfBNao6kqcu44S4A53sLFGVb3sSHYkU5xJ5PuPejsPv7yPm59upL6ikI+8duGkK2Lx+zxUFPr3zwVgScCY/DHaf/sGEXm3qv4udaFbnr8pnY2r6t3A3cOWfT7l8fljiDUrGjtD9B5FURDA49vauXl1I6+qL+cDZ82fNJ3BqksC+L0eppcFqS0tyHY4xpgsGS0RfBi4U0SuAdbiVPSeAhQCb56A2LIqHEvQ0hOmpefIh4jY3t7P843d3PPSXhbPKOUDZ8/PenFQSdBHSYGPWRWFVBYHshqLMWZyGG3QuT3AqSJyHnAcTiuge1T1wYkKLltUlcbOEI0dRzaTWGNHiN8+uXP/TGTL6sr40LkLCfiyO/haXUUhS2aW2pj/xpgDpDN5/UPAQxMQy6Sxq+PIk0DPYIzvP7QFgIuXzeDcY6dRNQmuvKtKApYEjDEjshrBEXQfQZ1AR3+EBzftY/WOTgajCT5z8WLqK7PbK7jA76GqOEBFUYC68qAlAWPMiCwRDKOqdIeiaa8fTybZ0TbAjU/uoq0/wuLppVywdPqEJoH6qkLKC/2Eogl6BmPE4knKCv3Mry3Oep2EMWbys0QwTFt/hHji8J2Xw7EET2zr4KntHWxvH0AErj9vEctmlY9rPD6vUOj3Eokn8XqEpNuiN+j3EvR5KSv0Mae6eFw/0xiTXywRuFSV1t4I29pG7zOgqjyzs4uHNu1ja1s/JQU+3nXaHJbVlVFdMv5NMI+dUcrM8sJx364xxgyxRAB0h6Js2ttHfzh+2HU37e3j5//cDsDVp8/l9PnVGRt8bXpZ0JKAMSbjLBFA2kkA4NEtbfg8wrf+9YRx7307q7KQskI/Po9QFvRTGLDyfWNM5uV9IugLxw6bBOKJJGsbu3iusZs1u7q4+FUzxi0JeD2C1yNUFPlZPMOadxpjJl5eJ4K9PWHWN/eMuk5/OM6373+Z3V2DFPg8vPbYWi4/YdYRf2ZRwIvHI0wvC9JQWYjPm91OZsYYk9eJoHMgysHD6r2iKxTlS6s20B+Jc91Z81k+t/KIp4yc7Q7kVldhZf7GmMklrxPBYOzQRUItPYP83yPbCceSfPINi1k4reSIP6e2tIBjppce8fuNMSaT8joRhA4xxWRjR4jvPLAZEfjwaxccVRII+DyWBIwxk1reJoJYIkkkljxoeVtfhC/dvQGAz1+ylIYj6CHs93k4saGC8kL/UcdpjDGZlreJ4FBzDt+/sRVV+OQbjh1zEphVWciMsiClQZ9VAhtjpow8TgQHjyf0+LZ2Htq0j7MX1Yy5OKey2M+SmWXjFZ4xxkyYvEwE4ViCPd2DByx7rrGL3z2xiyUzSrlyxey0t+X1CMtmlVNZZMVAxpipKS8Twc6OgQMGlrtvw15uX9PE3OoiPnTuwrSLdY6ZXkpNaYCiQF4eRmNMjsi7M5iqHjD95O6uEH9c28RJDRVce9b8tGcRC/g81FcWZmycIWOMmSh5lwj6InESKXcD96zbS8Dn4erT56adBGZWBJlXU2xJwBiTE/KuaUtPSmuhf2zax9M7OzlnUS3FaY4d5PUIC6eVWHGQMSZn5F8icKeh3NbWz81PN7Ksrow3nZT+2EELakts1i9jTE7Ju8vaXjcRPLmtg4DXwwfPWYA/jcphr1eoLSmgocrGCjLG5Ja8SgThWIJQNEE8mWTNri5OaCgn6B/96r6y2E9tSZAZ5cG06xCMMWYqyatEsKXVmYby2V3d9EfinDqvetT16yoKWVpnncSMMbktry5x48kk/ZE4v3tqJw2VhSwb5SRfEvRx7AwbLM4Yk/vy6o4A4PGt7YRjSa45c96IHcdqSgtoqCyksihgzUONMXkh7xLBuj09NFQWjjigXGVxgBMbKrIQlTHGZE9eFQ0lksqO9oER5xfwecWKgowxeSmvEsGujgEi8STzaw5MBCJw0uzKcZuQ3hhjppK8SgT3bWjF55GDWgLVlhbYJDLGmLyVN4mgcyDKPza1cdr86gNO+h4PzKspzmJkxhiTXXmTCG56ahfRRJILlk4/YPm8mhJKg3Y3YIzJXxlNBCJyoYi8LCJbReSGEV4/W0SeFZG4iFyRyVjevqKBj7x2AbMqnCEiaksLWFJXxpyqsc9JbIwxuSRjiUBEvMCPgYuApcA7RGTpsNUagauBWzIVx5BppUFet8S5G6gs9rNsVjmzKmw+AWOMyWQzmRXAVlXdDiAitwKXAxuGVlDVne5ryQzGcZCGqiK8lgCMMQbIbNHQLGB3yvMmd9mYich1IrJGRNa0tbUdVVAiUF1ccFTbMMaYXJLJRDDSJbeOsOywVPXnqrpcVZfX1tYeVVDFBT67GzDGmBSZTARNQEPK83qgOYOfd1hej1BZFMhmCMYYM+lkMhE8AywSkXkiEgDeDqzM4OcdVnVJgQ0jYYwxw2QsEahqHPgIcC+wEbhdVdeLyBdF5DIAETlFRJqAfwV+JiLrMxUPQHWx3Q0YY8xwGR1cR1XvBu4etuzzKY+fwSkymhCHm43MGGPyUd70LDbGGDMySwTGGJPnLBEYY0yes0RgjDF5zhKBMcbkOUsExhiT5ywRGGNMnrNEYIwxec4SgTHG5DlRPaIBQbNGRNqAXUf49hqgfRzDmQpsn/OD7XN+OJp9nqOqIw7fPOUSwdEQkTWqujzbcUwk2+f8YPucHzK1z1Y0ZIwxec4SgTHG5Ll8SwQ/z3YAWWD7nB9sn/NDRvY5r+oIjDHGHCzf7giMMcYMY4nAGGPyXN4kAhG5UEReFpGtInJDtuMZLyLyaxHZJyIvpSyrEpH7RWSL+7vSXS4i8gP3GLwoIidnL/IjJyINIvIPEdkoIutF5Hp3ec7ut4gEReRpEXnB3ecvuMvnichqd59vc+cHR0QK3Odb3dfnZjP+IyUiXhF5TkRWuc9zen8BRGSniKwTkedFZI27LKPf7bxIBCLiBX4MXAQsBd4hIkuzG9W4+S1w4bBlNwAPquoi4EH3OTj7v8j9uQ746QTFON7iwMdVdQlwGvBh9++Zy/sdAc5T1ROAE4ELReQ04OvAd9197gLe567/PqBLVRcC33XXm4qux5nzfEiu7++Q16rqiSl9BjL73VbVnP8BXgPcm/L808Cnsx3XOO7fXOCllOcvAzPdxzOBl93HPwPeMdJ6U/kH+AtwQb7sN1AEPAucitPL1Ocu3/89B+4FXuM+9rnrSbZjH+N+1rsnvfOAVYDk8v6m7PdOoGbYsox+t/PijgCYBexOed7kLstV01W1BcD9Pc1dnnPHwS0COAlYTY7vt1tM8jywD7gf2AZ0q2rcXSV1v/bvs/t6D1A9sREfte8BnwSS7vNqcnt/hyhwn4isFZHr3GUZ/W77jiLYqURGWJaP7WZz6jiISAnwJ+CjqtorMtLuOauOsGzK7beqJoATRaQC+DOwZKTV3N9Tep9F5BJgn6quFZFzhxaPsGpO7O8wZ6hqs4hMA+4XkU2jrDsu+50vdwRNQEPK83qgOUuxTIRWEZkJ4P7e5y7PmeMgIn6cJHCzqt7pLs75/QZQ1W7gYZz6kQoRGbqgS92v/fvsvl4OdE5spEflDOAyEdkJ3IpTPPQ9cnd/91PVZvf3PpyEv4IMf7fzJRE8AyxyWxwEgLcDK7McUyatBN7jPn4PThn60PJ3uy0NTgN6hm43pxJxLv1/BWxU1e+kvJSz+y0ite6dACJSCJyPU4n6D+AKd7Xh+zx0LK4AHlK3EHkqUNVPq2q9qs7F+X99SFWvIkf3d4iIFItI6dBj4PXAS2T6u53tipEJrIC5GNiMU6762WzHM4779QegBYjhXB28D6ds9EFgi/u7yl1XcFpPbQPWAcuzHf8R7vOZOLe/LwLPuz8X5/J+A8cDz7n7/BLweXf5fOBpYCtwB1DgLg+6z7e6r8/P9j4cxb6fC6zKh/119+8F92f90Lkq099tG2LCGGPyXL4UDRljjDkESwTGGJPnLBEYY0yes0RgjDF5zhKBMcbkOUsEZkoQkX7391wRuXKct/2ZYc+fGM/tjzcRuVpEfpTtOEzusERgppq5wJgSgTv67GgOSASqevoYY5pS0jgeJs9YIjBTzdeAs9yx2v/DHYjtmyLyjDse+wcARORcceYsuAWnow0icpc7kNf6ocG8RORrQKG7vZvdZUN3H+Ju+yV3fPi3pWz7YRH5o4hsEpGbZYSBjtx1vi7OPAKbReQsd/kBV/QismpoPB0R6Xffs1ZEHhCRFe52tovIZSmbbxCRv4szx8Z/p2zrne7nPS8iPxs66bvb/aKIrMYZtdOYV2S7J5392E86P0C/+/tc3F6m7vPrgP9yHxcAa4B57noDwLyUdYd6Yxbi9M6tTt32CJ/1LzijfHqB6UAjzhDA5+KMblmPczH1JHDmCDE/DHzbfXwx8ID7+GrgRynrrQLOdR8rcJH7+M/AfYAfOAF4PuX9LTi9TYf2ZTnOIHR/Bfzuej8B3p2y3bdm++9oP5PzJ19GHzW56/XA8SIyNP5MOc4kHVHgaVXdkbLuv4vIm93HDe56HaNs+0zgD+qM+tkqIo8ApwC97rabANyhoecCj42wjaEB8da66xxOfM11LgAAAaNJREFUFPi7+3gdEFHVmIisG/b++1W1w/38O91Y48CrgWfcG5RCXhmcLIEzSJ8xB7FEYKY6Af6fqt57wEKnqGVg2PPzcSYvCYnIwzjj0xxu24cSSXmc4ND/S5ER1olzYLFsahwxVR0a9yU59H5VTaaMugkHDzWsbrw3qv7/9u6QJYIoCsPwe7YZxGQUi4i/wmY3CP4FxS4I/gmrVbBatcoumBZ2satNMAoiKFzDmYFltCxj2b3v04Zh4KY5M/fC95WzP9bx2Qw06RfPCLRo3oHVmes74LiJpSYitpvUxq41ssrwIyJ2yAjn1lf7fMc9cNicQ6wDu2SgWV/PZK/AICI2yJjhee1F9tiuAPvAiAwjO4jMsW97bjf/Yb1acv4RaNFMge+ImJB9zRfklsm4ObB9I1+MXbfAUURMyTq/h5l7l8A0IsYlo45bN+TB6oT84j4tpbw2g6SPEfBEbv08krWT8xoCV8AWcF1KaUvOz8l2qwGZSHsCvPRcr5ac6aOSVDm3hiSpcg4CSaqcg0CSKucgkKTKOQgkqXIOAkmqnINAkir3A5c1vmg9V1o/AAAAAElFTkSuQmCC\n" + }, + "metadata": { + "needs_background": "light" + } + } + ], + "source": [ + "mean = np.average(costs, axis=0)\n", + "std = np.std(costs, axis=0)\n", + "plt.plot(np.arange(max_iter), mean)\n", + "plt.fill_between(np.arange(max_iter), mean-std, mean+std, alpha=0.3)\n", + "plt.xlabel('Iteration number')\n", + "plt.ylabel('Optimal function value')\n", + "plt.title('Optimal function value vs. iterations')\n", + "plt.legend(['Mean', 'Std'])" ] } ], @@ -19,9 +247,13 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": 3 + "version": "3.7.3-final" }, - "orig_nbformat": 2 + "orig_nbformat": 2, + "kernelspec": { + "name": "python3", + "display_name": "Python 3" + } }, "nbformat": 4, "nbformat_minor": 2 diff --git a/Images/reflected_gaussian.png b/Images/reflected_gaussian.png new file mode 100644 index 0000000..70cb7ee Binary files /dev/null and b/Images/reflected_gaussian.png differ diff --git a/README.md b/README.md index 14554ab..4b517c7 100644 --- a/README.md +++ b/README.md @@ -9,6 +9,7 @@ Global optimization attempts to find the global minima / maxima of a function or * Genetic Algorithms for Path Finding and Travelling Salesman Problem * Multi Objective Optimization using Elitist Non-Dominated Sorting Genetic Algorithm (NSGA-II) * Tabu Search for Travelling Salesman Problem and Capacitated Vehicle Routing Problem +* Dynamically Dimensioned Search (DDS) ## Some visualizations * **Simulated Annealing:** Below we see the comparison between simulated annealing (SA) and greedy search (GS) on an example objective. The search algorithm attempts to minimize the function based on two variables. In the image, green areas are maximas and purple areas are minimas, with the intensity representing magnitude.