diff --git a/chapter02_supervised-learning/perceptron.ipynb b/chapter02_supervised-learning/perceptron.ipynb index 1744658..1cd0545 100644 --- a/chapter02_supervised-learning/perceptron.ipynb +++ b/chapter02_supervised-learning/perceptron.ipynb @@ -271,10 +271,10 @@ "Here the first equality follows from the definition of the weight updates. The next inequality follows from the fact that $(w^*, b^*)$ separate the problem with margin at least $\\epsilon$, and the last inequality is simply a consequence of iterating this inequality $t+1$ times. Growing alignment between the 'ideal' and the actual weight vectors is great, but only if the actual weight vectors don't grow too rapidly. So we need a bound on their length:\n", "\n", "$$\\begin{eqnarray}\n", - "\\|(w_{t+1}, b_{t+1}\\|^2 \\geq & \\|(w_t, b_t)\\|^2 + 2 y_t x_t^\\top w_t + 2 y_t b_t + \\|(x_t, 1)\\|^2 \\\\\n", + "\\|(w_{t+1}, b_{t+1})\\|^2 \\leq & \\|(w_t, b_t)\\|^2 + 2 y_t x_t^\\top w_t + 2 y_t b_t + \\|(x_t, 1)\\|^2 \\\\\n", "= & \\|(w_t, b_t)\\|^2 + 2 y_t \\left(x_t^\\top w_t + b_t\\right) + \\|(x_t, 1)\\|^2 \\\\\n", - "\\geq & \\|(w_t, b_t)\\|^2 + R^2 + 1 \\\\\n", - "\\geq & (t+1) (R^2 + 1)\n", + "\\leq & \\|(w_t, b_t)\\|^2 + R^2 + 1 \\\\\n", + "\\leq & (t+1) (R^2 + 1)\n", "\\end{eqnarray}$$\n", "\n", "Now let's combine both inequalities. By Cauchy-Schwartz, i.e. $\\|a\\| \\cdot \\|b\\| \\geq a^\\top b$ and the first inequality we have that $t \\epsilon \\leq (w_t, b_t)^\\top (w^*, b^*) \\leq \\|(w_t, b_t)\\| \\sqrt{2}$. Using the second inequality we furthermore get $\\|(w_t, b_t)\\| \\leq \\sqrt{t (R^2 + 1)}$. Combined this yields\n",