From 03adfc2e4fe51ec21d133443074e9b069afe8cf2 Mon Sep 17 00:00:00 2001 From: ZiqianXie Date: Thu, 16 Nov 2017 00:20:00 -0500 Subject: [PATCH] fix a typo in perceptron.ipynb --- chapter02_supervised-learning/perceptron.ipynb | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/chapter02_supervised-learning/perceptron.ipynb b/chapter02_supervised-learning/perceptron.ipynb index faa0c11..3e5865a 100644 --- a/chapter02_supervised-learning/perceptron.ipynb +++ b/chapter02_supervised-learning/perceptron.ipynb @@ -271,10 +271,10 @@ "Here the first equality follows from the definition of the weight updates. The next inequality follows from the fact that $(w^*, b^*)$ separate the problem with margin at least $\\epsilon$, and the last inequality is simply a consequence of iterating this inequality $t+1$ times. Growing alignment between the 'ideal' and the actual weight vectors is great, but only if the actual weight vectors don't grow too rapidly. So we need a bound on their length:\n", "\n", "$$\\begin{eqnarray}\n", - "\\|(w_{t+1}, b_{t+1}\\|^2 \\geq & \\|(w_t, b_t)\\|^2 + 2 y_t x_t^\\top w_t + 2 y_t b_t + \\|(x_t, 1)\\|^2 \\\\\n", + "\\|(w_{t+1}, b_{t+1})\\|^2 \\leq & \\|(w_t, b_t)\\|^2 + 2 y_t x_t^\\top w_t + 2 y_t b_t + \\|(x_t, 1)\\|^2 \\\\\n", "= & \\|(w_t, b_t)\\|^2 + 2 y_t \\left(x_t^\\top w_t + b_t\\right) + \\|(x_t, 1)\\|^2 \\\\\n", - "\\geq & \\|(w_t, b_t)\\|^2 + R^2 + 1 \\\\\n", - "\\geq & (t+1) (R^2 + 1)\n", + "\\leq & \\|(w_t, b_t)\\|^2 + R^2 + 1 \\\\\n", + "\\leq & (t+1) (R^2 + 1)\n", "\\end{eqnarray}$$\n", "\n", "Now let's combine both inequalities. By Cauchy-Schwartz, i.e. $\\|a\\| \\cdot \\|b\\| \\geq a^\\top b$ and the first inequality we have that $t \\epsilon \\leq (w_t, b_t)^\\top (w^*, b^*) \\leq \\|(w_t, b_t)\\| \\sqrt{2}$. Using the second inequality we furthermore get $\\|(w_t, b_t)\\| \\leq \\sqrt{t (R^2 + 1)}$. Combined this yields\n",