2
$\begingroup$

enter image description here

I don't really understand how to approach this problem. I know that $w=\sum_{n=1}^{N}{a_ny_nx_n}$ and $y_n(w^T\cdot x_n+b)=1. $ So I can solve for $b$ from that equation but I can't figure out how to apply equation(1) that I'm supposed to use.

$\endgroup$
2
$\begingroup$

From $$y^{(n)}(w^T\cdot x^{(n)} + b)=1,$$

since $y^{(n)}$ is binary, we have $$w^T\cdot x^{(n)} + b = y^{(n)}$$

That is $$b=y^{(n)}-w^T\cdot x^{(n)}.\tag{2}$$

Now, let's examine

\begin{align}&\sum_{m \in S} \alpha_m y^{(m)}\langle x^{(n)}, x^{(m)}\rangle\\&=\sum_{m \in S} \alpha_m y^{(m)}\langle x^{(n)}, x^{(m)}\rangle + \sum_{m \notin S} 0\cdot y^{(m)}\langle x^{(n)}, x^{(m)}\rangle \\ &=\sum_{m \in S} \alpha_m y^{(m)}\langle x^{(n)}, x^{(m)}\rangle + \sum_{m \notin S} \alpha_m\cdot y^{(m)}\langle x^{(n)}, x^{(m)}\rangle\\ &=\sum_{m=1}^N \alpha_m y^{(m)}\langle x^{(n)}, x^{(m)}\rangle \\ &=\langle \sum_{m=1}^N \alpha_my^{(m)}x^{(m)}, x^{(n)}\rangle\\ &= w^Tx^{(n)} \tag{3}\end{align}

Using $(2)$ and $(3)$, you should be able to obtain the conclusion.

$\endgroup$
  • 1
    $\begingroup$ Nice derivation. One extra point: the averaging over ${\mathcal{M}}$ is done only for "numerical stability" (this is mentioned in 2006 Bishop's book Page 330), if the optimization process is exact (which is not), $b$ could be calculated using only one point from ${\mathcal{M}}$ without averaging. $\endgroup$ – Esmailian May 3 '19 at 14:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.