# Soft SVM solving for $b$

I don't really understand how to approach this problem. I know that $$w=\sum_{n=1}^{N}{a_ny_nx_n}$$ and $$y_n(w^T\cdot x_n+b)=1.$$ So I can solve for $$b$$ from that equation but I can't figure out how to apply equation(1) that I'm supposed to use.

From $$y^{(n)}(w^T\cdot x^{(n)} + b)=1,$$

since $$y^{(n)}$$ is binary, we have $$w^T\cdot x^{(n)} + b = y^{(n)}$$

That is $$b=y^{(n)}-w^T\cdot x^{(n)}.\tag{2}$$

Now, let's examine

\begin{align}&\sum_{m \in S} \alpha_m y^{(m)}\langle x^{(n)}, x^{(m)}\rangle\\&=\sum_{m \in S} \alpha_m y^{(m)}\langle x^{(n)}, x^{(m)}\rangle + \sum_{m \notin S} 0\cdot y^{(m)}\langle x^{(n)}, x^{(m)}\rangle \\ &=\sum_{m \in S} \alpha_m y^{(m)}\langle x^{(n)}, x^{(m)}\rangle + \sum_{m \notin S} \alpha_m\cdot y^{(m)}\langle x^{(n)}, x^{(m)}\rangle\\ &=\sum_{m=1}^N \alpha_m y^{(m)}\langle x^{(n)}, x^{(m)}\rangle \\ &=\langle \sum_{m=1}^N \alpha_my^{(m)}x^{(m)}, x^{(n)}\rangle\\ &= w^Tx^{(n)} \tag{3}\end{align}

Using $$(2)$$ and $$(3)$$, you should be able to obtain the conclusion.

• Nice derivation. One extra point: the averaging over ${\mathcal{M}}$ is done only for "numerical stability" (this is mentioned in 2006 Bishop's book Page 330), if the optimization process is exact (which is not), $b$ could be calculated using only one point from ${\mathcal{M}}$ without averaging. – Esmailian May 3 '19 at 14:35