Understanding SVM mathematics

Question

I was referring SVM section of Andrew Ng's course notes for Stanford CS229 Machine Learning course. On pages 14 and 15, he says:

Consider the picture below:

How can we find the value of $\gamma^{(i)}$? Well, $w/\Vert w\Vert$ is a unit-length vector pointing in the same direction as $w$. Since, point $A$ represents $x^{(i)}$, we therefore find that the point $B$ is given by $x^{(i)} − \gamma^{(i)}·w/\Vert w\Vert$. But this point lies on the decision boundary, and all points $x$ on the decision boundary satisfy the equation $w^Tx + b = 0$. Hence, $$w^T\left(x^{(i)}-\gamma^{(i)}\frac{w}{\Vert w \Vert}\right)+b=0$$ Solving for $\gamma^{(i)}$ yields $$\color{red}{\gamma^{(i)}=\frac{w^Tx^{(i)}+b}{\Vert w\Vert}}$$

I am not getting how the last red-colored equality is arrived. I am getting something like this: $$w^T\left(x^{(i)}-\gamma^{(i)}\frac{w}{\Vert w \Vert}\right)+b=0$$ $$\rightarrow w^Tx^{(i)}-\gamma^{(i)}\frac{w^Tw}{\Vert w \Vert}+b=0$$ $$\rightarrow w^Tx^{(i)}+b=\gamma^{(i)}\frac{w^Tw}{\Vert w \Vert}$$

How can I proceed further to equality in red color? Do I have to divide both the sides again by $\Vert w \Vert$ to get the following? $$\rightarrow \frac{w^Tx^{(i)}+b}{\Vert w \Vert}=\gamma^{(i)}\frac{w^Tw}{\Vert w \Vert\Vert w \Vert}$$

But then how $\frac{w^Tw}{\Vert w \Vert\Vert w \Vert}$ equals to $1$?

Answer 1

Hint: $w^Tw = \Vert w \Vert^2$ this stems directly from the definitions of norm and matrix product (assuming $w$ is column vector as usually taken) and one can expand the two sides to prove it easily.

Note that technically $w^Tw$ is a $1 \times 1$ matrix but any such matrix is identified with its single scalar entry. So it is simply a scalar number. Or equivalently any scalar value is also a $1 \times 1$ matrix.