# Understanding SVM's Lagrangian dual optimization problem

I was going through SVM section of Stanford CS229 course notes by Andrew Ng. On page 18 and 19, he explains Lagrangian and its dual:

He first defines the generalized primal optimization problem: \begin{align} \color{red}{ \min_w } & \quad \color{red}{f(w)} \\ s.t. & \quad g_i(w)\leq 0, i=1,...,k \\ & \quad h_i(w)=0, i=1,...,l \end{align} Then, he defines generalized Lagrangian: $$\mathcal{L}(w,\alpha,\beta)=f(w)+\sum_{i=1}^k\alpha_ig_i(w)+\sum_{i=1}^l\beta_ih_i(w)$$ Then, he defines primal in terms of $$\mathcal{L}$$ $$=\color{red}{\min}_w\underbrace{\color{red}{\max}_{\alpha,\beta:\alpha_i\geq0}\color{red}{\mathcal{L}}(w,\alpha,\beta)}_{\text{call it }\theta_\mathcal{P}(w,b)}$$ (Since $$\max\mathcal{L}=f$$ when constraints are satisfied, else $$\infty$$.)

Similarly, he defines dual optimization in terms of $$\mathcal{L}$$ $$=\color{blue}{\max}_{\alpha,\beta:\alpha_i\geq0}\underbrace{\color{blue}{\min}_w\color{blue}{\mathcal{L}}(w,\alpha,\beta)}_{\text{call it }\theta_\mathcal{D}(\alpha)}$$

Then, on page 21, he defines SVM's primal optimization problem:

\begin{align} \color{red}{ \min_{w,b} } & \quad \underbrace{\color{red}{\frac{1}{2}\Vert w\Vert^2}}_{\text{call it}\color{red}{f}} \\ s.t. & \quad y^{(i)}(w^Tx^{(i)}+b)\geq 1, i=1,...,n \end{align} Then, he defines the SVM's Lagrangian as follows: $$\mathcal{L}=\frac{1}{2}\Vert w\Vert^2-\sum_{i=1}^n\alpha_i[y^{(i)}(w^Tx^{(i)}+b)-1]$$ Then, he minimizes $$\mathcal{L}$$ with respect to $$w$$ and $$b$$ to get: $$\mathcal{L}(w,b,\alpha)=\sum_{i=1}^n\alpha_i-\frac{1}{2}\sum_{i,j=1}^n y^{(i)}y^{(j)}\alpha_i\alpha_j(x^{(i)})^Tx^{(j)}\quad\quad\quad \text{...equation (1)}$$ Then, he gives SVM's dual optimization problem: \begin{align} \max_\alpha & \quad W(\alpha)=\sum_{i=1}^n\alpha_i-\frac{1}{2}\sum_{i,j=1}^n y^{(i)}y^{(j)}\alpha_i\alpha_j(x^{(i)})^Tx^{(j)} \\ \text{s.t.} & \quad \alpha_i\geq 0, 0=1,...,n \\ & \quad \sum_{i=1}^n\alpha_iy^{(i)}=0 \\ & & \text{...equation (2)} \end{align}

I am unable to map / relate SVM's dual in equation (2) to the dual in blue color. So after a bit thinking, I guess equation (1) is giving

$$W(\alpha) = \theta_{\mathcal{D}}(\alpha) = \color{blue}{\min}_{w,b}\color{blue}{\mathcal{L}}(w,b,\alpha)$$ and SVM's dual is $$\max_\alpha W(\alpha) =\max_\alpha \theta_{\mathcal{D}}(\alpha) = \color{blue}{\max}_{\alpha}\color{blue}{\min}_{w,b}\color{blue}{\mathcal{L}}(w,b,\alpha)$$ I guess this correctly maps with earlier dual in blue color, right?

Rephrasing the doubt, I guess the confusion was that I felt equation (2) is simply renaming $$\mathcal{L}(w,b,\alpha)$$ in equation (1) as $$\max_\alpha W(\alpha)$$. But that is not the case right?

Again rephrasing the doubt, equation (2) is:

\begin{align}\max_\alpha & \quad \left[W(\alpha)=\sum_{i=1}^n\alpha_i-\frac{1}{2}\sum_{i,j=1}^n y^{(i)}y^{(j)}\alpha_i\alpha_j(x^{(i)})^Tx^{(j)} \right]\\ \text{s.t.} & \quad \alpha_i\geq 0, 0=1,...,n \\ & \quad \sum_{i=1}^n\alpha_iy^{(i)}=0 \end{align}

and not:

\begin{align}\color{red}{[}\max_\alpha & \quad W(\alpha)\color{red}{]}=\sum_{i=1}^n\alpha_i-\frac{1}{2}\sum_{i,j=1}^n y^{(i)}y^{(j)}\alpha_i\alpha_j(x^{(i)})^Tx^{(j)} \\ \text{s.t.} & \quad \alpha_i\geq 0, 0=1,...,n \\ & \quad \sum_{i=1}^n\alpha_iy^{(i)}=0 \end{align}

Am I correct with this understanding?

In that vein, I think Ng's notation would be more informative in equation (1) if they wrote the lagrangian as $$L(\alpha)$$ rather than $$L(w,b,\alpha)$$, since we have already minimized over $$w$$ and $$b$$. Notice how they are not present in the expression anymore?