I was going through SVM section of Stanford CS229 course notes by Andrew Ng. On page 18 and 19, he explains Lagrangian and its dual:
He first defines the generalized primal optimization problem: $$ \begin{align} \color{red}{ \min_w } & \quad \color{red}{f(w)} \\ s.t. & \quad g_i(w)\leq 0, i=1,...,k \\ & \quad h_i(w)=0, i=1,...,l \end{align} $$ Then, he defines generalized Lagrangian: $$\mathcal{L}(w,\alpha,\beta)=f(w)+\sum_{i=1}^k\alpha_ig_i(w)+\sum_{i=1}^l\beta_ih_i(w)$$ Then, he defines primal in terms of $\mathcal{L}$ $$=\color{red}{\min}_w\underbrace{\color{red}{\max}_{\alpha,\beta:\alpha_i\geq0}\color{red}{\mathcal{L}}(w,\alpha,\beta)}_{\text{call it }\theta_\mathcal{P}(w,b)}$$ (Since $\max\mathcal{L}=f$ when constraints are satisfied, else $\infty$.)
Similarly, he defines dual optimization in terms of $\mathcal{L}$ $$=\color{blue}{\max}_{\alpha,\beta:\alpha_i\geq0}\underbrace{\color{blue}{\min}_w\color{blue}{\mathcal{L}}(w,\alpha,\beta)}_{\text{call it }\theta_\mathcal{D}(\alpha)}$$
Then, on page 21, he defines SVM's primal optimization problem:
$$ \begin{align} \color{red}{ \min_{w,b} } & \quad \underbrace{\color{red}{\frac{1}{2}\Vert w\Vert^2}}_{\text{call it}\color{red}{f}} \\ s.t. & \quad y^{(i)}(w^Tx^{(i)}+b)\geq 1, i=1,...,n \end{align} $$ Then, he defines the SVM's Lagrangian as follows: $$\mathcal{L}=\frac{1}{2}\Vert w\Vert^2-\sum_{i=1}^n\alpha_i[y^{(i)}(w^Tx^{(i)}+b)-1]$$ Then, he minimizes $\mathcal{L}$ with respect to $w$ and $b$ to get: $$\mathcal{L}(w,b,\alpha)=\sum_{i=1}^n\alpha_i-\frac{1}{2}\sum_{i,j=1}^n y^{(i)}y^{(j)}\alpha_i\alpha_j(x^{(i)})^Tx^{(j)}\quad\quad\quad \text{...equation (1)}$$ Then, he gives SVM's dual optimization problem: $$\begin{align} \max_\alpha & \quad W(\alpha)=\sum_{i=1}^n\alpha_i-\frac{1}{2}\sum_{i,j=1}^n y^{(i)}y^{(j)}\alpha_i\alpha_j(x^{(i)})^Tx^{(j)} \\ \text{s.t.} & \quad \alpha_i\geq 0, 0=1,...,n \\ & \quad \sum_{i=1}^n\alpha_iy^{(i)}=0 \\ & & \text{...equation (2)} \end{align}$$
I am unable to map / relate SVM's dual in equation (2) to the dual in blue color. So after a bit thinking, I guess equation (1) is giving
$$W(\alpha) = \theta_{\mathcal{D}}(\alpha) = \color{blue}{\min}_{w,b}\color{blue}{\mathcal{L}}(w,b,\alpha)$$ and SVM's dual is $$\max_\alpha W(\alpha) =\max_\alpha \theta_{\mathcal{D}}(\alpha) = \color{blue}{\max}_{\alpha}\color{blue}{\min}_{w,b}\color{blue}{\mathcal{L}}(w,b,\alpha)$$ I guess this correctly maps with earlier dual in blue color, right?
Rephrasing the doubt, I guess the confusion was that I felt equation (2) is simply renaming $\mathcal{L}(w,b,\alpha)$ in equation (1) as $\max_\alpha W(\alpha)$. But that is not the case right?
Again rephrasing the doubt, equation (2) is:
$$\begin{align}\max_\alpha & \quad \left[W(\alpha)=\sum_{i=1}^n\alpha_i-\frac{1}{2}\sum_{i,j=1}^n y^{(i)}y^{(j)}\alpha_i\alpha_j(x^{(i)})^Tx^{(j)} \right]\\ \text{s.t.} & \quad \alpha_i\geq 0, 0=1,...,n \\ & \quad \sum_{i=1}^n\alpha_iy^{(i)}=0 \end{align} $$
and not:
$$\begin{align}\color{red}{[}\max_\alpha & \quad W(\alpha)\color{red}{]}=\sum_{i=1}^n\alpha_i-\frac{1}{2}\sum_{i,j=1}^n y^{(i)}y^{(j)}\alpha_i\alpha_j(x^{(i)})^Tx^{(j)} \\ \text{s.t.} & \quad \alpha_i\geq 0, 0=1,...,n \\ & \quad \sum_{i=1}^n\alpha_iy^{(i)}=0 \end{align} $$
Am I correct with this understanding?
