Recently I have been studying one-class SVM and am a little bit confused about the offset $\rho$. The common optimization problem is to find a function $f(x)= w^\top x-\rho$ by solving $$\begin{array}{l} \min\limits_{w, \rho} \frac{1}{2}\| w\|^2 + \frac{1}{n\nu}\sum\limits_{i=1}^n\xi_i -\rho\\ \text{s.t. } \ \ w^\top x \geq \rho - \xi_i,\ \ i \in[n]\\ \qquad \xi_i\geq 0, \end{array}$$ whose dual problem is $$\begin{array}{l} \min\limits_{\alpha_i} \frac{1}{2}\sum\limits_i\sum\limits_j\alpha_i\alpha_j x_i^\top x_j\\ \text{s.t. } \ \ \sum\limits_{i=1}^n\alpha_i=1\\ \qquad 0\leq\alpha_i\leq \frac{1}{n\nu}, i\in[n] \end{array}$$

The above dual problem can only give us $ w$ but the $\rho$ has to be determined by KKT condition, which is involved with finding out those $\alpha_i$ between $0$ and $\frac{1}{n\nu}$ (both exclusively). However, my consideration is due to numerical results of those $\alpha_i$'s, what kind of value can be treated as $0$ or $\frac{1}{n\nu}$? Different person may take different threshold or even sometimes we cannot find those $0<\alpha_i<\frac{1}{n\nu}$ due to an improper threshold, so it may cause the offset estimation unstable or troublesome, although maybe we can take the average over those indces $i$ whose corresponding $\alpha_i$ between $0$ and $\frac{1}{n\nu}$ to get an relative nice offset for former case.

Based on this argument, if we can directly set $\rho = 1$? If it is, then we needn't to estimate the $\rho$ any more, which may be safe. If not (or in general not), what will happen?

  • $\begingroup$ Welcome to DataScienceSE. In case nobody answers, I'd suggest asking on stats.stackexchange.com. Theoretical questions often get good answers there. $\endgroup$
    – Erwan
    Jun 13 at 23:29
  • $\begingroup$ Thank you @Erwan and your editing as well as suggestion. $\endgroup$
    – Zhou Wang
    Jun 14 at 0:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.