1
$\begingroup$

Recently I have been studying one-class SVM and am a little bit confused about the offset $\rho$. The common optimization problem is to find a function $f(x)= w^\top x-\rho$ by solving $$\begin{array}{l} \min\limits_{w, \rho} \frac{1}{2}\| w\|^2 + \frac{1}{n\nu}\sum\limits_{i=1}^n\xi_i -\rho\\ \text{s.t. } \ \ w^\top x \geq \rho - \xi_i,\ \ i \in[n]\\ \qquad \xi_i\geq 0, \end{array}$$ whose dual problem is $$\begin{array}{l} \min\limits_{\alpha_i} \frac{1}{2}\sum\limits_i\sum\limits_j\alpha_i\alpha_j x_i^\top x_j\\ \text{s.t. } \ \ \sum\limits_{i=1}^n\alpha_i=1\\ \qquad 0\leq\alpha_i\leq \frac{1}{n\nu}, i\in[n] \end{array}$$

The above dual problem can only give us $ w$ but the $\rho$ has to be determined by KKT condition, which is involved with finding out those $\alpha_i$ between $0$ and $\frac{1}{n\nu}$ (both exclusively). However, my consideration is due to numerical results of those $\alpha_i$'s, what kind of value can be treated as $0$ or $\frac{1}{n\nu}$? Different person may take different threshold or even sometimes we cannot find those $0<\alpha_i<\frac{1}{n\nu}$ due to an improper threshold, so it may cause the offset estimation unstable or troublesome, although maybe we can take the average over those indces $i$ whose corresponding $\alpha_i$ between $0$ and $\frac{1}{n\nu}$ to get an relative nice offset for former case.

Based on this argument, if we can directly set $\rho = 1$? If it is, then we needn't to estimate the $\rho$ any more, which may be safe. If not (or in general not), what will happen?

$\endgroup$
2
  • $\begingroup$ Welcome to DataScienceSE. In case nobody answers, I'd suggest asking on stats.stackexchange.com. Theoretical questions often get good answers there. $\endgroup$
    – Erwan
    Jun 13, 2021 at 23:29
  • $\begingroup$ Thank you @Erwan and your editing as well as suggestion. $\endgroup$
    – Zhou Wang
    Jun 14, 2021 at 0:32

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Browse other questions tagged or ask your own question.