I'm working on a regression problem. While tunning the Parameters of SVR I got the following values c=100, gamma= 10 and epsilon =100. For which I got 95 percent r-square. My question is what is the theoretical range of these parameters values.?
-
$\begingroup$ What is $\gamma$? Is this the parameter in your basis function? $\endgroup$ – MachineLearner Mar 22 '19 at 8:44
I support vector regression the inverse regularization parameter $C$ can be selected from the interval $[0,\infty)$. In which $C=0$ means that we are very heavily regularizing and $C\to \infty$ no regularization.
The parameter $\varepsilon$ is also from the interval $[0,\infty)$. In which $\varepsilon=0$ forces the regression to penalize every point that is not exactly on the regression line. Whereas $\varepsilon > 0$ allows an indifference margin around the regression in which a deviation will not be counted as an error.
Additionally, there are slack variables of $\xi\geq 0$ and $\hat{\xi}\geq 0$. These are zero if a point is inside the indifference margin. If a data point lies above and outside the indifference margin we will have $\xi>0$ and if a data point lies below and outside the indifference margin we will have $\hat{\xi}<0$.
I think you mean the form parameter of your radial basis function when you talk about $\gamma$. If we have
$$\varphi(\boldsymbol{x}_i,\boldsymbol{x}_j|\gamma)=\exp\left[-\gamma||\boldsymbol{x}_i-\boldsymbol{x}_j||^2\right]$$
then $\gamma \in (0,\infty)$ (Note the minus sign in front of $\gamma$). For $\gamma \to 0$ we make the kernel flatter as $\varphi \approx 1$. If $\gamma \to \infty$ we will get a very peaked kernel. Which will be 1 when $\boldsymbol{x}_i\approx \boldsymbol{x}_i$ and almost zero everywhere else.
You should also have a look at the documentation for the implementation of these parameters. It might happen that these parameters are not implemented as you think (see note on $\gamma$).