The optimisation problem for support vector regression is (see http://alex.smola.org/papers/2003/SmoSch03b.pdf):
minimise: \begin{align*} C\sum_{i=0}^{l}(\xi_{i} +\xi^{*}_{i})+ \frac{1}{2}\lVert w \rVert^{2} \end{align*}
subject to the constraints:
\begin{align*} & y_{i} - <w,x_{i}> - b \leq \epsilon + \xi_{i} \\ & <w,x_{i}> + b - y_{i} \leq \epsilon + \xi^{*}_{i} \\ & \xi_{i}, \xi^{*}_{i} \geq 0 \end{align*}
I do not understand where the $\lVert w \rVert^{2}$ comes from. I understand how the $\lVert w \rVert^{2}$ is derived in the case of support vector classification (by maximizing the margin), but not in the case of regression.
The paper says that the "goal is to find a function $f(x)$ that has at most $\epsilon$ deviation from the actually obtained targets $y_{i}$ for all the training data, and at the same time is as flat as possible. In other words, we do not care about errors as long as they are less than $\epsilon$, but will not accept any deviation larger than this."
But I am not sure what "flat" means.
Does someone know?