3
$\begingroup$

The "kernel" $k(x,y) = e^{-\|x-y\|} $ is used in the context of SVM. Here $x,y \in \mathbb R^n$ and $\|.\|$ is the Euclidean norm.

Is there a Hilbert space H and a map $\varphi:\mathbb R^n \rightarrow H$ such that $k(x,y) = <\varphi(x),\varphi(y)> $ for all $x,y \in \mathbb R^n$?

$\endgroup$
2
$\begingroup$

Yes, this is guaranteed by the Moore–Aronszajn theorem.

$K(\mathbf{x},\mathbf{y}) = e^{-\|\mathbf{x} - \mathbf{y}\|}$ is a positive definite kernel. This means it is a symmetric function satisfying $$\sum_{i,j=1}^{n} c_i c_j K(\mathbf{x_i},\mathbf{x_j}) \ge 0$$ for all $n \in \mathbb{N}$, all $\mathbf{x_1},\dotsb ,\mathbf{x_n} \in \mathbb{R}^n$, and all $c_1, \dotsb , c_n \in \mathbb{R}$.

The Moore–Aronszajn theorem says that for each such function there exists a unique reproducing kernel Hilbert space with reproducing kernel $K(\mathbf{x},\mathbf{y})$.

That is, there must be a unique Hilbert space $H$ of functions $f: \mathbb{R}^n \to \mathbb{R}$ with a mapping $\phi: \mathbb{R}^n \to H$ such that $K(\mathbf{x},\mathbf{y}) = \langle \phi(\mathbf{x}), \phi(\mathbf{y}) \rangle$ for all $\mathbf{x}, \mathbf{y} \in \mathbb{R}^n$, and such that $K(\mathbf{x},\mathbf{y})$ satisfies the reproducing property: $$\langle f, K(\cdot, \mathbf{x}) \rangle = f(\mathbf{x}) \qquad \forall \mathbf{x} \in \mathbb{R}^n, \enspace \forall f \in H$$

While the reproducing kernel Hilbert space is unique for a given positive definite kernel $K(\mathbf{x}, \mathbf{y})$, there can be other mappings $\phi$ to other Hilbert spaces which also satisfy $K(\mathbf{x},\mathbf{y}) = \langle \phi(\mathbf{x}), \phi(\mathbf{y}) \rangle$ but without the reproducing property.

| improve this answer | |
$\endgroup$
  • $\begingroup$ Tim, do you know a reference where the positive definite condition for $k$ is verified in $\mathbb R^n$ (with $n>1$)? $\endgroup$ – VictorZurkowski Oct 17 '16 at 2:13
  • 1
    $\begingroup$ Yes, this can be shown from a theorem by Schoenberg - see this answer and the references it links to: math.stackexchange.com/questions/248976/… $\endgroup$ – Tim Goodman Oct 17 '16 at 9:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.