3
$\begingroup$

The "kernel" $k(x,y) = e^{-\|x-y\|} $ is used in the context of SVM. Here $x,y \in \mathbb R^n$ and $\|.\|$ is the Euclidean norm.

Is there a Hilbert space H and a map $\varphi:\mathbb R^n \rightarrow H$ such that $k(x,y) = <\varphi(x),\varphi(y)> $ for all $x,y \in \mathbb R^n$?

$\endgroup$
0

1 Answer 1

2
$\begingroup$

Yes, this is guaranteed by the Moore–Aronszajn theorem.

$K(\mathbf{x},\mathbf{y}) = e^{-\|\mathbf{x} - \mathbf{y}\|}$ is a positive definite kernel. This means it is a symmetric function satisfying $$\sum_{i,j=1}^{n} c_i c_j K(\mathbf{x_i},\mathbf{x_j}) \ge 0$$ for all $n \in \mathbb{N}$, all $\mathbf{x_1},\dotsb ,\mathbf{x_n} \in \mathbb{R}^n$, and all $c_1, \dotsb , c_n \in \mathbb{R}$.

The Moore–Aronszajn theorem says that for each such function there exists a unique reproducing kernel Hilbert space with reproducing kernel $K(\mathbf{x},\mathbf{y})$.

That is, there must be a unique Hilbert space $H$ of functions $f: \mathbb{R}^n \to \mathbb{R}$ with a mapping $\phi: \mathbb{R}^n \to H$ such that $K(\mathbf{x},\mathbf{y}) = \langle \phi(\mathbf{x}), \phi(\mathbf{y}) \rangle$ for all $\mathbf{x}, \mathbf{y} \in \mathbb{R}^n$, and such that $K(\mathbf{x},\mathbf{y})$ satisfies the reproducing property: $$\langle f, K(\cdot, \mathbf{x}) \rangle = f(\mathbf{x}) \qquad \forall \mathbf{x} \in \mathbb{R}^n, \enspace \forall f \in H$$

While the reproducing kernel Hilbert space is unique for a given positive definite kernel $K(\mathbf{x}, \mathbf{y})$, there can be other mappings $\phi$ to other Hilbert spaces which also satisfy $K(\mathbf{x},\mathbf{y}) = \langle \phi(\mathbf{x}), \phi(\mathbf{y}) \rangle$ but without the reproducing property.

$\endgroup$
2
  • $\begingroup$ Tim, do you know a reference where the positive definite condition for $k$ is verified in $\mathbb R^n$ (with $n>1$)? $\endgroup$ Oct 17, 2016 at 2:13
  • 1
    $\begingroup$ Yes, this can be shown from a theorem by Schoenberg - see this answer and the references it links to: math.stackexchange.com/questions/248976/… $\endgroup$ Oct 17, 2016 at 9:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.