# Understanding SVM Kernels

Following Andrew Ng's machine learning course, he explains SVM kernels by manually selecting 3 landmarks and defining 3 gaussian function based on them. Then he says that we are actually defining 3 new features which are $$f_1$$, $$f_2$$ and $$f_3$$.

$$\hskip0.9in$$

And by applying these 3 gaussian functions on every input data: $$x=(x_1,x_2)\to \hat{x}=(f_1(x), f_2(x), f_3(x))$$

it seems that we are mapping our data from $$\mathbb R^2$$ space to a $$\mathbb R^3$$ space. Now our goal is to find a hyperplane in the 3 dimensional space, where our transformed data is linearly separable. Is my understanding correct? If not, how these 3 new features should be interpreted?

$$\hskip1in$$

In some blog posts, i have read that by using a gaussian kernel, we are mapping our data to an infinite dimensional space (where gaussian kernel computes the dot product of transformed input data) which contradicts with my above understandings.

## 1 Answer

Yep, this is the correct interpretation. The kernels make a difficult classification problem into a much simpler one by making the data linearly separable by transforming it into a higher dimension. I think this image does a good job of illustrating that.

• So how this interpretation is correct when it's said that gaussian kernels map data to an infinite dimensional space? – Mehran Torki Apr 23 at 19:30
• Yes, the kernel maps to infinite space and then also projects it to finite n-dimensional space. This is a pretty good and concise post: shapeofdata.wordpress.com/2013/07/23/gaussian-kernels – LiamFiddler Apr 23 at 19:52
• Thanks, i read the post, but i'm not sure i understand. By using a gaussian kernel $K$, we are implicitly mapping our $\mathbb R^2$ input space to an infinite dimensional space (using a mapping function $\phi(x)$ that $K(x,y)=\phi(x) \cdot \phi(y)$). Then by selecting 3 landmarks $l^{(1)}$, $l^{(2)}$, $l^{(3)}$, we are actually selecting 3 dimensions of that infinite dimensional space. Is that right? Are those dimensions $f_1(x)$, $f_2(x)$ and $f_3(x)$ ? – Mehran Torki Apr 24 at 20:52
• Essentially, yes. We calculate the distance to each landmark, take the Gaussian function of those distances, and the result gives the new coordinates in the N-Dimensional space. – LiamFiddler Apr 27 at 15:18