I am a starter on data mining. I have question that why in graph based clustering, it's preferred to use things like Gaussian Kernel as similarity metric, rather than just use the distance between two points as the similarity metric?

  • 1
    $\begingroup$ I have an idea that for similarity, we want it to between 0and 1. Gaussian Kernel satisfies this and the weight becomes bigger when the distance between two points becomes bigger. Is there any other reason? $\endgroup$ – zfb Mar 4 '17 at 1:04
  • $\begingroup$ here you can watch a video which explain the function very well > coursera.org/lecture/machine-learning/… $\endgroup$ – Jozani Hosein Oct 11 '18 at 10:30

Let's be precise. "Distance" has lots of meanings in data science, I think you're talking about Euclidean distance.

The Gaussian kernel is a non-linear function of Euclidean distance.

enter image description here

  • The kernel function decreases with distance and ranges between zero and one. In euclidean distance, the value increases with distance. Thus, the kernel function is a more useful metrics for weighting observations.

  • The fact that it's bounded between zero and one is a nice property, whereas the absolute distance (it can be anything) in Euclidean distance can cause instability and difficulty in modelling.

  • Euclidean distance (without the negative sign) is not a similarity measure, it's a distance function. The gaussian kernel is a similarity measure.

  • You can think the Gaussian kernel like a normalization function for Euclidean distance.

  • 1
    $\begingroup$ I also have another question about the σ in the expression. Does it have any meaning? In my opinion, I think it might be related to the scale of the clustering (for example, the radius a circular clustering). $\endgroup$ – zfb Mar 5 '17 at 3:55
  • $\begingroup$ @zfb It's a scaling parameter. The denominator can be written like a constant. $\endgroup$ – SmallChess Mar 5 '17 at 3:56
  • $\begingroup$ So how this scaling parameter affect the value of K(x, x') or the similarity? If it becomes bigger, then K(x,x') becomes bigger, can I say distance is being scaled smaller? And in this case, we are looking at large scale clustering (for example, if the cluster is identified by circle, then the radius of the circlar should be big, or several points together are redifined to be aggregate "points", and then cluster those aggregated points ), rather than looking at a smaller one? $\endgroup$ – zfb Mar 5 '17 at 4:08

From the euclidian distance, you can derive many similarity meausures from kernel functions (polynomial, exponential, Matern, custom...), of which none is a priori better or worse than the gaussian kernel. It all depends on your data and what you expect.

Given a kernel function, you can also choose any definition of distance thats suits your feeling : weighted euclidian distance, $L^1$ norm, $L^{\infty}$ norm, earth mover's distance...

Now, the gaussian kernel with euclidian distance is very common as it is quite intuitive, and provides useful properties such as smoothness.


Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.