I have exactely no idea of where to start when it come to cluster distribution and find out similar the similar one. is there a package in R that do the job ?


There are quite a few ways to measure the difference between two distributions. Take a look at this overview article on wikipedia.

One very common way that is used often in Machine and Deep Learning is the Kullback-Leibler (KL) Divergence. It is most commonly used in minimising the cross-entropy between the distribution of your training data and the expected (generalised) distribution of the problem you are analysing. In general, a value close to 0 indicates that two distributions are expected to show similar behaviour, while a large value indicates the distributions behave very differently - so knowing the first distribution doesn't help you know anything about the second.

A more general class of measures of dissimilarity between distributions is the so called $f$-divergence, "an average, weighted by the function $f$ of the odds ratio given by [probability measures] $P$ and $Q$". The Wikipedia article contains a formal definition and a table with functions $f$ that give some popular dissimilarities/distances, including the KL-divergence.

The following table lists many of the common divergences between probability distributions and the $f$ function to which they correspond (cf. Liese & Vajda (2006))

Distribution distance metrics

KL-Divergence is combined with the entropy itself, to define the cross-entropy. Take a look here, under the Information Theory View, for a bit more info.


Use the popular K-means clustering algorithm combined with Hellinger distance as a metric of distance.

Hellinger distance quantifies the similarity between two distributions / histograms, thus it can be very easily merged with K-means for your purpose :)

  • $\begingroup$ Can you say anything more about why you recommend the Hellinger distance (as opposed to any other distance metric)? $\endgroup$ – D.W. Jun 12 '18 at 23:13
  • $\begingroup$ to be honest I am more biased towards it because of some publications I have been recently reading that explicitly used it, such as this one link.springer.com/article/10.1007/s10618-017-0538-6. I don't recommend it over other metrics, I suggested it as a starting point because OP said he has no idea where to start and it is actually a concrete starting point when combined with K-means for his problem :) $\endgroup$ – pcko1 Jun 12 '18 at 23:21
  • $\begingroup$ actually, from the publication I cited above: "It is a proper metric, it is quick to compute and it has no problem with empty bins". Hope it helps ! $\endgroup$ – pcko1 Jun 12 '18 at 23:23

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.