4
$\begingroup$

I have 192 countries where each country has some value for 1 million attributes which sum up to 1 (a discrete probability distribution). For any one country most of the values for the attributes are 0.

Now I am trying to find the distance/similarity between those countries using these attributes. I know we can use Jensen Shannon Divergence between two discrete probability distributions to get a distance measure, but the caveat is that all the values have to be non-zero.

Given that there are zero valued attributes for the countries, is there any other suitable statistical distance measure that can help me to cluster these countries using these 1 million attributes?

Improve this question
$\endgroup$
4
  • $\begingroup$ Use regularization so none of the probabilities are precisely zero. You should be using regularization anyway; it reduces variance. Welcome to the site. $\endgroup$
    – Emre
    Jul 3, 2018 at 22:55
  • $\begingroup$ Could you elaborate a bit? Do you mean replacing 0 with very small non-zero value? $\endgroup$
    – Syed Arefinul Haque
    Jul 3, 2018 at 23:41
  • $\begingroup$ For example, while ensuring the result still adds to unity. Read about Bayesian priors and conjugates. $\endgroup$
    – Emre
    Jul 3, 2018 at 23:44
  • $\begingroup$ You can use zero values in the JSD by invoking the limit $\lim \limits_{x\to 0^{+}}x\log (x)=0$ $\endgroup$
    – Mari153
    yesterday

3 Answers 3

Reset to default
1
$\begingroup$

Yes, plenty.

Get the book "encyclopedia of distances".

For example, you can use Histogram Intersection distance. Since your data is already normalized, that reduces to Manhattan distance, if I am not mistaken. Yes: this can be appropriate for distributions.

Improve this answer
$\endgroup$
1
$\begingroup$

I suggest using some kind of smoothing to the probability distribution, e.g., Laplace smoothing (sometimes known as "add-one smoothing"). Then, you will be able to use the Jensen-Shannon distance.

Or, you can use some other distance metric, such as earth-mover's distance or total variation distance. Which metric is most suitable might depend on details of the application domain and the desired interpretation of these distances.

Improve this answer
$\endgroup$
0
$\begingroup$

You could use Kullback–Leibler divergence, aka relative entropy, which is a measure of how one probability distribution diverges from a second, expected probability distribution. It handles attributes that have 0 value.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Kullback–Leibler divergence only handles zeros through applying the limit $\lim \limits_{x\to 0^{+}}x\log (x)=0$. This limit also applies to the Jensen Shannon Divergence (the 'distance' suggested by the OP). So, in the sense of zeros, the KLD is no different to the JSD. $\endgroup$
    – Mari153
    yesterday

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.