4
$\begingroup$

I have 192 countries where each country has some value for 1 million attributes which sum up to 1 (a discrete probability distribution). For any one country most of the values for the attributes are 0.

Now I am trying to find the distance/similarity between those countries using these attributes. I know we can use Jensen Shannon Divergence between two discrete probability distributions to get a distance measure, but the caveat is that all the values have to be non-zero.

Given that there are zero valued attributes for the countries, is there any other suitable statistical distance measure that can help me to cluster these countries using these 1 million attributes?

$\endgroup$
  • $\begingroup$ Use regularization so none of the probabilities are precisely zero. You should be using regularization anyway; it reduces variance. Welcome to the site. $\endgroup$ – Emre Jul 3 '18 at 22:55
  • $\begingroup$ Could you elaborate a bit? Do you mean replacing 0 with very small non-zero value? $\endgroup$ – Syed Arefinul Haque Jul 3 '18 at 23:41
  • $\begingroup$ For example, while ensuring the result still adds to unity. Read about Bayesian priors and conjugates. $\endgroup$ – Emre Jul 3 '18 at 23:44
1
$\begingroup$

Yes, plenty.

Get the book "encyclopedia of distances".

For example, you can use Histogram Intersection distance. Since your data is already normalized, that reduces to Manhattan distance, if I am not mistaken. Yes: this can be appropriate for distributions.

| improve this answer | |
$\endgroup$
1
$\begingroup$

I suggest using some kind of smoothing to the probability distribution, e.g., Laplace smoothing (sometimes known as "add-one smoothing"). Then, you will be able to use the Jensen-Shannon distance.

Or, you can use some other distance metric, such as earth-mover's distance or total variation distance. Which metric is most suitable might depend on details of the application domain and the desired interpretation of these distances.

| improve this answer | |
$\endgroup$
0
$\begingroup$

You could use Kullback–Leibler divergence, aka relative entropy, which is a measure of how one probability distribution diverges from a second, expected probability distribution. It handles attributes that have 0 value.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.