1
$\begingroup$

I have data census(name, sex, age, capital_gain) and I want to plot all possible views (in a histogram) and find the most interesting view based on the distinction to other views.

So, I have to compute the similarity between two histogram plots, for example:

  1. Average Capital Gain vs Sex (which computed by aggregate Capital gain to Sex)

Capital gain vs Sex

  1. Average age vs Sex (which computed by aggregate average age to sex)

Age vs Sex

My questions is, what kind of measurement that can be used to check the similarity between those two histogram plots? I consider the features that want to be compared are (x-axis, y-axis, and pattern of the histogram)

I really not sure about distance function that can be used for this case. I know Jaccard distance, but not sure it can be used in this situation or not.

Thank you,

$\endgroup$
1
$\begingroup$

I'm curious what other people will say, but one option is to use KL-divergence. If your two histograms have the same x-axis, you can divide every column by the total count to convert counts to proportions. Then you could treat the histograms as probability mass, and use the KL-divergence. KL-divergence is really a measure of the distance between two probability distributions, but histograms are an approximation of a discrete distribution. You'd pick one of your distributions to be P and one to be Q, then calculate $-\sum_x P(X)\log{\frac{P(X)}{Q(X)}}$ where $x$ is each bin on the x-axis of the histogram. Be aware that this distance metric is directional, meaning you'll get a different answer if you swap P and Q. The reason is that the KL-divergence is a measure of information transfer (like, the information you gain by moving from Q to P), but I think that's irrelevant to your question.

|improve this answer|||||
$\endgroup$
0
$\begingroup$

Those plots you're showing are not histograms, they're just bar charts. In a histogram, the height of the bars is a count of how many data points fall in the bucket represented by that value on the X axis (e.g. how many people of that particular age or age group).

What you could do is plot a histogram of the men's ages (how many men of each age or age group) and compare it with a histogram of the women's ages. It's not clear what it would mean to compare a histogram of the men's ages with, say, a histogram of their wealth.

Computing the similarity between two histograms (or distributions) of the same variable can be done by adapting Jaccard similarity (see this paper for an example).

You might find a high level of similarity (say 0.9) for age distribution but a low similarity for wealth. The interesting ones are perhaps those where the histograms are dissimilar (things are very different for men and women).

|improve this answer|||||
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.