# Which features can help to differentiate these two density?

I'm wondering that is there any features that can help in differentiating the following two images. I mean differentiating in related numbers.

• What do you mean by "differentiating in related numbers"; the model parameters? If so, do you know what family they come from, e.g., GMM?
– Emre
Aug 6 '16 at 19:30
• Thank you so much. No, I do not have any assumption and knowldge about distribution family. These are density plot of two vectors. as we see there is difference in these densities. Now, I mean be able to compute a statistical feature(value) for these 2 vectors in which helps us in relatively clustering two vectors points (2 clusters). Features like mean, mode, median, skewness (I have tested these but don't yield desired result) Aug 6 '16 at 20:10

1. Regarding two probability distributions $A,B$ on the line, that have densities, the following are equivalent:
• $A = B$ (i.e.: given any Borel set $\mathscr S$, the probability assign to $\mathscr S$ by A equals the probability assign to $\mathscr S$ by B)
• $A(-\infty,a] = B(-\infty,a]$, for any $a \in \mathbb R$, i.e.: $A,B$ have the same cumulative distribution function
• $A,B$ have the same densities.
In the case of your question, the two densities are clearly different, so the two probabilities are different.

From your question, it seems you want something you can measure (perhaps on sample from these distributions). There are infinitely many quantities that you can measure that when apply to $A$ and $B$ produce different results. The statement above gives some hints for a possible solution.

From your graphs, calling $A$ the first pdf, and $B$ the second: $$F_A(0.35) = A(-\infty, 0.35] = \text{ probability that an observation from the first distribution is < 0.35 } = 0$$ since the density is 0 to the left of 0.35. On the other hand, $$F_B(0.35) = B(-\infty, 0.35] = \text{ probability that an observation from the second distribution is < 0.35 } > 0$$ since the density is positive from -0.2 to 0.35.

We can a little to the right of 0.35 and find a number $a_0$ such that $0 < F_A(a_0) < F_B(a_0)$.

Being less than $a_0$ is something one can measure on samples.

Let $A_1, .., A_n$ be an iid sample from the first distribution, and let $B_1,...,B_m$ be an iid sample from the second distribution. Then $x_j = I(A_j \le a_0)$ is a measure (Note: I am using the notation $I(S)$ = indicator function of a set $S$, or, in more typical notation in the context of probabilitie, $I(S)$ = indicator function of event $S$ = 1 if $S$ is true, 0 if $S$ is false).

Likewise, one can measure $y_j = I(B_j \le a_0).$

$\{x_j\}$ are iid Bernoulli, with $P(x_j=1) = P(A_j \le a_0) = F_A(a_0).$
$\frac{1}{n} \sum_1^n x_j$ is an estimator of $F_A(a_0)$

$\{y_j\}$ are iid Bernoulli, with $P(y_j=1) = P(B_j \le a_0) = F_B(a_0).$
$\frac{1}{m} \sum_1^m y_j$ is an estimator of $F_B(a_0)$

Now one can run a rest of the hyposesis that the tw means are the same.

Before proceeding, does this construction answer your question?

• Thank you so much Victor. I think that you have mentioned completely the right point as it sounds logical based on the figures. But if possible, please let me know how can I implement it in Matlab for a vector as I'm not so good at statistic. Aug 7 '16 at 7:53
• Sorry, my Matlab skills are not any good. :( Aug 7 '16 at 15:15
• Any other language is welcomed. I mean the technical approach. Aug 7 '16 at 17:26
1. There must be some moments that distinguish these two, so keep on looking at higher powers.

2. Kolmogorov-Smirnov statistic should work in this case.

• Thank you. But Kolmogorov-Smirnov test just say that these are from the same family or not. As you mentioned I want to extract feature like moment. I have tested all moments but haven't get desired result. Aug 6 '16 at 23:25
• Sorry Mostafa. KS applies to convergence of the empirical cumulative distribution function and a test for its limit. It doesn't apply to your problem. I'll write some points as answer. Aug 7 '16 at 0:35