1
$\begingroup$

I'm relatively new to data science and I am struggling to understand why the Mahalanobis distance (or any other distance measure) applied to an imbalanced data-set becomes inaccurate. I have a data set that consists of three classes A, B and C. There are 100 observations for class A, 60 for class B, and 20 for class C. When I calculate the Mahalanobis distance between each class, the results do not appear consistent with my PCA (principal component analysis) plot. In the PCA plot, class C is the most separate class; however, the Mahalanobis distance does not reflect this.

For balanced data sets, i.e., where classes A, B and C have the same number of observations, this has never been an issue. The Mahalanobis distance has always quite accurately reflected the results of PCA for balanced data.

I have read some similar questions and answers on here about why imbalanced data must be handled carefully for classification algorithms, but is this the same for distance measures? From what I can tell the Mahalanobis distance doesn't explicitly depend on sample size.

Therefore, I ask why does this measure lose reliability for imbalanced data?

Improve this question
$\endgroup$
2
  • $\begingroup$ could you please elaborate on: "When I calculate the Mahalanobis distance between each class"? Like previously stated Mahalanobis Distance is the distance between a point and a distribution . How are you using it to measure the distance between two classes? $\endgroup$
    – Gaurav Chawla
    Commented Mar 17, 2021 at 20:20
  • $\begingroup$ Hi, I believe it is possible to define the Mahalanobis distance as a dissimilarity/distance measure between two means, X and Y, of distributions. In my case, this is in 2D scores space. X would be the mean of PCA scores for one class, Y the other. Using the distance between two means gives a measure for the distance between two distributions. This literature, in particular the methods section, explains it very well: ncbi.nlm.nih.gov/pmc/articles/PMC3534867 $\endgroup$
    – Jimonty
    Commented Mar 17, 2021 at 22:57

1 Answer 1

Reset to default
1
$\begingroup$

Mahalanobis distance is defined as a distance between a point and a distribution. The key is how you define the distribution and I would say the imbalance of classses is not the problem in itself here. What could be a problem is mahalanobis is sensitive to initialization and the sample sizes of your classes are not that big. You could check the covariance matrix if it is reasonable for you task.

About more general question -- the distance between two points obviously does not depend on size of the classes. If we talk about distance between a point and a set, then it could affect the result: i.e. you defined the distance as the distance to the closest point in set, then obviously the more points, the bigger chance to get the closer point.

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ Thank you for the answer you have provided. Very helpful. Thinking about what you have said it makes sense to consider the covariance matrix. I believe this will be what causes the issues with small data sets, because the sample covariance is used to calculate the distance. And the covariance becomes more accurate with more data, like the mean and variance does, I think. Perhaps heavily imbalanced data leads to such inaccuracies as well. $\endgroup$
    – Jimonty
    Commented Mar 17, 2021 at 14:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.