# My data is highly overlapping, but when I apply logistic regression, it is giving an impressive accuracy of 79%. Why?

Logistic regression is supposed to work well only on data that is linearly separable. As we can see in the pair plot, the data points heavily overlap. The logistic regression model is in fact showing better accuracy than Decision Tree, KNN, & Random Forest (Methods that are suitable for overlapping data). Even the cross validation score is higher for my logistic regression model.

Decision Tree, KNN, & Random Forest (Methods that are suitable for overlapping data)

This statement is false. All those methods are good when the decision surface (separating surface) has a highly nonlinear form. They act as a non-parametric local approximation - all parameters are not in fact parameters of the decision function but are meta parameters of the model. When the decision separator is linear then it is to be expected logistic regression to perform better.

To see if your data is linearly separable it is not enough to see pairwise scatter plots. Imagine to clouds of points, one nested inside another. If they are round, no matter how you project them in any pairwise axes they will look overlapped.

Your data is multidimensional, it is possible that any two dimensional projection overlaps while still existing an hyperplane on the original dimensionality that separates the two classes well

Say for instance you have 3 data points from 2 labels in 2d that are linearly separable X:(0,-1) O:(1,2) X:(4,3)

           X
O

X


In the x axis they look like. X. O. X In the y axis they look like. X. O. X They are not 1d-separable in any axis

Yet there's a line in 2d that separates them (y=x)

This gets easier to happen the higher you get(in dimensional space xD)