1
$\begingroup$

I am quite new to data science. I am trying to use Logistic Regression to predict my target (either 1 or 0). But the problem is when I use a heatmap to find the correlation between the columns and the target variable, the highest value I get is around 0.17 (out of 1). So it appears to me that my target variable doesn't correlate with any of the columns in the dataset. My question is, is it normal to have a such target variable? And what can I do to increase correlation between the target variable and other columns?

$\endgroup$
6
  • $\begingroup$ You can use the parameter class_weight="balanced" to adapt your classifier to an unbalanced dataset $\endgroup$
    – Adept
    Jul 22, 2021 at 9:04
  • $\begingroup$ But my problem is the correlation. The target variable hardly correlates with any of the columns. How does it affect to the accuracy of the model? $\endgroup$
    – section117
    Jul 22, 2021 at 9:50
  • 1
    $\begingroup$ Sometimes it's possible that there's no individual feature with a high correlation with the target, but the combination of multiple features can be correlated (that's the whole point of supervised learning btw: exploiting all the features in order to predict the target). That being said, do you have any indication that the task you're trying to achieve is doable? Because if the features have no relation to the target, then obviously it's not going to work. $\endgroup$
    – Erwan
    Jul 22, 2021 at 10:26
  • $\begingroup$ There's no way to "increase correlation", that doesn't make sense: either a feature is a good indicator or it isn't. For example age can be a good indicator for predicting if somebody has Alzheimer disease, but it cannot be a good indicator to predict somebody's favourite colour. ML is not magic, it needs relevant indicators to work. $\endgroup$
    – Erwan
    Jul 22, 2021 at 10:31
  • $\begingroup$ Your target is binary. Are you using tetrachoric correlations? Because if your other variables are continuous, Pearson correlations and heatmaps based on it ar inappropriate. $\endgroup$
    – Aolon
    Jul 22, 2021 at 12:19

1 Answer 1

3
$\begingroup$

If your predictors have nothing to do with the outcome, you should not be able to build a model that works out-of-sample. This is a feature, not a bug, of machine learning. For instance, do you consider what time I set my alarm in the morning to be predictive whether or not you have cereal for breakfast?

Features can, however, have just a small relationship with the outcome and combine to be quite predictive. Perhaps my alarm does not influence your breakfast choice, but there are a number of factors that do, each of which might be poor at predicting the outcome, but the combination of $3$ or $10$ might be very predictive. At an extreme, consider individual pixels of the MNIST digits. Does the middle pixel, on its own, have much ability to distinguish between the digits? What about some other pixel? Every individual pixel is a poor predictor of the digit, yet all $784$ combined result in strong performance.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.