2
$\begingroup$

I'm relatively new to DS, so forgive me if this is a dumb question or in the wrong forum

When evaluating features it seems that almost everywhere a correlation matrix is used [df.corr(), cor(df, method="pearson")].

The way I understand it is that a correlation matrix describes the stregnth and directionality of the linear relationship (strong negative through strong positive) between each feature/predictor and all others.

HOWEVER

If $R^2$ indicates the amount of variability explained by the linear relationship, between each feature/predictor (as a proportion), wouldn't that provide more information for model selection or feature engineering?

THEREFORE

  1. Would it make sense to always square a correlation matrix to get the $R^2$ values without reviewing the correlation matrix?

  2. How relevant to model selection or feature engineering is understanding if there's an postive or negative correlation among features?

$\endgroup$
2
  • 2
    $\begingroup$ You get no additional information by doing this, but you do lose the information about the sign of the correlation. Any rule you use in terms of $R^2$ can be phrased in terms of the correlation, but no rule involving the sign of $R$ can be phrased in terms of $R^2$ (except for something silly like $sign(R) \times R^2$. // "Squaring" a matrix $S$ typically means $S \times S$, not squaring every element of $S$. This might coincide (e.g., identity matrix, zero matrix), depending on the matrix elements, but they do not have to. $\endgroup$
    – Dave
    Jun 29 at 17:20
  • $\begingroup$ Thanks for the feedback!! $\endgroup$
    – Donovin
    Jun 30 at 1:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.