1
$\begingroup$

Is only a subset of algorithms are affected by the multicollinearity problem or all the machine learning algorithms? What is the solution for this?

$\endgroup$
4

1 Answer 1

0
$\begingroup$

Only a subset of algorithms are affected by multicollinearity.

Algorithms from the general linear model (GLM) family are the ones most likely to have issues with multicollinearity. It is because of the structure of those models:

$$ Y = XB + U$$

Other machine learning algorithms have other structures and do not have similar issues with multicollinearity. For example, tree-based algorithms automatically do feature selection during model fitting and will only choose one of the multicollinear features at a time.

The most common solutions for multicollinearity are:

  • Choose an algorithm robust to multicollinearity
  • Test for multicollinearity, then choose an algorithm based on the result
  • Ignore multicollinearity, then choose not to do any use cases where multicollinearity is an issue (i.e., statistical inference)
$\endgroup$
1
  • $\begingroup$ Thanks for the reply, Brian $\endgroup$ Aug 7 at 9:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.