1
$\begingroup$

Can we optimize regression problems that have categorical variables by encoding them if, on the other hand, we are inserting multicollinearity?

$\endgroup$
2
$\begingroup$

Multicollinearity can be a problem if you choose to optimize linear regression with ordinary least squares (OLS). Because the data matrix $X$ can have less than full rank, therefore the moment matrix $XᵀX$ cannot be inverted.

If you choose to optimize linear regression with gradient descent, multicollinearity is not an issue in finding an optimal solution.

| improve this answer | |
$\endgroup$
  • $\begingroup$ "if... gradient descent...multicollinearity is not an issue" in that an optimal solution will be found. But there are infinitely many optimal solutions given perfect multicollinearity, and you don't get much control over which one you find. Also, statistical inferences / interpretations will fail. $\endgroup$ – Ben Reiniger Feb 19 at 21:12

Not the answer you're looking for? Browse other questions tagged or ask your own question.