# Can we optimize regression problems that have categorical variables by encoding them if on the other hand we are inserting multicollinearity? [duplicate]

Can we optimize regression problems that have categorical variables by encoding them if, on the other hand, we are inserting multicollinearity?

## 1 Answer

Multicollinearity can be a problem if you choose to optimize linear regression with ordinary least squares (OLS). Because the data matrix $$X$$ can have less than full rank, therefore the moment matrix $$XᵀX$$ cannot be inverted.

If you choose to optimize linear regression with gradient descent, multicollinearity is not an issue in finding an optimal solution.

• "if... gradient descent...multicollinearity is not an issue" in that an optimal solution will be found. But there are infinitely many optimal solutions given perfect multicollinearity, and you don't get much control over which one you find. Also, statistical inferences / interpretations will fail. – Ben Reiniger Feb 19 at 21:12