Is it possible to explain why Lasso models eliminated certain coefficient?

Question

Is it possible to understand why Lasso models eliminated specific coefficients?. During the modelling, many of the highly correlated features in data is being eliminated by Lasso regression. Is it possible why precisely these features are being eliminated from the model? (Is it the presence of any other features/multicollinearity etc.? I want to explain the lasso model behaviour. Your help is highly appreciated.

Answer 1

Have a look at "Introduction to Statistical Learning" (Chapter 6.2.2). The Lasso adds an aditional penalty term to the original OLS penalty. In addition to the residual sum of squares (RSS, which is minimized in OLS), there is an additional term, the shrinkage penalty. Basically, coefficients which do not contribute much to a "good fit" are shrunken. Actually there is a set of $\beta_j$ for each $\lambda$ (the tuning parameter).

Not knowing your data, I can only guess that many of your highly correlated features "do not contribute much" to a "good fit" in your model. This may be a consequence from multicollinearity. However, the Lasso does not per se "shrink" features with high correlation. It rather looks for a subset of coefficients $\beta$ which produce a "good fit".

You may check whether multicollinearity is a severe issue in your data, e.g. by looking at the Variance Inflation Factor (VIF) and remove possible features with (very) high correlation.