Return to Answer

One thing that you might be overlooking with your reasoning is the fact that increased predictors do not necessarily lead to a better model. In this case, the more predictors you have the higher the risk of collinearity that there is between them - thus increasing the utility of ridge regression.

In fact, a second point:

The larger the number of predictors that you have in your model the more useful techniques like ridge regression is. This is because with so many predictors is very difficult to determine if collinearity or other relationships exist between predictors. When you have a smaller model with say 5 predictors, you could verify this fairly easily. With 1,000 predictors this would be much more difficult.

In the case of the Lasso regression, the benefits are even more obvious. Since predictor coefficients values can be shrunk fully to zero this acts as a form of feature selection. Here, you are potentially removing predictors that capture redundant information.

Also - in answer to the second part of your question - OLS does not perform any form of feature selection. Therefore, it will not be able to pick out which parameters are the most important by adding more. The more parameters you add to your model the higher the variance gets. Why is this? You have more estimations to make and therefore collectively your model is less reliable. Consequently, your model will likely inflate the coefficients on the predictors giving undue "weight" or importance to certain predictors.

One thing that you might be overlooking with your reasoning is the fact that increased predictors does not necessarily lead to a better model. In this case, the more predictors you have the higher the risk of collinearity that there is between them - thus increasing the utility of ridge regression.

In fact, a second point:

The larger the number of predictors that you have in your model the more useful techniques like ridge regression are. This is because with so many predictors it is very difficult to determine if collinearity or other relationships exist between predictors. When you have a smaller model with say 5 predictors, you could verify this fairly easily. With 1,000 predictors this would be much more difficult.

In the case of the Lasso regression, the benefits are even more obvious. Since predictor coefficients values can be shrunk fully to zero this acts as a form of feature selection. Here, you are potentially removing predictors that capture redundant information.

Also - in answer to the second part of your question - OLS does not perform any form of feature selection. Therefore, it will not be able to pick out which parameters are the most important by adding more. The more parameters you add to your model the higher the variance gets. Why is this? You have more estimations to make and therefore collectively your model is less reliable. Consequently, your model will likely inflate the coefficients on the predictors giving undue "weight" or importance to certain predictors.

One thing that you might be overlooking with your reasoning is the fact that increased predictors does not necessarily lead to a better model. In this case, the more predictors you have the higher the risk of collinearity that there is between them - thus increasing the utility of ridge regression.

In fact, a second point:

The larger the number of predictors that you have in your model the more useful techniques like ridge regression are. This is because with so many predictors is is very difficult to determine if collinearity or other relationships exist between predictors. When you have a smaller model with say 5 predictors, you could verify this fairly easily. With 1,000 predictors this would be much more difficult.

In the case of the Lasso regression the benefits are even more obvious. Since predictor coefficients values can be shrunk fully to zero this acts as a form of feature selection. Here, you are potentially removing predictors that capture redundant information.

Also - in answer to the second part of your question - OLS does not perform any form of feature selection. Therefore, it will not be able to pick out which parameters are the most important by adding more. The more parameters you add to your model the higher the variance gets. Why is this? You have more estimations to make and therefore collectively your model is less reliable. Consequently, your model will likely inflate the coefficients on the predictors giving undue "weight" or importance to certain predictors.

One thing that you might be overlooking with your reasoning is the fact that increased predictors do not necessarily lead to a better model. In this case, the more predictors you have the higher the risk of collinearity that there is between them - thus increasing the utility of ridge regression.

In fact, a second point:

The larger the number of predictors that you have in your model the more useful techniques like ridge regression is. This is because with so many predictors is very difficult to determine if collinearity or other relationships exist between predictors. When you have a smaller model with say 5 predictors, you could verify this fairly easily. With 1,000 predictors this would be much more difficult.

In the case of the Lasso regression, the benefits are even more obvious. Since predictor coefficients values can be shrunk fully to zero this acts as a form of feature selection. Here, you are potentially removing predictors that capture redundant information.

Also - in answer to the second part of your question - OLS does not perform any form of feature selection. Therefore, it will not be able to pick out which parameters are the most important by adding more. The more parameters you add to your model the higher the variance gets. Why is this? You have more estimations to make and therefore collectively your model is less reliable. Consequently, your model will likely inflate the coefficients on the predictors giving undue "weight" or importance to certain predictors.

One thing that you might be overlooking with your reasoning is the fact that increased predictors do not necessarily lead to a better model. In this case, the more predictors you have the higher the risk of collinearity that there is between them - thus increasing the utility of ridge regression.

In fact, a second point:

The larger the number of predictors that you have in your model the more useful techniques like ridge regression is. This is because with so many predictors is very difficult to determine if collinearity or other relationships exist between predictors. When you have a smaller model with say 5 predictors, you could verify this fairly easily. With 1,000 predictors this would be much more difficult.

In the case of the Lasso regression, the benefits are even more obvious. Since predictor coefficients values can be shrunk fully to zero this acts as a form of feature selection. Here, you are potentially removing predictors that capture redundant information.

Also - in answer to the second part of your question - OLS does not perform any form of feature selection. Therefore, it will not be able to pick out which parameters are the most important by adding more. The more parameters you add to your model the higher the variance gets. Why is this? You have more estimations to make and therefore collectively your model is less reliable. Consequently, your model will likely inflate the coefficients on the predictors giving undue "weight" or importance to certain predictors.

One thing that you might be overlooking with your reasoning is the fact that increased predictors does not necessarily lead to a better model. In this case, the more predictors you have the higher the risk of collinearity that there is between them - thus increasing the utility of ridge regression.

In fact, a second point:

The larger the number of predictors that you have in your model the more useful techniques like ridge regression are. This is because with so many predictors is is very difficult to determine if collinearity or other relationships exist between predictors. When you have a smaller model with say 5 predictors, you could verify this fairly easily. With 1,000 predictors this would be much more difficult.

In the case of the Lasso regression the benefits are even more obvious. Since predictor coefficients values can be shrunk fully to zero this acts as a form of feature selection. Here, you are potentially removing predictors that capture redundant information.

Also - in answer to the second part of your question - OLS does not perform any form of feature selection. Therefore, it will not be able to pick out which parameters are the most important by adding more. The more parameters you add to your model the higher the variance gets. Why is this? You have more estimations to make and therefore collectively your model is less reliable. Consequently, your model will likely inflate the coefficients on the predictors giving undue "weight" or importance to certain predictors.