1
$\begingroup$

enter image description here

Can anybody let me know the definition of these terms? I know we solve this for Beta but I want to have the definition

$\endgroup$
2
  • $\begingroup$ Please clarify your specific problem or provide additional details to highlight exactly what you need. As it's currently written, it's hard to tell exactly what you're asking. $\endgroup$
    – Community Bot
    Feb 7, 2022 at 15:00
  • $\begingroup$ Have a look at Introduction to Statistical Learning. Well explained there statlearning.com $\endgroup$
    – Peter
    Feb 12, 2022 at 20:59

1 Answer 1

1
$\begingroup$

The function you are trying to minimize when using ridge regression consists of two parts, a standard loss function that describes how well the model fits the data (in your example this is the mean-squared error) and a penalty term. The penalty term is simply the summed absolute values of the model's parameters multiplied by lambda, which is a hyperparameter of the ridge regression. This describes how much of an impact the penalty term has on the total loss, a large lambda means that the model will likely favor a simpler model with smaller (in value) parameters. The model therefore has to make a trade-off between the two, improving the fit with regards to the data while making sure to keep the values of the parameters relatively small to prevent the penalty term from becoming too big.

$\endgroup$
6
  • $\begingroup$ Thank you so much. so what is n exactly as a term definition? SECOND TERM AS YOU SAID IS SUM OF THE MODEL FEATURES BUT WHAT IS N EXACTLY IN TERM DEFINITION? WHAT DOES IT SHOW? $\endgroup$ Feb 7, 2022 at 13:49
  • $\begingroup$ when you talk about penalty term that becomes too big, do you refer to overfitting ? $\endgroup$ Feb 7, 2022 at 13:52
  • $\begingroup$ @AhmadTurani The n you see above the sum symbol simply means that the sum is taken over all of the parameters, i.e. parameter 1 to n. If the model has three parameters all three are squared and added together. $\endgroup$
    – Oxbowerce
    Feb 7, 2022 at 14:08
  • $\begingroup$ Yes yes @Oxbowerce. But what the n exactly is? for example parameters (data set numbers or ...)? $\endgroup$ Feb 7, 2022 at 14:09
  • $\begingroup$ @HelpNeederStudent Within this context I indeed had overfitting in mind, but I wouldn't say that a large penalty term always means that you are overfitting. There could be cases where the model can offset the large penalty term by a significant decrease of the MSE. $\endgroup$
    – Oxbowerce
    Feb 7, 2022 at 14:10

Not the answer you're looking for? Browse other questions tagged or ask your own question.