Interpreting the evaluation result of multiple linear regression

Question

I am learning the multiple linear regression model. I've built a model and using R command:

summary(model)

I got this result:

Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 253.2 on 44 degrees of freedom Multiple
R-squared:  0.3336, Adjusted R-squared:  0.2579  F-statistic: 4.405 on
5 and 44 DF,  p-value: 0.002444

How can I interpret this result in order to have a decision regarding the goodness of the model? In specific, what is the 44 degrees of freedom means for this case?

Also, why do we have adjusted and multiple r squared parameters?

Answer 1

I am going to answer your questions one after another.

First, what do the 44 degrees of freedom mean?

It simply means that the model you built is constructed by using 44 independent variables. For example a model that looks like y = ax + b has 1 independent variable (i.e. a) and thus 1 degree of freedom. A model that looks like y= ax1 + b*x2 + c would have 2 independent variables (i.e. a and b) and thus 2 degrees of freedom.

Second, what is the Multiple R-squared?

Here for the purpose of interpretating it Multiple R-squared is equivalent to the (simple) R-squared you would have for a linear regression model with 1 degree of freedom. Multiple R-squared tells us the share of the observed variance that is explained by the model. For example if you have a Multiple R-squared of 0.79 it means that your model explains 79% of the observed variance in your data.

Third, what is the Adjusted R-squared and why do we need it?

There are several problems with Multiple R-squared.

Problem 1: Every time you add a predictor to a model, the R-squared increases, even if due to chance alone. It never decreases. Consequently, a model with more independent variables (more degrees of freedom) may appear to have a better fit simply because it has more independent variables.

Problem 2: If a model has too many predictors and higher order polynomials, it begins to model the random noise in the data. This condition is known as overfitting the model and misleadingly high R-squared values and a lessened ability to make predictions.

Problem 1 is caused by Problem 2. And this is where Adjusted R-squared is coming in handy. Adjusted R-squared is an attempt at fixing these problems by factoring in the number of independent variables. The adjusted R-squared tells you the percentage of variation explained by only the independent variables that actually affect the dependent variable.

where:

• n is the number of data points you have,
• and k is the number of independent variables used to explain their distribution, excluding the constant

If you add more and more useless variables to a model, adjusted r-squared will decrease. If you add more useful variables, adjusted r-squared will increase. Adjusted R-squared will always be less than or equal to R-squared . You only need R-squared when working with samples. In other words, R-squared isn’t necessary when you have data from an entire population.

---

Here is an interesting series of articles which will help you understand how to use R-squared to interpret the results of your model even better.

• Regression Analysis: How Do I Interpret R-squared and Assess the Goodness-of-Fit?
• Multiple Regression Analysis: Use Adjusted R-Squared and Predicted R-Squared to Include the Correct Number of Variables

Answer 2

The details are just a help(summary.lm) in R. The meaning of the codes and the terms are as follows:

Considering R = Correlation Coefficient

1. The residual standard error is nothing more than the positive square root of the mean square error.
2. Regression degrees of freedom = number of independent variables (factors) in $y$ = $a_1x_1 + a_2x_2 + a_3x3 + ...$ Wikipedia link for a better in-depth explanation
3. R Square = $(Multiple R)^2$ = $R^2$ = $1 - (Residual SS / Total SS) = (Regress SS / Total SS)$
4. Adjusted R Square = $1 - (Total df / Residual df)(Residual SS / Total SS)$

A wonderful post on how to understand degrees of freedom.