4
$\begingroup$

I am learning the multiple linear regression model. I've built a model and using R command:

summary(model)

I got this result:

Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 253.2 on 44 degrees of freedom Multiple
R-squared:  0.3336, Adjusted R-squared:  0.2579  F-statistic: 4.405 on
5 and 44 DF,  p-value: 0.002444

How can I interpret this result in order to have a decision regarding the goodness of the model? In specific, what is the 44 degrees of freedom means for this case?

Also, why do we have adjusted and multiple r squared parameters?

$\endgroup$
  • $\begingroup$ you need to give input on how did you do it and what are the number of data elements to help others offer help. $\endgroup$ – Espanta Jul 13 '15 at 1:48
3
$\begingroup$

I am going to answer your questions one after another.

First, what do the 44 degrees of freedom mean?

It simply means that the model you built is constructed by using 44 independent variables. For example a model that looks like y = ax + b has 1 independent variable (i.e. a) and thus 1 degree of freedom. A model that looks like y= ax1 + b*x2 + c would have 2 independent variables (i.e. a and b) and thus 2 degrees of freedom.

Second, what is the Multiple R-squared?

Here for the purpose of interpretating it Multiple R-squared is equivalent to the (simple) R-squared you would have for a linear regression model with 1 degree of freedom. Multiple R-squared tells us the share of the observed variance that is explained by the model. For example if you have a Multiple R-squared of 0.79 it means that your model explains 79% of the observed variance in your data.

Third, what is the Adjusted R-squared and why do we need it?

There are several problems with Multiple R-squared.

Problem 1: Every time you add a predictor to a model, the R-squared increases, even if due to chance alone. It never decreases. Consequently, a model with more independent variables (more degrees of freedom) may appear to have a better fit simply because it has more independent variables.

Problem 2: If a model has too many predictors and higher order polynomials, it begins to model the random noise in the data. This condition is known as overfitting the model and misleadingly high R-squared values and a lessened ability to make predictions.

Problem 1 is caused by Problem 2. And this is where Adjusted R-squared is coming in handy. Adjusted R-squared is an attempt at fixing these problems by factoring in the number of independent variables. The adjusted R-squared tells you the percentage of variation explained by only the independent variables that actually affect the dependent variable.

enter image description here

where:

  • n is the number of data points you have,
  • and k is the number of independent variables used to explain their distribution, excluding the constant

If you add more and more useless variables to a model, adjusted r-squared will decrease. If you add more useful variables, adjusted r-squared will increase. Adjusted R-squared will always be less than or equal to R-squared . You only need R-squared when working with samples. In other words, R-squared isn’t necessary when you have data from an entire population.


Here is an interesting series of articles which will help you understand how to use R-squared to interpret the results of your model even better.

$\endgroup$
2
$\begingroup$

The details are just a help(summary.lm) in R. The meaning of the codes and the terms are as follows:

Considering R = Correlation Coefficient

  1. The residual standard error is nothing more than the positive square root of the mean square error.
  2. Regression degrees of freedom = number of independent variables (factors) in $y$ = $a_1x_1 + a_2x_2 + a_3x3 + ...$ Wikipedia link for a better in-depth explanation
  3. R Square = $(Multiple R)^2$ = $R^2$ = $1 - (Residual SS / Total SS) = (Regress SS / Total SS)$
  4. Adjusted R Square = $1 - (Total df / Residual df)(Residual SS / Total SS)$

A wonderful post on how to understand degrees of freedom.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.