I have two models: one a simple linear regression and the other a multi linear regression.

The strongest relationship is between the two variables in the simple linear regression. The multi linear regression includes two extra variables that can be connected with the dependent, however, statistically they are insignificant (P > 0).

How do I interpret whether the best model to use is the simple linear regression or the multi-linear regression?

Here are some values from the regression:

Simple Linear : F(1, 77) = 21.07 , Prob > F = 0.0000 , Rsquared = 0.2148, Adj Rsquared = 0.2046

Multi Linear: F(3, 75) = 7.29 , Prob > F = 0.0002, Rsquared = 0.2258, Adj Rsquared = 0.1948


Assuming that the "best" model is the model that makes the most accurate out-of-training sample predictions, there are two types of measures that you can use. R squared will only increase as you add more variables hence it is not a suitable measure.

There are two common approaches:

(1) Indirectly estimate test error by making an adjustment to the training error (e.g. Adjusted Rsquared / BIC/ AIC). In your case, the simple linear regression performed slightly better in terms of Adj Rsquared.

(2) Directly estimate the test error using either a validation set or cross-validation.


Most software packages allow you to perform a joint wald test of whether the additional parameters are significant. In other words, you estimate the more complex model and test whether you can reduce it to the simpler model by setting some parameters to zero.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.