Can a linear regression model with a single input variable x with x not significant but the model has a high R2?
Can this exist? If yes, what are the reasons?
Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.Sign up to join this community
This can happen, but only with very small amounts of data. An example (in R) with three data points:
x <- c(0,1,2) y <- c(-0.1,0.7,1.2) summary(lm(y ~ x)) Coefficients: Estimate Std. Error t value Pr(>|t|) (Intercept) -0.0500 0.1118 -0.447 0.7323 x 0.6500 0.0866 7.506 0.0843 . --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 Residual standard error: 0.1225 on 1 degrees of freedom Multiple R-squared: 0.9826, Adjusted R-squared: 0.9651 F-statistic: 56.33 on 1 and 1 DF, p-value: 0.08432
Notice the R-squared is very high, but the
x parameter is not 95%-significant.
With lots of data, one cannot have a very high R-squared parameter without the corresponding parameter being significant.