I built a Linear model which has an adjusted r-squared value of 1. I understand that this is a near perfect number. Upon further investigation, I found that one of the 96 independent variables in the dataset is highly correlated with the dependent variable. This is also a variable which I would like to keep (and not drop). Are there any additional steps that I should undertake to handle this?

Sample df reproducing the situation above:

df1 <- data.frame("y" =   c(0.0166 , -0.2380 , -0.3192 , -0.2774 ,  9.3148 , 0.3142) , 
          "x1" =  c(0.0103 , -0.2347 ,  -0.3182 , -0.2793 ,  9.4638  , 0.3297) , 
          "x2" = c( -0.1838 , -0.2458 , -0.2581 ,-0.2533 , 6.7566 ,-0.0835) , 
          "x3" = c(0.3426 ,-0.0543 ,-0.4512 ,-0.0543, 10.4637 , 0.3426) ,
          "x4" = c(-0.161 , -0.270 ,-0.318, -0.280 , 8.279 , 0.169))
df1_lm <- lm(y~. , data = df1)
  • $\begingroup$ Question: is your „good“ x feature a linear combination of y? $\endgroup$ – Peter Jun 21 '19 at 19:59

You are lucky to have a good model, no need to handle this if there is no information leakage from y to your x1 variable.

Checking the data you provided you may want to eliminate variables that are not significant though.

  • $\begingroup$ Thanks Viktor . Btw , what is 'information leakage' that you are referring to ? and how can i check for it ? $\endgroup$ – DataScienceAspirant Jun 21 '19 at 17:13
  • $\begingroup$ I meant if there was some error during data collection or variable selection and somehow your y variable effected your x variable. I.e. you want to predict who will win the lottery (y) and you use the bank account balance (x1) to predict. If x1 was collected after the payment of the prize you will have probably a very good model even though in practice it will be useless. If I see such an almost deterministic model I always get suspicious and check this option. It shall depend on your domain but it can be normal too to have a good value. $\endgroup$ – Viktor Jun 21 '19 at 17:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.