3
$\begingroup$

I have just run a Linear regression model on the Dataset having 7 independent variable and 1 target variable. Below is the R squared and MSE values.

  • Mean squared error for training set : 36530921.0123
  • $R^2$ value for training set : 0.7477

Can anybody please give me some tips to increase the efficiency of this model.

Edit: I have just implemented the same problem using Linear regression with Normalization of the features. I got the below output: Mean squared error for training set : 5.468490570335696e-10 R2 value for training set : 0.9275088299658416 Mean squared error for training set : 4.111793316375822e-10 R2 value for training set : 0.9342888671422529

So can we consider normalizing the dataset to get better accuracy ?

|improve this question
$\endgroup$
  • $\begingroup$ Did you perform Predictor Importance test? if yes update the question with those results? Did you perform outlier removal? $\endgroup$ – Toros91 Apr 18 '18 at 7:09
  • 1
    $\begingroup$ First check for correlation between independent variables. I would exclude those whose correlation comes out to be >= 0.8 Again this number (0.8) depends on your task, it is not rule of thumb. Also, use feature scaling if feature ranges are different. $\endgroup$ – Ankit Seth Apr 18 '18 at 7:34
  • $\begingroup$ Checkout the feature Importance of the features also via the abs coefficients and the softmax $\endgroup$ – Aditya Apr 18 '18 at 8:04
  • $\begingroup$ Try rescaling your dataset and donadd a snapshot of your dataset $\endgroup$ – Aditya Apr 18 '18 at 12:49
  • $\begingroup$ @Toros91, I have not performed the Predictor Importance test.. Can you please give me a useful link how to do it. Outlier removal - Yes, I have done the outlier removal. $\endgroup$ – deepguy Apr 18 '18 at 14:36
4
$\begingroup$

You can build more complex models to try to capture the remaining variance. Here are several options:

  • Add interaction terms to model how two or more independent variables together impact the target variable
  • Add polynomial terms to model the nonlinear relationship between an independent variable and the target variable

  • Add spines to approximate piecewise linear models

  • Fit isotonic regression to remove any assumption of the target function form

  • Fit non-parametric models, such as MARS
|improve this answer
$\endgroup$
  • $\begingroup$ Dr. Brian Spiering Thanks for the inputs. These options Look interesting.. I will try these methods and update the outcome. $\endgroup$ – deepguy Apr 18 '18 at 17:57
1
$\begingroup$

Multicollinearity could be a reason for poor perfomance when using Linear Regression Models. Multicollinearity refers to a situation where a number of independent variables in a Linear Regression model are closely correlated to one another and it can lead to skewed results. In general, multicollinearity can lead to wider confidence intervals and less reliable probability values for the independent variables. Also maybe other assumptions of Linear Regrresion do not hold. Linear regression needs the relationship between the independent and dependent variables to be linear. It is also important to check for outliers since linear regression is sensitive to outlier effects. The linearity assumption can best be tested with scatter plots. Linear regression analysis requires that there is little or no autocorrelation in the data. Autocorrelation occurs when the residuals are not independent from each other.

|improve this answer
$\endgroup$
  • $\begingroup$ Thanks for the answer @Christos Karatsalos, however I have checked the correlation between the independent variables, and found no significant correlations. $\endgroup$ – deepguy Apr 18 '18 at 19:16
  • $\begingroup$ Check if any other assumptions of Linear Regression do not hold for the case you are working. $\endgroup$ – Christos Karatsalos Apr 18 '18 at 21:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.