0
$\begingroup$

I have implemented a Linear regression model on a dataset of 7 independent variable and 1 target with the below 2 approaches

1) Without normalization of the data, resulted in a Mean squared error of 36530921.0123 and R2 value 0.7477

2) With normalization of the data, resulted in a Mean squared error of 5.468490570335696e-10 and R2 value 0.92750882996584. But in this case even the Target variable is also normalized.

Which is the better approach adding new features to the 1st case or considering the 2nd case and it has a better R2 and MSE value.

If we are considering the 2nd case, what is the steps need to be taken to denormalize the target variable.

Thank you.

$\endgroup$
  • $\begingroup$ Why have you normalized the target variable? Normalization is done for features (independent variables) only. $\endgroup$ – Ankit Seth Apr 24 '18 at 9:23
  • $\begingroup$ @AnkitSeth My bad. I will try without normalizing the target variables and post the updates. $\endgroup$ – deepguy Apr 24 '18 at 11:58
  • $\begingroup$ @AnkitSeth I have tried the same dataset by normalising only independent variables, Below is the accuracy I have got. Mean squared error : 55818887.351718076 R2 value : 0.6299156105971107. $\endgroup$ – deepguy Apr 30 '18 at 11:06
  • $\begingroup$ Can you post the part of code where you are normalizing data and implementing linear model? With normalization there can be improvement in error, but it should not increase. $\endgroup$ – Ankit Seth May 1 '18 at 12:26
1
$\begingroup$

Unless you normalize the MSE in scenario 1 (or denormalize the MSE in scenario 2), comparing two MSE with two different scales is irrelevant.

You can have data with values varying from 10 to 30 millions, centered then normalized to -1/+1. Suppose you have a MSE of 1000 in the first case, and 0.1 in the second, you will easily see that the second MSE is way more impacting than the first one after (de)normalization.

That said, if you want to retrieve the target from scenario 2, you need to apply the reverse operations to what has been done to get the "normalized" target. Assuming for instance that you centered / reduced the target.

$$ Z = \frac{Y-\bar{Y}}{\bar{Y}} $$ $$ Z = \alpha X_1 + \beta X_2 + \gamma X_3 $$

Where $Y$ is your initial target, $\bar{Y}$ its average, $Z$ your normalized target and $X_i$ your predictors.

When you apply your model and get a prediction, say $\hat{z}$, you can calculate its corresponding value $\hat{y}$ after applying the reverse transformations to your normalization.

$$ \hat{z} = \alpha x_1 + \beta x_2 + \gamma x_3 $$ $$ \frac{\hat{y}-\bar{y}}{\bar{y}} = \alpha x_1 + \beta x_2 + \gamma x_3 $$ $$ \hat{y} = \bar{y} *(\alpha x_1 + \beta x_2 + \gamma x_3 ) + \bar{y} $$

Now in this particular case, as Ankit Seth pointed out, you need not normalizing your target. The linear model would have adjusted its coefficients automatically, because you have been using linear operands.

However, if you proceed to non linear operations for target normalization, the same logic should apply. Say for instance that your model predicts the logarithm of your target.

$$ log(Y) = \alpha X_1 + \beta X_2 + \gamma X_3 $$ The reverse operation would be to apply its reciprocal function, e.g. exponential.

$$ Y = exp(\alpha X_1 + \beta X_2 + \gamma X_3) $$

|improve this answer|||||
$\endgroup$
  • 1
    $\begingroup$ Thanks @AshOfFire. Apologies. Now I will try with only normalizing independent variables and post the outcomes. $\endgroup$ – deepguy Apr 24 '18 at 11:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.