# How to obtain original coefficients after performing linear regression on normalized data?

I am reading data from a file using pandas which looks like this:

data.head()

ldr1  ldr2  servo
0   971   956     -2
1   691   825   -105
2   841   963    -26
3   970   731     44
4   755   939    -69


I proceed to normalize this data to perform gradient descent:

my_data = (my_data - my_data.mean())/my_data.std()

ldr1      ldr2     servo
0  1.419949  1.289668  0.366482
1 -0.242834  0.591311 -1.580420
2  0.647943  1.326984 -0.087165
3  1.414011  0.090200  1.235972
4  0.137231  1.199041 -0.899949


I perform multivariate regression and end up with fitted parameters on the normalized data:

Thetas:  [[-3.86865143e-17,  8.47885685e-01, -5.39083511e-01]]


I would like to plot the plane of best fit on the original data and not the normalized data using the normalized thetas.

I used scipy.optimize.curve_fit to perform multivariate linear regression and come up with the optimal fitted parameters. I know that the original thetas should be close to the following:

[   0.26654135   -0.15218007 -107.79915373]


How can I get the 'original' thetas for the original data-set in order to plot, without using Scikit-Learn?

Any suggestions will be appreciated.

### Coefficients of the linear regression for unnormalized features

If parameters in the normalized space are denoted as $$(\theta_0', \theta_1', \theta_2')$$, parameters in the original space $$(\theta_0, \theta_1, \theta_2)$$ can be derived as follows \begin{align*} y' &= \theta_2'x_2'+\theta_1'x_1'+\theta_0'\\ \frac{y-\mu_Y}{\sigma_Y} &= \theta_2'\frac{x_2 - \mu_{X_2}}{\sigma_{X_2}} + \theta_1'\frac{x_1 - \mu_{X_1}}{\sigma_{X_1}} +\theta_0' \\ y &= \overbrace{\left(\frac{\sigma_{Y}}{\sigma_{X_2}}\theta_2'\right)}^{\theta_2}x_2+ \overbrace{\left(\frac{\sigma_{Y}}{\sigma_{X_1}}\theta_1'\right)}^{\theta_1}x_1 + \overbrace{\sigma_Y\left(-\theta_2'\frac{\mu_{X_2}}{\sigma_{X_2}}-\theta_1'\frac{\mu_{X_1}}{\sigma_{X_1}} + \theta_0'\right) + \mu_Y}^{\theta_0} \end{align*}

### Generalization to D features

\begin{align*} \theta_d = \left\{\begin{matrix} \sigma_Y \left(\theta_0' - \sum_{i=1}^{D}\theta_i'\frac{\mu_{X_i}}{\sigma_{X_i}} \right) + \mu_Y& d=0\\ \frac{\sigma_{Y}}{\sigma_{X_d}}\theta_d' & d > 0 \end{matrix}\right. \end{align*}

### A trick

For visualization, we can plot the plane in original, un-normalized space without changing the parameters (Thetas). We only need to re-label (re-scale) the plot axes.

For example, a point $$(x_1', x_2', y')$$ in the normalized space corresponds to point $$(x_1, x_2, y)=(\sigma_{X_1}x_1'+\mu_{X_1}, \sigma_{X_2}x_2'+\mu_{X_2}, \sigma_{Y}y'+\mu_{Y})$$

in the original space. So you just need to rename the plot axes from $$(x_1', x_2', y')$$ to $$(x_1, x_2, y)$$.

Note that $$y'=\theta_2'x_2'+\theta_1'x_1'+\theta_0'$$ is still calculated using normalized features.

• Thank you, this helps! I will try your suggestions and get back to you. Confirming that I understood correctly, in reference to your last equation, should I multiply the standard deviation of both normalized and original parameters and add to it the variance of the original parameter? – rrz0 Apr 23 '19 at 12:52
• @Rrz0 There is no variance or std for parameters $\theta$. $\sigma$ and $\mu$ are calculated from original data. – Esmailian Apr 23 '19 at 12:59
• Right, thanks for confirming. I am still not able to visualize the proper plane. I managed to convert the normalized data into the original space, however the plane appears incorrectly. I edited the question. – rrz0 Apr 23 '19 at 13:06
• @Rrz0 you should plot (x_scaled, y_scaled, z_scaled), also check out the manual on plot_surface maybe that is the problem, see whether you can plot an arbitrary surface. I suggest also implementing the first approach to better find the problem. – Esmailian Apr 23 '19 at 14:02
• @Rrz0 remove $x_2$, it should not be included – Esmailian Apr 23 '19 at 14:48