I am reading data from a file using pandas which looks like this:


   ldr1  ldr2  servo
0   971   956     -2
1   691   825   -105
2   841   963    -26
3   970   731     44
4   755   939    -69

I proceed to normalize this data to perform gradient descent:

my_data = (my_data - my_data.mean())/my_data.std()

       ldr1      ldr2     servo
0  1.419949  1.289668  0.366482
1 -0.242834  0.591311 -1.580420
2  0.647943  1.326984 -0.087165
3  1.414011  0.090200  1.235972
4  0.137231  1.199041 -0.899949

I perform multivariate regression and end up with fitted parameters on the normalized data:

Thetas:  [[-3.86865143e-17,  8.47885685e-01, -5.39083511e-01]]

I would like to plot the plane of best fit on the original data and not the normalized data using the normalized thetas.

I used scipy.optimize.curve_fit to perform multivariate linear regression and come up with the optimal fitted parameters. I know that the original thetas should be close to the following:

[   0.26654135   -0.15218007 -107.79915373]

How can I get the 'original' thetas for the original data-set in order to plot, without using Scikit-Learn?

Any suggestions will be appreciated.


Coefficients of the linear regression for unnormalized features

If parameters in the normalized space are denoted as $(\theta_0', \theta_1', \theta_2')$, parameters in the original space $(\theta_0, \theta_1, \theta_2)$ can be derived as follows $$\begin{align*} y' &= \theta_2'x_2'+\theta_1'x_1'+\theta_0'\\ \frac{y-\mu_Y}{\sigma_Y} &= \theta_2'\frac{x_2 - \mu_{X_2}}{\sigma_{X_2}} + \theta_1'\frac{x_1 - \mu_{X_1}}{\sigma_{X_1}} +\theta_0' \\ y &= \overbrace{\left(\frac{\sigma_{Y}}{\sigma_{X_2}}\theta_2'\right)}^{\theta_2}x_2+ \overbrace{\left(\frac{\sigma_{Y}}{\sigma_{X_1}}\theta_1'\right)}^{\theta_1}x_1 + \overbrace{\sigma_Y\left(-\theta_2'\frac{\mu_{X_2}}{\sigma_{X_2}}-\theta_1'\frac{\mu_{X_1}}{\sigma_{X_1}} + \theta_0'\right) + \mu_Y}^{\theta_0} \end{align*}$$

Generalization to D features

$$\begin{align*} \theta_d = \left\{\begin{matrix} \sigma_Y \left(\theta_0' - \sum_{i=1}^{D}\theta_i'\frac{\mu_{X_i}}{\sigma_{X_i}} \right) + \mu_Y& d=0\\ \frac{\sigma_{Y}}{\sigma_{X_d}}\theta_d' & d > 0 \end{matrix}\right. \end{align*}$$

A trick

For visualization, we can plot the plane in original, un-normalized space without changing the parameters (Thetas). We only need to re-label (re-scale) the plot axes.

For example, a point $(x_1', x_2', y')$ in the normalized space corresponds to point $$(x_1, x_2, y)=(\sigma_{X_1}x_1'+\mu_{X_1}, \sigma_{X_2}x_2'+\mu_{X_2}, \sigma_{Y}y'+\mu_{Y})$$

in the original space. So you just need to rename the plot axes from $(x_1', x_2', y')$ to $(x_1, x_2, y)$.

Note that $y'=\theta_2'x_2'+\theta_1'x_1'+\theta_0'$ is still calculated using normalized features.

  • $\begingroup$ Thank you, this helps! I will try your suggestions and get back to you. Confirming that I understood correctly, in reference to your last equation, should I multiply the standard deviation of both normalized and original parameters and add to it the variance of the original parameter? $\endgroup$ – Rrz0 Apr 23 at 12:52
  • 1
    $\begingroup$ @Rrz0 There is no variance or std for parameters $\theta$. $\sigma$ and $\mu$ are calculated from original data. $\endgroup$ – Esmailian Apr 23 at 12:59
  • $\begingroup$ Right, thanks for confirming. I am still not able to visualize the proper plane. I managed to convert the normalized data into the original space, however the plane appears incorrectly. I edited the question. $\endgroup$ – Rrz0 Apr 23 at 13:06
  • 1
    $\begingroup$ @Rrz0 you should plot (x_scaled, y_scaled, z_scaled), also check out the manual on plot_surface maybe that is the problem, see whether you can plot an arbitrary surface. I suggest also implementing the first approach to better find the problem. $\endgroup$ – Esmailian Apr 23 at 14:02
  • 1
    $\begingroup$ @Rrz0 remove $x_2$, it should not be included $\endgroup$ – Esmailian Apr 23 at 14:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.